-----
DKIM KEY RETRIEVAL

When borrowing a signature, borrow the key as well.


-----
ENVELOPE_SENDER

Save that domain and add it to aggregate reports.


-----
MULTIPLE FROM:

On special occasions, a message has multiple authors.  That is, a From:
header with multiple mailboxes.  Thunderbird allows it but doesn't set
Sender: in that case.  DMARC doesn't allow it.  What can a filter do?

* On signing, set Sender to the first mailbox of From:.

* On verifying, check if Sender: domain exists and matches one of the
  From: mailboxes.  Probably the first mailbox has most visibility, not
  clear if Sender: should match that.


In any case, multiple From: fields are forbidden.  Check libopendkim
function dkim_headercheck().  There should be an A-R method for that.


-----
ZAGGREGATE QUERY

In order to report a variable number of signatures, must run a query for
every line of the reported table.  The reported "table" (once converted)
indeed has a variable number of columns.


Various attempts at having the job carried out by DBMS:


Alternative query with no max-dkim-sigs:
https://dba.stackexchange.com/questions/289374/can-cte-simplify-repeated-and-possibly-recursive-joins/
https://dbfiddle.uk/?rdbms=mariadb_10.5&fiddle=3a8a2af688f5d32e2e0e2e5b74a7903a


A query like:

SELECT INET_NTOA(CONV(HEX(m.ip),16,10)) AS source,
  COUNT(DISTINCT(m.id)) AS n,
  m.dmarc_dispo AS disposition,
  m.dmarc_dkim AS d_dkim,
  m.dmarc_spf AS d_spf,
  m.dmarc_reason AS reason,
  da.domain AS author,
  dspf.domain AS spf,
  rspf.spf AS spf_re,
  dhelo.domain AS helo,
  rhelo.spf AS helo_re,
  GROUP_CONCAT(DISTINCT CONCAT(ddk.domain, '=', rdk.dkim_selector, '=', rdk.dkim, '=', rdk.dkim_trans)) AS dkim_all
FROM message_in AS m
  LEFT JOIN (msg_ref AS rd INNER JOIN domain AS dd ON rd.domain = dd.id)
    ON m.id = rd.message_in AND FIND_IN_SET('dmarc', rd.auth)
  LEFT JOIN (msg_ref AS ra INNER JOIN domain AS da ON ra.domain = da.id)
    ON m.id = ra.message_in AND FIND_IN_SET('author', ra.auth)
  LEFT JOIN (msg_ref AS rspf INNER JOIN domain AS dspf ON rspf.domain = dspf.id)
    ON m.id = rspf.message_in AND FIND_IN_SET('spf', rspf.auth)
  LEFT JOIN (msg_ref AS rhelo INNER JOIN domain AS dhelo ON rhelo.domain = dhelo.id)
    ON m.id = rhelo.message_in AND FIND_IN_SET('spf_helo', rhelo.auth)
  LEFT JOIN (msg_ref AS rdk INNER JOIN domain AS ddk ON rdk.domain = ddk.id)
    ON m.id = rdk.message_in AND rdk.dkim_order > 0
WHERE rd.domain = $(domain_ref) AND $(period_start) <= m.mtime AND m.mtime < $(period_end)
GROUP BY source, disposition, d_dkim, d_spf, reason, author,  spf, spf_re, helo, helo_re, dkim_all;


Says cannot group by dkim_all.  OTOH, not grouping by dkim_all reports all dkim in one record.



Current query:
SELECT INET_NTOA(CONV(HEX(m.ip),16,10)) AS source, COUNT(DISTINCT(m.id)) AS n,\
m.dmarc_dispo AS disposition, m.dmarc_dkim AS d_dkim, m.dmarc_spf AS d_spf,\
m.dmarc_reason AS reason, da.domain AS author,\
dspf.domain AS spf, rspf.spf AS spf_re,\
dhelo.domain AS helo, rhelo.spf AS helo_re,\
d1.domain AS dkim1, r1.dkim_selector AS dkim1_se, r1.dkim AS dkim1_re, r1.dkim_trans AS dkim1_t,\
d2.domain AS dkim2, r2.dkim_selector AS dkim2_se, r2.dkim AS dkim2_re, r2.dkim_trans AS dkim2_t,\
d3.domain AS dkim3, r3.dkim_selector AS dkim3_se, r3.dkim AS dkim3_re, r3.dkim_trans AS dkim3_t,\
d4.domain AS dkim4, r4.dkim_selector AS dkim4_se, r4.dkim AS dkim4_re, r4.dkim_trans AS dkim4_t\
FROM message_in AS m\
LEFT JOIN (msg_ref AS rd INNER JOIN domain AS dd ON rd.domain = dd.id)\
  ON m.id = rd.message_in AND FIND_IN_SET('dmarc', rd.auth)\
LEFT JOIN (msg_ref AS ra INNER JOIN domain AS da ON ra.domain = da.id)\
  ON m.id = ra.message_in AND FIND_IN_SET('author', ra.auth)\
LEFT JOIN (msg_ref AS rspf INNER JOIN domain AS dspf ON rspf.domain = dspf.id)\
  ON m.id = rspf.message_in AND FIND_IN_SET('spf', rspf.auth)\
LEFT JOIN (msg_ref AS rhelo INNER JOIN domain AS dhelo ON rhelo.domain = dhelo.id)\
  ON m.id = rhelo.message_in AND FIND_IN_SET('spf_helo', rhelo.auth)\
LEFT JOIN (msg_ref AS r1 INNER JOIN domain AS d1 ON r1.domain = d1.id)\
  ON m.id = r1.message_in AND r1.dkim_order = 1\
LEFT JOIN (msg_ref AS r2 INNER JOIN domain AS d2 ON r2.domain = d2.id)\
  ON m.id = r2.message_in  AND r2.dkim_order = 2\
LEFT JOIN (msg_ref AS r3 INNER JOIN domain AS d3 ON r3.domain = d3.id)\
  ON m.id = r3.message_in AND r3.dkim_order = 3\
LEFT JOIN (msg_ref AS r4 INNER JOIN domain AS d4 ON r4.domain = d4.id)\
  ON m.id = r4.message_in  AND r4.dkim_order = 4\
WHERE rd.domain = $(domain_ref) AND $(period_start) <= m.mtime AND m.mtime < $(period_end)\
GROUP BY source, disposition, d_dkim, d_spf, reason, author,\
 spf, spf_re, helo, helo_re, dkim1, dkim1_se, dkim1_re,\
 dkim2, dkim2_se, dkim2_re, dkim3, dkim3_se, dkim3_re, dkim4, dkim4_se, dkim4_re



MariaDB [mail]> explain WITH RECURSIVE dkim_sigs (dk_in, dk_domain, dkim, dkim_se, dkim_re, dkim_t, dkim_o) AS
(
  SELECT r.message_in AS dk_in, r.domain AS dk_domain, d.domain AS dkim, r.dkim_selector AS dkim_se, r.dkim AS dkim_re, r.dkim_trans AS dkim_t, 1 AS dkim_o
  FROM msg_ref AS r INNER JOIN domain AS d ON r.domain = d.id WHERE r.dkim_order = 1
  UNION
  SELECT dk_in, dk_domain, d.domain AS dkim, r.dkim_selector AS dkim_se, r.dkim AS dkim_re, r.dkim_trans AS dkim_t, dkim_o + 1
  FROM dkim_sigs, msg_ref AS r INNER JOIN domain AS d ON r.domain = d.id WHERE r.dkim_order = dkim_sigs.dkim_o + 1
)
SELECT INET_NTOA(CONV(HEX(m.ip),16,10)) AS source, COUNT(DISTINCT(m.id)) AS n, dkim, dkim_se, dkim_re, dkim_t, dkim_o
FROM message_in AS m, dkim_sigs WHERE m.id = dk_in AND m.mtime > 1636844400 AND dk_domain = 6;

+------+-----------------+------------+--------+---------------+---------+---------+---------------+-------+-------------------------------------------------+
| id   | select_type     | table      | type   | possible_keys | key     | key_len | ref           | rows  | Extra                                           |
+------+-----------------+------------+--------+---------------+---------+---------+---------------+-------+-------------------------------------------------+
|    1 | PRIMARY         | m          | range  | PRIMARY,mtime | mtime   | 4       | NULL          | 730   | Using index condition; Using where              |
|    1 | PRIMARY         | <derived2> | ref    | key0          | key0    | 5       | mail.m.id     | 10    | Using where                                     |
|    2 | DERIVED         | r          | ALL    | by_dom_msg    | NULL    | NULL    | NULL          | 36426 | Using where                                     |
|    2 | DERIVED         | d          | eq_ref | PRIMARY       | PRIMARY | 4       | mail.r.domain | 1     |                                                 |
|    3 | RECURSIVE UNION | r          | ALL    | by_dom_msg    | NULL    | NULL    | NULL          | 36426 |                                                 |
|    3 | RECURSIVE UNION | d          | eq_ref | PRIMARY       | PRIMARY | 4       | mail.r.domain | 1     |                                                 |
|    3 | RECURSIVE UNION | <derived2> | ALL    | NULL          | NULL    | NULL    | NULL          | 36426 | Using where; Using join buffer (flat, BNL join) |
| NULL | UNION RESULT    | <union2,3> | ALL    | NULL          | NULL    | NULL    | NULL          | NULL  |                                                 |
+------+-----------------+------------+--------+---------------+---------+---------+---------------+-------+-------------------------------------------------+
8 rows in set (0.007 sec)

Query was interrupted after 15 min.

Alt.: Repeat WHERE clause in all SELECTs:
WITH RECURSIVE dkim_sigs (dk_in, dk_domain, dkim, dkim_se, dkim_re, dkim_t, dkim_o) AS
(
  SELECT r.message_in AS dk_in, r.domain AS dk_domain, d.domain AS dkim, r.dkim_selector AS dkim_se, r.dkim AS dkim_re, r.dkim_trans AS dkim_t, 1 AS dkim_o
  FROM msg_ref AS r INNER JOIN domain AS d ON r.domain = d.id
  WHERE r.dkim_order = 1 AND r.domain = 6
  UNION
  SELECT dk_in, dk_domain, d.domain AS dkim, r.dkim_selector AS dkim_se, r.dkim AS dkim_re, r.dkim_trans AS dkim_t, dkim_o + 1
  FROM dkim_sigs, msg_ref AS r INNER JOIN domain AS d ON r.domain = d.id
  WHERE r.dkim_order = dkim_sigs.dkim_o + 1 AND r.domain = 6
)
SELECT INET_NTOA(CONV(HEX(m.ip),16,10)) AS source, COUNT(DISTINCT(m.id)) AS n, dkim, dkim_se, dkim_re, dkim_t, dkim_o
FROM message_in AS m, dkim_sigs WHERE m.id = dk_in AND m.mtime > 1636844400 AND dk_domain = 6;

Also interrupted.

