Re: [exim-dev] Mailop list: exim and google fighting over DKIM
On 2019-04-28 at 16:42 +0100, Andrew C Aitchison via Exim-dev wrote:
> Do the DKIM exim experts subscribe to the mailop list ?
I do, but I just started a new job and am behind on public
mailing-lists.
> There is an ongoing discussion on the mai...@mailop.org
> about a snafu with DKIM which implicates exim and google.
It looks like an instance of this:
https://bridge.grumpy-troll.org/2014/04/dmarc-stance/
but for p=none, not p=reject. It's been five years since I wrote that.
DMARC breaks mail for domains sending non-transactional non-bulk mail,
becauase DMARC breaks mailing-lists.
The only "out" is to implement ARC and hope that the receiving domain
does too. Gmail does, but you need to send enough mail for them to
decide to trust you as an ARC sender.
For the @exim.org mail-hub, we run with
8< exim.conf snippets >8
# This file is renewed daily by cron as user _exim:
.ifdef _HAVE_DMARC
dmarc_tld_file = /var/cache/exim/opendmarc.tlds
.elifdef _HAVE_ARC
dmarc_tld_file = /var/cache/exim/opendmarc.tlds
.endif
### ACLS section
acl_check_content:
# Outbound ARC-sign is disabled by default because people can auth to us
warn set acl_m_should_arcsign = false
# Do add Message-ID: and Date: headers
warn condition = ${if !def:h_Message-ID: {1}}
message = Message-ID:
warn condition = ${if !def:h_Date: {1}}
message = Date: $tod_full
# If there's an AAR (ARC-Authentication-Results) header which claims to be
# from us, but we're not receiving the mail from Mailman or other local
# system or authenticated user, then we consider it to be fraudulent or a
# mail-loop.
#
# We need to allow authenticated users for when someone is bouncing onwards
# a received message; OUTSIDE->exim.org->f...@exim.org, reads, bounces
# onwards for processing elsewhere, but being sent from f...@exim.org gets
# routed back via us (for DKIM/SPF alignment), so we appear in the AAR
# headers already.
#
# Does not need to be _HAVE_ARC-guarded, this is pure Exim logic and safe
# even when we're not signing.
deny message= Mail-loop or fraudulent headers: found ourselves in
ARC-Authentication-Results:
!hosts = +relay_from_hosts
!authenticated = *
condition = ${if def:h_ARC-Authentication-Results: {yes}}
condition = ${if inlist{ADMD}{<; $h_ARC-Authentication-Results: }}
warn hosts = <; ; 127.0.0.1 ; ::1
condition = ${if def:h_X-Exim-Org-Should-ARC-Sign: {yes}}
set acl_m_should_arcsign = $h_X-Exim-Org-Should-ARC-Sign:
remove_header = X-Exim-Org-Should-ARC-Sign
# Bypass the content scanning for stuff injected from mailman etc
accept hosts = <; ; 127.0.0.1 ; ::1
### THERE ARE MORE STEPS HERE IN THIS ACL (unrelated to ARC)
# list stuff & authenticated
accept hosts = +relay_from_hosts
accept authenticated = *
# Insert DMARC headers for non-authenticated non-local senders
warndmarc_status = accept : reject : quarantine : none : norecord :
nofrom : temperror : off
log_message= DMARC DEBUG: $dmarc_status $dmarc_used_domain
warncondition = ${if !eq{$dmarc_status}{accept}}
log_message= DMARC DEBUG: not-accept '$dmarc_status' for
$dmarc_used_domain
warncondition = ${if eq{$dmarc_status}{quarantine}}
set acl_m_quarantine = 1
denycondition = ${if eq{$dmarc_status}{reject}}
message= Message from $dmarc_used_domain failed sender's
DMARC policy, REJECT
.ifdef _HAVE_ARC
# and ARC chaining
warnverify = arc/pass:none:fail
logwrite = ARC: state=<$arc_state>${if def:arc_state_reason{
reason=<$arc_state_reason>}}
.endif
### TRANSPORTS SECTION
remote_smtp:
driver = smtp
rcpt_include_affixes
# hosts_try_chunking =
dnssec_request_domains = *
hosts_try_dane = *
tls_require_ciphers = TLS_OUTBOUND_DEFAULT
dane_require_tls_ciphers = TLS_OUTBOUND_HIGHSEC
dkim_domain = ${domain:$sender_address}
dkim_selector = ${lookup
{$dkim_domain}lsearch{/etc/exim/dkim/domains-mapping} {$value}{SKIP}}
dkim_private_key = ${if
eq{$dkim_selector}{SKIP}{false}{/etc/exim/dkim/rsa.private.$dkim_selector.$dkim_domain}}
dkim_strict = 1
.ifdef _HAVE_ARC
arc_sign = ${if bool{$acl_m_should_arcsign}{${lookup
{ADMD}lsearch{/etc/exim/dkim/domains-mapping}{ADMD : $value :
/etc/exim/dkim/rsa.private.$value.ADMD }fail}}fail}
.endif
.ifdef WANT_MAILMAN
mailman_transport:
envelope_to_add
driver = pipe
command = MM_WRAP \
'${if def:local_part_suffix \
{${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} \
{post}}' \
$local_part
headers_add = X-Exim-Org-Should-ARC-Sign: $acl_m_should_arcsign
current_directory = MM_HOME
home_directory = MM_HOME
user = MM_UID
group = MM_GID
.endif
8< exim.conf snippets
Re: [exim-dev] Mailop list: exim and google fighting over DKIM
On 30 Apr 2019, at 00:03, Jeremy Harris via Exim-dev wrote: > Bzzt. > > Stop right there. > > The ML adds body content. This will invalidate the signature. > > The header thing, even if you do game it, will not help that. *shrug* I was merely explaining the cogent points of the discussion on mailop, Jeremy, given comments by other list subscribers/bz posters on the private nature of the list. Graeme -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] Mailop list: exim and google fighting over DKIM
On Mon, Apr 29, 2019 at 01:33:09PM -0600, Brielle Bruns via Exim-dev wrote:
> Version: 4.92-2~bpo9+1
>
> My configuration files are ancient. IIRC, I originally built them from the
> debian config back around 2005 or 2006 (We've actually been using exim since
> 2003). They got updated over the years as needed and include a bunch of
> special stuff that is integrated with my DNSbl work.
>
> So, the DKIM section is based on what I found in Debian's config. Under the
> remote_smtp transport, I have the following:
>
> dkim_domain = DKIM_DOMAIN
> dkim_selector = DKIM_SELECTOR
> dkim_private_key = DKIM_PRIVATE_KEY
> dkim_canon = DKIM_CANON
>
> Those variables are defined as:
>
> DKIM_SELECTOR = default
> DKIM_DOMAIN = ${sg{${lc:${domain:$h_from:}}}{^www\.}{}}
> DKIM_FILE = /etc/exim4/dkim/${lc:${domain:$h_from:}}-private.pem
> DKIM_PRIVATE_KEY = ${if exists{DKIM_FILE}{DKIM_FILE}{0}}
> DKIM_CANON = relaxed
JFTR, Debian's default configuration does have all settings in the
remote_smtp transport bracketed in .ifdefs:
.ifdef DKIM_DOMAIN
dkim_domain = DKIM_DOMAIN
.endif
.ifdef DKIM_SELECTOR
dkim_selector = DKIM_SELECTOR
.endif
.ifdef DKIM_PRIVATE_KEY
dkim_private_key = DKIM_PRIVATE_KEY
.endif
.ifdef DKIM_CANON
dkim_canon = DKIM_CANON
.endif
.ifdef DKIM_STRICT
dkim_strict = DKIM_STRICT
.endif
.ifdef DKIM_SIGN_HEADERS
dkim_sign_headers = DKIM_SIGN_HEADERS
.endif
and doesn't define any of those macros in the default. I verified this
in the unstable version of Debian, exim4 4.92-5.
Greetings
Marc
--
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim
details at http://www.exim.org/ ##
Re: [exim-dev] Mailop list: exim and google fighting over DKIM
On 4/29/2019 5:00 PM, Jeremy Harris via Exim-dev wrote: You'll get the default headers signed, then. That includes signing the lack-of-presence of a header in the set. However: it's irrelevant. The ML adding a header making the signature bad does not matter. The ML also appends to the body... making your signature bad. That's what signatures are for (I'm sure you know this, just wanting to be clear in case...) So. DKIM signatures do not survive transmission through a ML. (Sometime phrased as "DKIM breaks mailinglists"). The DKIM RFCs say "do not treat a lack of verifiable DKIM signature as cause for rejecting a message". Google is ignoring that, and the brou-ha-ha is not really, IMHO, Exim's problem. It's a 800Kg gorilla problem. Thank you for confirming. I suspected that was the case - there was a lot of conflicting feedback, and I didn't think there was technically anything wrong with my setup. Just a quick thanks to all the Exim devs for their work over the years! -- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org/ http://www.ahbl.org -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] Mailop list: exim and google fighting over DKIM
On 29/04/2019 20:06, Graeme Fowler via Exim-dev wrote: > On 29 Apr 2019, at 19:26, Andrew C Aitchison via Exim-dev > wrote: >> I will do so, either here or in that bug, when I can do so without causing >> more heat. > > The gist of the discussion (I’m a mailop subscriber) is manyfold: > > 1. Google accept messages over IPv6 but require stricter verification over > IPv6 than IPv4 > 2. mailop.org has no SPF record > 3. mailop.org can deliver to Google over IPv6 > 4. Signed messages inbound to mailop.org (and other lists!) from > Debian-derived and other setups using the default macro defined in pdkim.h > can have headers added which have been declared signed when they’re not > present* > 5. Adding them invalidates the signature, and if the MLM software doesn’t > remove or re-sign the message, that can be problematic if a receiving system > is taking extra care with respect to verification Bzzt. Stop right there. The ML adds body content. This will invalidate the signature. The header thing, even if you do game it, will not help that. > 6. It appears Google will reject messages that combine elements 1 thru 5 > > * this is the bit I’m confused by; I have a large historical pile of messages > to lists from me and I can’t see a signature with those headers included > despite me using the defaults for a long period. > > RFC4871 stated that a suggested list of headers SHOULD be signed, if existing. > RFC6376 dropped the SHOULD and provided an example of what constitutes the > “core” of a message. > > So *either* the Debian-derived configuration (of which the original poster > mentioned they were using unaltered for DKIM purposes, inheriting defaults) > does something different to what is expected by defaukt, or Exim’s behaviour > changed somewhere after 4.86/4.87 (still trying to pinpoint that) but I can’t > see or replicate the issue. > > Puzzled, perplexed… and given that this has only just been raised, I’m even > more puzzled by it. There must be receivers out there who have insanely > strict validation policies who would have seen this before! > > Graeme > -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] Mailop list: exim and google fighting over DKIM
On 29/04/2019 20:33, Brielle Bruns via Exim-dev wrote: > Heya, original cause of the havoc on mailop here! > > I'll try and answer whatever questions I can. See below. > > > On 2019-04-29 19:06, Graeme Fowler wrote:> So *either* the > Debian-derived configuration (of which the original poster mentioned > they were using unaltered for DKIM purposes, inheriting defaults) does > something different to what is expected by defaukt, or Exim’s behaviour > changed somewhere after 4.86/4.87 (still trying to pinpoint that) but I > can’t see or replicate the issue. 4.87 introduced support for oversigning. > Version: 4.92-2~bpo9+1 > > My configuration files are ancient. IIRC, I originally built them from > the debian config back around 2005 or 2006 (We've actually been using > exim since 2003). They got updated over the years as needed and include > a bunch of special stuff that is integrated with my DNSbl work. > > So, the DKIM section is based on what I found in Debian's config. Under > the remote_smtp transport, I have the following: > > dkim_domain = DKIM_DOMAIN > dkim_selector = DKIM_SELECTOR > dkim_private_key = DKIM_PRIVATE_KEY > dkim_canon = DKIM_CANON You'll get the default headers signed, then. That includes signing the lack-of-presence of a header in the set. However: it's irrelevant. The ML adding a header making the signature bad does not matter. The ML also appends to the body... making your signature bad. That's what signatures are for (I'm sure you know this, just wanting to be clear in case...) So. DKIM signatures do not survive transmission through a ML. (Sometime phrased as "DKIM breaks mailinglists"). The DKIM RFCs say "do not treat a lack of verifiable DKIM signature as cause for rejecting a message". Google is ignoring that, and the brou-ha-ha is not really, IMHO, Exim's problem. It's a 800Kg gorilla problem. -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] Mailop list: exim and google fighting over DKIM
Heya, original cause of the havoc on mailop here!
I'll try and answer whatever questions I can. See below.
On 2019-04-29 19:06, Graeme Fowler wrote:> So *either* the
Debian-derived configuration (of which the original poster mentioned
they were using unaltered for DKIM purposes, inheriting defaults) does
something different to what is expected by defaukt, or Exim’s behaviour
changed somewhere after 4.86/4.87 (still trying to pinpoint that) but I
can’t see or replicate the issue.
Version: 4.92-2~bpo9+1
My configuration files are ancient. IIRC, I originally built them from
the debian config back around 2005 or 2006 (We've actually been using
exim since 2003). They got updated over the years as needed and include
a bunch of special stuff that is integrated with my DNSbl work.
So, the DKIM section is based on what I found in Debian's config. Under
the remote_smtp transport, I have the following:
dkim_domain = DKIM_DOMAIN
dkim_selector = DKIM_SELECTOR
dkim_private_key = DKIM_PRIVATE_KEY
dkim_canon = DKIM_CANON
Those variables are defined as:
DKIM_SELECTOR = default
DKIM_DOMAIN = ${sg{${lc:${domain:$h_from:}}}{^www\.}{}}
DKIM_FILE = /etc/exim4/dkim/${lc:${domain:$h_from:}}-private.pem
DKIM_PRIVATE_KEY = ${if exists{DKIM_FILE}{DKIM_FILE}{0}}
DKIM_CANON = relaxed
There's nothing else adding dkim headers in the config file.
Puzzled, perplexed… and given that this has only just been raised, I’m even
more puzzled by it. There must be receivers out there who have insanely strict
validation policies who would have seen this before!
Let me know how I can be of help at all!
--
Brielle Bruns
The Summit Open Source Development Group
http://www.sosdg.org/ http://www.ahbl.org
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim
details at http://www.exim.org/ ##
Re: [exim-dev] Mailop list: exim and google fighting over DKIM
On 29/04/2019 20:06, Graeme Fowler via Exim-dev wrote: > The gist of the discussion (I’m a mailop subscriber) is manyfold: > 4. Signed messages inbound to mailop.org (and other lists!) from > Debian-derived and other setups using the default macro defined in pdkim.h > can have headers added which have been declared signed when they’re not > present* > * this is the bit I’m confused by; I have a large historical pile of messages > to lists from me and I can’t see a signature with those headers included > despite me using the defaults for a long period. The list of headers has not changed but the behaviour has. I'm using Exim 4.86.2 and it doesn't do this. The documented behaviour of the latest version is that it will sign them if they're absent. I use a custom pairing of IPv4-only followed by IPv4+IPv6 routers for all hosts to avoid Google's actively hostile behaviour towards IPv6. -- Simon Arlott -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] Mailop list: exim and google fighting over DKIM
On 29 Apr 2019, at 19:26, Andrew C Aitchison via Exim-dev wrote: > I will do so, either here or in that bug, when I can do so without causing > more heat. The gist of the discussion (I’m a mailop subscriber) is manyfold: 1. Google accept messages over IPv6 but require stricter verification over IPv6 than IPv4 2. mailop.org has no SPF record 3. mailop.org can deliver to Google over IPv6 4. Signed messages inbound to mailop.org (and other lists!) from Debian-derived and other setups using the default macro defined in pdkim.h can have headers added which have been declared signed when they’re not present* 5. Adding them invalidates the signature, and if the MLM software doesn’t remove or re-sign the message, that can be problematic if a receiving system is taking extra care with respect to verification 6. It appears Google will reject messages that combine elements 1 thru 5 * this is the bit I’m confused by; I have a large historical pile of messages to lists from me and I can’t see a signature with those headers included despite me using the defaults for a long period. RFC4871 stated that a suggested list of headers SHOULD be signed, if existing. RFC6376 dropped the SHOULD and provided an example of what constitutes the “core” of a message. So *either* the Debian-derived configuration (of which the original poster mentioned they were using unaltered for DKIM purposes, inheriting defaults) does something different to what is expected by defaukt, or Exim’s behaviour changed somewhere after 4.86/4.87 (still trying to pinpoint that) but I can’t see or replicate the issue. Puzzled, perplexed… and given that this has only just been raised, I’m even more puzzled by it. There must be receivers out there who have insanely strict validation policies who would have seen this before! Graeme -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] Mailop list: exim and google fighting over DKIM
On Sun, 28 Apr 2019, Andrew C Aitchison via Exim-dev wrote: Do the DKIM exim experts subscribe to the mailop list ? In https://bugs.exim.org/show_bug.cgi?id=2394 Simon Arlott suggests that the answer is "no". In that case (and especially since the links below are on a server which has an out of date certificate), I need to answer his questions carefully; I will do so, either here or in that bug, when I can do so without causing more heat. There is an ongoing discussion on the mai...@mailop.org about a snafu with DKIM which implicates exim and google. The original report of the snafu (google rejections caused the list to auto-unsubscribe over a hundred subscribers of the list): https://chilli.nosignal.org/cgi-bin/mailman/private/mailop/2019-April/013974.html A description of the sending system that caused the issue: https://chilli.nosignal.org/cgi-bin/mailman/private/mailop/2019-April/013991.html A suggestion for the exim developers: https://chilli.nosignal.org/cgi-bin/mailman/private/mailop/2019-April/013994.html I believe that the list archive links above are member only, -- Andrew C. Aitchison Cambridge, UK and...@aitchison.me.uk -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] Mailop list: exim and google fighting over DKIM
On 28/04/2019 16:42, Andrew C Aitchison via Exim-dev wrote: > Do the DKIM exim experts subscribe to the mailop list ? > > There is an ongoing discussion on the mai...@mailop.org > about a snafu with DKIM which implicates exim and google. > > The original report of the snafu (google rejections caused the list to > auto-unsubscribe over a hundred subscribers of the list): > https://chilli.nosignal.org/cgi-bin/mailman/private/mailop/2019-April/013974.html > > > A description of the sending system that caused the issue: > https://chilli.nosignal.org/cgi-bin/mailman/private/mailop/2019-April/013991.html > > > A suggestion for the exim developers: > https://chilli.nosignal.org/cgi-bin/mailman/private/mailop/2019-April/013994.html > > > Basically a user with a stock debian exim setup (version number yet given) > sent a message to the list with some signed non-existent headers; when the > list passed the message on it generated these headers and google failed > on the signature discrepancy. Most mailinglists, including mailop, append to the body of submissions distributed as non-digests, so DKIM signatures will become invalid due to that (assuming the mostly-deprecated DKIM bodylength feature is not used). Trying to game the adding of list headers would be applying lipstick to a pig. The snafu is Google's fault for ignoring the part of the DKIM standard that says "a lack of verifiable signature should not be grounds for rejection" (my paraphrase, RFC 6376 Section 6.3), and DKIM's fault for being an enabler of breaking traditional uses of email (mailinglists, in this case). -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[exim-dev] Mailop list: exim and google fighting over DKIM
Do the DKIM exim experts subscribe to the mailop list ? There is an ongoing discussion on the mai...@mailop.org about a snafu with DKIM which implicates exim and google. The original report of the snafu (google rejections caused the list to auto-unsubscribe over a hundred subscribers of the list): https://chilli.nosignal.org/cgi-bin/mailman/private/mailop/2019-April/013974.html A description of the sending system that caused the issue: https://chilli.nosignal.org/cgi-bin/mailman/private/mailop/2019-April/013991.html A suggestion for the exim developers: https://chilli.nosignal.org/cgi-bin/mailman/private/mailop/2019-April/013994.html Basically a user with a stock debian exim setup (version number yet given) sent a message to the list with some signed non-existent headers; when the list passed the message on it generated these headers and google failed on the signature discrepancy. I believe that the list archive links above are member only, -- Andrew C. Aitchison Cambridge, UK and...@aitchison.me.uk -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
