Re: [dmarc-ietf] Composition Kills: A Case Study of Email Sender Authentication

[ned+dmarc]

On 2020-04-23 02:20, John Levine wrote:
> In article <51be5654-94c4-38c6-8f6b-dca403d66...@dcrocker.net> you
> write:



>> The paper asserts that AR is used as DMARC input.  I suspect that is
>> rarely, if ever, true.  Yes? No?
>
> I'd say never.  To do DMARC rejects you have to do all of the
> validation in the SMTP daemon, which is before anything has a chance
> to create an A-R header.



Of course this is possible with the Milter protocol introduced by
Sendmail and used also by Postfix. The mail traverses during the SMTP
phase through different milters, e.g. a SPF milter, followed by a DKIM
milter, and every milter injects an AR header with its results. The last
milter is a DMARC milter that processes the AR headers and signals the
SMTP daemon do either accept or reject the message. This is how OpenDKIM
& OpenDMARC work together.


This is done all the time, and not just by Sendmail and Postfix.


Other projects do it all in one milter (rspamd?)...


Yes, but when you do it all in one go it's more difficult to customize.

Ned

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Composition Kills: A Case Study of Email Sender Authentication

[Juri Haberland]

On 2020-04-23 02:20, John Levine wrote:
In article <51be5654-94c4-38c6-8f6b-dca403d66...@dcrocker.net> you 
write:



The paper asserts that AR is used as DMARC input.  I suspect that is
rarely, if ever, true.  Yes? No?


I'd say never.  To do DMARC rejects you have to do all of the
validation in the SMTP daemon, which is before anything has a chance
to create an A-R header.


Of course this is possible with the Milter protocol introduced by 
Sendmail and used also by Postfix. The mail traverses during the SMTP 
phase through different milters, e.g. a SPF milter, followed by a DKIM 
milter, and every milter injects an AR header with its results. The last 
milter is a DMARC milter that processes the AR headers and signals the 
SMTP daemon do either accept or reject the message. This is how OpenDKIM 
& OpenDMARC work together.


Other projects do it all in one milter (rspamd?)...


  Juri

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Composition Kills: A Case Study of Email Sender Authentication

[John Levine]

In article <51be5654-94c4-38c6-8f6b-dca403d66...@dcrocker.net> you write:
>The paper has a simplistic model of email anti-abuse processing and this 
>distorts some of its analysis.  At the least, the paper needs to 
>distinguish between theory and practice.

Yup.

>> SPF implementations treat “(a...@legitimate.com” as anempty MAIL FROM 
>> address, and thus forward the resultsof checking HELO to the DMARC 
>> component, because thestring in the parentheses can be parsed as a 
>> comment ac-cording to RFC 5322 [10]. Some DMARC 
>> implementations,however, may take it as a normal non-empty address,
>
>1. This issues goes away if SPF supplies DMARC the domain name, rather 
>than DMARC having to fetch it

Their assumptions about DMARC validators may be true in some cases but
not in general.  In OpenDMARC which is pretty widely used, the caller
tells the DMARC library what domain SPF checked, whether it was the
mail_from or helo, and what the result was.

>The paper asserts that AR is used as DMARC input.  I suspect that is 
>rarely, if ever, true.  Yes? No?

I'd say never.  To do DMARC rejects you have to do all of the
validation in the SMTP daemon, which is before anything has a chance
to create an A-R header.

>The rfc5322.From: field is independent of MIME. MIME pertains only to 
>the body.

The display name can be a MIME encoded word, but that has nothing to do with
parsing the MIME parts in the body of the message.

I'm still impressed at some of the really simple bugs they exploited,
like NUL characters in header fields.

R's,
John

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Composition Kills: A Case Study of Email Sender Authentication

[Scott Kitterman]

On Wednesday, April 22, 2020 4:28:44 PM EDT Dave Crocker wrote:
> > 4.1 HELO/MAIL FROM confusion
> 
> This seems to imply that tightening is needed in DMARC, so that it uses
> an SPF domain that SPF has actually been validated? I think the issue is
> that SPF validation needs to inform DMARC what domain it has validated,
> rather than have DMARC decide which domain to fetch.

The standard authentication results header field will do this in the form of:

Authentication-Results: mx.example.org; spf=pass smtp.mailfrom=example.com

> > SPF implementations treat “(a...@legitimate.com” as anempty MAIL FROM
> > address, and thus forward the resultsof checking HELO to the DMARC
> > component, because thestring in the parentheses can be parsed as a
> > comment ac-cording to RFC 5322 [10]. Some DMARC
> > implementations,however, may take it as a normal non-empty address,
> 
> 1. This issues goes away if SPF supplies DMARC the domain name, rather
> than DMARC having to fetch it
> 
> 2. I doubt this otherwise needs changes to language in the DMARC spec,
> but it's worth making sure.

I agree it's worth making sure.  The input for DMARC from SPF/DKIM should a 
both a result and a domain.

There is some inconsistency in how SPF verifiers report SPF results when using 
HELO as a fallback for an null Mail From.  RFC 7208 section 2.4 is, IMO, 
reasonably clear that for a null Mail From, the SPF 'Mail From' identity is 
postmaster@[HELO], so it should be reported as an smtp.mailfrom= result, which 
the paper claims is the threat vector.  I think they are wrong.  As long as 
the input domain is checked and is in alignment for DMARC, there's no issue 
there.

I tried their exact scenario with two of my domains and here's what the SPF 
verifier reported:

Authentication-Results: mx.example.org; spf=pass (sender SPF authorized) 
smtp.mailfrom=kitterman.com (client-ip=72.81.252.18; helo=kitterman.org; 
envelope-from=(sc...@kitterman.com; receiver=bog...@kitterman.org)

While this is not ideal (for envelope-from=(sc...@kitterman.com.  The string 
should be quoted [per the 5322 CFWS definition], it shouldn't lead to any 
confusion about the identity.

I figured out this is wrong and the example below is right per various RFCs, 
but it took some investigation to dig it all back up.  A more thorough review 
by someone that knows more than me would surely be a good idea.

> > 4.3 Authentication results injection
> 
> Another focus on what results are communicated and how.
> 
> The paper asserts that AR is used as DMARC input.  I suspect that is
> rarely, if ever, true.  Yes? No?

For integration of open source components such as opendkim and opendmarc, this 
is the standard method of communicating results data between components (also 
true for communicating SPF results to opendmarc is you don't use its internal 
SPF implementation).

For what it's worth, I invested some time yesterday and today testing the 
various open source components I maintain (pyspf, pyspf-milter, policyd-spf-
python, dkimpy, dkimpy-milter, and authres) and found some behaviors in the 
cases they describe that is sub-optimal, I could not replicate any of the 
security relevant issues they describe.  I did get some new test cases. ...

As an example, if the example they used is presented to the authres module as 
the d= domain in for a DKIM verification, d=legitimate.com(.attacker.com, it 
returns d="legitimate.com(.attacker.com" [note the quoting, which is correct 
per the 5322 CFWS definition].  If it's parsing that, it fails to return a 
domain at all, which is a bug, but doesn't cause the reported problem.

Scott K



___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Composition Kills: A Case Study of Email Sender Authentication

[Dave Crocker]

The paper concerns valid authentication and effective use of 
authenticated information.  That's an excellent goal.  The paper offers 
extensive technical analyses and empirical demonstrations of operational 
failures. Exercises like these are valuable and too infrequent.  Besides 
prompting specific repairs, work like this should motivate careful 
industry review of systems-level issues and standards-based documentation.


The paper does not adequately distinguish between claims of protocol 
design/specification errors, versus implementation errors, yet that 
distinction is essential.


The paper has a simplistic model of email anti-abuse processing and this 
distorts some of its analysis.  At the least, the paper needs to 
distinguish between theory and practice.


The paper has a flawed model of the role of recipient users in 
anti-abuse work and therefore the paper makes a basic error in its 
assessment of the importance of From: field validation.


The paper identifies a number of significant implementation deficiencies 
in various system.



Detailed comments:

to authenticate the sender’s purported identity, as the basis for 
displaying in email clients assurances of validity to userswhen they 
read messages


That's a rather basic misunderstanding of how these mechanisms are 
actually used.



In practice, attackers usually exploit SMTPby running their own email 
servers or clients.


This is largely a non-sequitur.  It has built-in assumptions that aren't 
explained and are wrong with respect to any of the domain name-based 
protection mechanisms.  (And it isn't SMTP that is being exploited, per 
se. That's like saying that if I send a postal letter purporting to be 
the president of the United States, I've exploited the postal system. )



SMTP’s design includes multiple “identities” when handling messages. 

Since this has been documented, they should cite it, for background.


Both theMAIL FROMandFromheadersidentify the email sender, but they 
have different meaningsin an SMTP conversation. The first represents 
the user whotransmittedthe message, and is usually not displayed to 
therecipient. 
It would be clearer and more helpful to say 'operator' or 'service 
provider' rather than 'user'.  A separate issue is whether their 
characterization of the differences between the two identifiers is valid 
in practice... The paper uses sender in a way that encourages confusion 
in the reader.  Especially since this is a technical paper, it should 
use language that carefully distinguishes roles, especially since 
terminology for making these distinctions is readily available.



In addition, SMTP introduces multiple other sender identi-ties, such 
as theHELOcommand,SenderandResent-Fromheaders.


I suspect that the failure to cite DKIM's d= field , until later, is a 
significant oversight.




http://www.ietf.org/internet-drafts/draft-blank-ietf-bimi-00.txt


heh.  the URL isn't valid.  Should be:

https://www.ietf.org/archive/id/draft-blank-ietf-bimi-00.txt


DKIM.DomainKeys Identified Mail (DKIM) uses cryp-tography to 
authenticate senders...


I'm being too picky, right?  Formal DKIM semantics don't produce exactly 
that result.



The general idea behind DKIM is to let senders sign parts ofmessages 
so that receivers can validate them.
This is poorly written, wrong, or both.  'them' is ambiguous.  And the 
signing is to affix the d= value, not to validate any of the data that 
is part of the signature.




it only need to have the same registered domain


Will typical readers understand what this means, in the context of this 
paper, since all of the domain name's components are 'registered'? It's 
not obvious what language would make the meaning clearer.




If the email passes theDMARC verification, it enters the user’s inbox.


This is simply wrong.  It needs to place this phase of processing inside 
a complex filtering engine, which is the primary venue for using any of 
the mechanisms discussed in the paper.



usually the MUA only displays theFromheader as the message sender.


These days, this isn't true either.  Most users only see the 
un-validated Display-Name.



Thus, theFromheaderprovides the key identity relevant for gaining the 
user’s trust


On its face, this seems a reasonable view.  In practice, it isn't.  Mail 
with obviously bogus email addresses in the rfc5322.From field are still 
effective for phishing, because what real users actually pay attention 
to is the body of the message and the identification information in it.  
That's why assertions about the display of validated source/author 
information to the end user are demonstrably wrong.



Malicious usersof legitimate email providers exploit thefailure of 
some email providers to perform sufficient valida-tion of emails 
received from local MUAs. These attackers cansend emails with 
spoofedFromheaders. The exploited emailproviders may automatically 
attach DKIM 

Re: [dmarc-ietf] Composition Kills: A Case Study of Email Sender Authentication

[John Levine]

In article <2656238.kvSPeydUtl@sk-desktop> you write:
>There is probably protocol improvement work that should be done based on:
>
>https://www.usenix.org/system/files/sec20fall_chen-jianjun_prepub_0.pdf

I didn't see any protocol issues other than the well known DKIM
multiple From: headers (the Doug Otis feature) and l=.  They certainly
did find a lot of implementation bugs, some of which I found pretty
surprising, like Gmail allowing and misinterpreting NUL characters in
DKIM signature headers.

This sounds like we need more test suites and perhaps more reminders
that when you're writing security software, being forgiving of other
people's bugs will backfire on you.

R's,
John

___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

Re: [dmarc-ietf] Composition Kills: A Case Study of Email Sender Authentication

[Kurt Andersen (b)]

Extracting from the abstract of a paper in process by Casey Deccio (now
researching SPF at BYU, formerly at Cisco):

Our techniques elicit SPF and DMARC validation activity of the servers,
> while minimizing spam and perceived abuse. We find that only 25% of mail
> servers and less than half of domains deploy SPF validation. Of domains
> that perform SPF validation, 7.6% exhibit inconsistent validation behavior
> across mail servers. We also find that 75% of organizations with mail
> servers for popular domains share DNS infrastructure with up to tens of
> thousands of others.


Most of his focus has been on SPF but it would be useful to loop him into
the DMARCbis discussions.

--Kurt

On Tue, Apr 21, 2020 at 9:11 AM Scott Kitterman  wrote:

> There is probably protocol improvement work that should be done based on:
>
> https://www.usenix.org/system/files/sec20fall_chen-jianjun_prepub_0.pdf
>
> Scott K
>
>
> ___
> dmarc mailing list
> dmarc@ietf.org
> https://www.ietf.org/mailman/listinfo/dmarc
>
___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc

[dmarc-ietf] Composition Kills: A Case Study of Email Sender Authentication

[Scott Kitterman]

There is probably protocol improvement work that should be done based on:

https://www.usenix.org/system/files/sec20fall_chen-jianjun_prepub_0.pdf

Scott K


___
dmarc mailing list
dmarc@ietf.org
https://www.ietf.org/mailman/listinfo/dmarc