Re: [dmarc-ietf] Two new fields in aggregate reports
In article <682972a4-38e4-f5b2-3180-c5a03a3a0...@tana.it> you write: >Looking at aggregate reports, you cannot tell whether an authentication failure >is a sacrosanct signaling of your domain being abused rather than a legitimate >user going through external forwarders. Sure you can, you look at the IP address and see who it is. In my reports I see bursts of authentication failures from hosts that are obviously mailing list servers, and lots of failures in China which are random spambots. >In theory, reports can be something more than a debugging aid. It has the >potential to assemble a community where bad actors are identified and >dismissed. No, that's not what they're for and they don't have the necessary info. There are systems that compile data for IP reputation but that's not what DMARC is. The point of DMARC is to try to tell "is this message really from X", not "is this message spam." R's, John ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Two new fields in aggregate reports
Hi Dilyan, On Fri 25/Oct/2019 12:51:43 +0200 Дилян Палаузов wrote: > > I do not see how this helps for DMARC. An email either validates DMARC, or > fails DMARC and the aggregate repors say per sending IP server (only direct > mail flow is reported), whether DMARC validates or fails. With this > information it is sufficient to determine, if the DMARC/DKIM implementations > on sender and receiver are either both bug-free, or both have the same > bugs. Looking at aggregate reports, you cannot tell whether an authentication failure is a sacrosanct signaling of your domain being abused rather than a legitimate user going through external forwarders. > I do not see, how the information you ask to add, while interesting, does > help DMARC.> > What is the purposes of the aggregate and non-aggregate reports? What are > non-goals? I asked several times here, nobody answered. Perhaps a > discussion on the goals and non-goal would help. That was probably discussed already. Now that we have some experience, we can discuss further. I know some very acknowledgeable WG participants accumulate aggregate report values in their own MySQL database (I'm not sure about the details). Many people, instead, outsource reports to specialized DMARC analyzers, who display nice graphical summaries. I run an XSLT transform of DMARC reports into an HTML tabular format of one row per record. In theory, reports can be something more than a debugging aid. It has the potential to assemble a community where bad actors are identified and dismissed. > If it is a goal to reuse the dmarc-reporting mechanism to report also about > perceived spam probability, then it can be discussed in more details how > this can be achieved. Well, spam score usually is hight for phishing too. To counter phishing is DMARC core business. > My experience is, that asking a provider, why an obviously non-spam mail was > evaluated as spam, virtually never leads to a useful answer. So nobody > wants to reveal how its spam system weigths factors and if there is lack of > such interest, extending the report format will not help, as nobody will be > willing the report the data. This is a problem, indeed. Large mailbox providers may fear that giving bad scores to an IP can result in senders complaining against against their weighting method, requiring more personnel to answer back. It should be made clear that reports are given out AS IS, as a favor to senders, without liability. Anyway, reporting MTAs don't have to reveal the method, just the result. > Exchanging information on hard-coded rules in Spam-Assassing (IP reputation, > HTML mime part without text/plain, the “Nigeria money” phrase), that is not > based on filter training, does not help neither, as sender can run the tests > on its own and predict how the recipient will evaluate these set of > criteria. Changing point of view, perspective also changes. In addition, by comparing external scores to internal predictions, one has a chance to evaluate the goodness of the reporting MTA. Best Ale ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Two new fields in aggregate reports
On Fri, Oct 25, 2019 at 1:53 PM Seth Blank wrote: > On Fri, Oct 25, 2019 at 10:49 AM John Levine wrote: > >> As far as I know, the point of DMARC reports is to help domain owners >> understand who is sending mail that purports to be from them. In a >> large organization it can be remarkably hard to track down every mail >> server in every department or every subcontractor that might be sending >> real mail with the domain in the From: header. >> >> The domain owners use the reports to do things like update SPF records >> to include all of the sending hosts, update server configs to add DKIM >> signatures, or to fix servers that are adding invalid signatures, and >> often to shut rogue servers down that shouldn't have been sending mail >> in the first place. >> >> I can't see how spam scores would be of any use for any of these tasks. >> > > +1000 > > The point of DMARC reports is to understand what is not authenticating in > an aligned fashion, so that you can get those mailstreams authenticating > properly and verify things are now correct. Spam, nor insight into receiver > mechanisms to combat spam (which change daily, per Brandon), is out of > scope of DMARC reporting. > > Seth > > Absolutely agree with the sentiments expressed by John and Seth. Michael Hammer ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Two new fields in aggregate reports
> I can't see how spam scores would be of any use for any of these > tasks. i don't think anyone will include those values even if they can/could for fear of it being used against them ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Two new fields in aggregate reports
On Fri, Oct 25, 2019 at 10:49 AM John Levine wrote: > As far as I know, the point of DMARC reports is to help domain owners > understand who is sending mail that purports to be from them. In a > large organization it can be remarkably hard to track down every mail > server in every department or every subcontractor that might be sending > real mail with the domain in the From: header. > > The domain owners use the reports to do things like update SPF records > to include all of the sending hosts, update server configs to add DKIM > signatures, or to fix servers that are adding invalid signatures, and > often to shut rogue servers down that shouldn't have been sending mail > in the first place. > > I can't see how spam scores would be of any use for any of these tasks. > +1000 The point of DMARC reports is to understand what is not authenticating in an aligned fashion, so that you can get those mailstreams authenticating properly and verify things are now correct. Spam, nor insight into receiver mechanisms to combat spam (which change daily, per Brandon), is out of scope of DMARC reporting. Seth -- *Seth Blank* | Director, Industry Initiatives *e:* s...@valimail.com *p:* 415.273.8818 This email and all data transmitted with it contains confidential and/or proprietary information intended solely for the use of individual(s) authorized to receive it. If you are not an intended and authorized recipient you are hereby notified of any use, disclosure, copying or distribution of the information included in this transmission is prohibited and may be unlawful. Please immediately notify the sender by replying to this email and then delete it from your system. ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Two new fields in aggregate reports
In article you write: >What is the purposes of the aggregate and non-aggregate reports? What are >non-goals? I asked several times here, >nobody answered. Perhaps a discussion on the goals and non-goal would help. As far as I know, the point of DMARC reports is to help domain owners understand who is sending mail that purports to be from them. In a large organization it can be remarkably hard to track down every mail server in every department or every subcontractor that might be sending real mail with the domain in the From: header. The domain owners use the reports to do things like update SPF records to include all of the sending hosts, update server configs to add DKIM signatures, or to fix servers that are adding invalid signatures, and often to shut rogue servers down that shouldn't have been sending mail in the first place. I can't see how spam scores would be of any use for any of these tasks. R's, John ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Two new fields in aggregate reports
> Correct. However, it is an average, so a spammer would have a hard > time trying to work out the detail of how the receiver's score is > computed. [randomly] permute the input, seen 1000s of messages and look at how the score varies ... repeat ... ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Two new fields in aggregate reports
On Fri, Oct 25, 2019 at 3:52 AM Дилян Палаузов wrote: > > If it is a goal to reuse the dmarc-reporting mechanism to report also > about perceived spam probability, then it can be > discussed in more details how this can be achieved. My experience is, > that asking a provider, why an obviously non-spam > mail was evaluated as spam, virtually never leads to a useful answer. So > nobody wants to reveal how its spam system > weigths factors and if there is lack of such interest, extending the > report format will not help, as nobody will be > willing the report the data. > Especially when the answer more or less boils down to "lots of recipients treated messages like this as spam". There are so many squishy terms there that it is essentially meaningless. --Kurt ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Two new fields in aggregate reports
Hello Alessandro, I do not see how this helps for DMARC. An email either validates DMARC, or fails DMARC and the aggregate repors say per sending IP server (only direct mail flow is reported), whether DMARC validates or fails. With this information it is sufficient to determine, if the DMARC/DKIM implementations on sender and receiver are either both bug-free, or both have the same bugs. I do not see, how the information you ask to add, while interesting, does help DMARC. What is the purposes of the aggregate and non-aggregate reports? What are non-goals? I asked several times here, nobody answered. Perhaps a discussion on the goals and non-goal would help. If it is a goal to reuse the dmarc-reporting mechanism to report also about perceived spam probability, then it can be discussed in more details how this can be achieved. My experience is, that asking a provider, why an obviously non-spam mail was evaluated as spam, virtually never leads to a useful answer. So nobody wants to reveal how its spam system weigths factors and if there is lack of such interest, extending the report format will not help, as nobody will be willing the report the data. Exchanging information on hard-coded rules in Spam-Assassing (IP reputation, HTML mime part without text/plain, the “Nigeria money” phrase), that is not based on filter training, does not help neither, as sender can run the tests on its own and predict how the recipient will evaluate these set of criteria. Regards Дилян On Thu, 2019-10-24 at 19:53 +0200, Alessandro Vesely wrote: > Hi all, > > it is difficult to tell what is each aggregate report's record. It is easier > if the source IP is known. Mailing lists can be told by their (unaligned) SPF > domain. Otherwise, it is difficult to tell abuse from legitimate users using > the wrong server. > > Getting a failure report for each source IP is not easy, because few mailbox > providers send failure reports. > > In order to ease the understanding of aggregate reports, I propose two > additional per-record fields: > > > *score*: The average score of the messages in the row; let's say an SA-like > number (< 0 good, > 10 bad, values in between may be worth human inspection). > > *list*: An enumerated type, for example "none", "black", "white", "both", > indicating if the source IP is listed in some public or private DNSxL that the > reporting MTA uses. > > > They're obviously subjective stuff. However, most MTAs deploy at least one of > them, and summing up per-IP results every day can bring useful indications. > > I haven't added those fields to http://bit.ly/dmarc-rpt-schema, yet. Let's > discuss. I hope they will make it to rfc7489bis. > > > Best > Ale ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
Re: [dmarc-ietf] Two new fields in aggregate reports
On Fri 25/Oct/2019 02:53:32 +0200 Brandon Long wrote: > On Thu, Oct 24, 2019 at 10:55 AM Alessandro Vesely wrote: >> >> In order to ease the understanding of aggregate reports, I propose two >> additional per-record fields:>> >> >> *score*: The average score of the messages in the row; let's say an >> SA-like number (< 0 good, > 10 bad, values in between may be worth human >> inspection).>> > > This assumes that your spam system is able to make a spamminess score that > approximates a continuum, I'm not sure that's true. The spam score is a rather common concept. Of course, an MTA which does not implement any such thing would omit this field. > Also, it's a lot of spamminess feedback, not sure if that can be abused or > not. Correct. However, it is an average, so a spammer would have a hard time trying to work out the detail of how the receiver's score is computed. Again, a reporter may compute the std deviation along with the average and omit this field if all messages have equal score. OTOH, this is indeed a valuable feedback. Mail sites could build their own reputation system based on that. A community IP database. Revolutionary, isn't it? > Also, as rejected at smtp time, especially for DMARC, we're unlikely to have > run a full spam assessment on the message. Perhaps we could add a second count. Or break the record into two rows. >> *list*: An enumerated type, for example "none", "black", "white", >> "both", indicating if the source IP is listed in some public or private >> DNSxL that the reporting MTA uses.>> > > At first I was wondering if you were going to have the values of the > list-id header. Ah, that way we'd get rid of all those "on behalf of"... :-) > That said, though I've never used any DMARC aggregator services, I would > think that one of the values they're likely to provide is IP address > identification, useful so you can hunt down what things aren't sending DMARC > compliant email yet (ie, looks like you're sending mail from ESP A and ESP > B, fix those)... Exactly. > they can just as easily use the various public IP reputation sources. Yes, could do, but it's more difficult. Here the reporting MTA already has the datum, and can include it for free. This datum would be vouched by the reporting MTA. It can be internal, like a fail2ban database. How about also reporting the source, if external? > Anyways, I doubt we'd expose anything internally... I always wonder why Google doesn't publish their internal IP database. > also, this is kind of the opposite of the spamminess score, in that it > expects that things are black & white, and we virtually never classify > things as such. The aim is to be able to look at an aggregate report and understand it. For example, if people can tell an IP is blackish, they'd just smile and be happy for a good failure case. What field would you suggest instead? Best Ale -- ___ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
