Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-18 Thread Joseph Brennan


--On October 18, 2016 at 02:06:38 -0400 Ruga  wrote:
> 
> >
 

... unless you're applying DMARC, which says the "From:" should instead
"align" with something other than the author of the message in some cases.

--Joseph Brennan







Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-18 Thread Dianne Skoll
On Tue, 18 Oct 2016 02:06:38 -0400
Ruga  wrote:

> < does not belong to the author(s) of the message.>>

A Quoted-String phrase is NOT a mailbox.  It's just a quoted string
that is not subject to any further interpretation.

Regards,

Dianne.


Re: The real spoofing issue

2016-10-18 Thread Ralph Seichter
On 18.10.16 00:52, Ruga wrote:

> https://tools.ietf.org/html/rfc5322#section-3.6.2

The header line

  From: "John Doe " 

does not violate the RFC section you linked. It may be unusual, and you
are of course free to personally (!) use it as a spam indicator, but it
is definitely RFC-compliant, so don't try to tell people otherwise.

-Ralph



Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-18 Thread Dianne Skoll


On October 18, 2016 2:09:37 AM EDT, Ruga  wrote:
>RFC 2822 and 5322 are in the "Standards Track".
>RFC 822 is still the standard.

Interesting, but the example is still RFC-compliant, even with 822.

Regards, 

Dianne. 



Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-18 Thread Dianne Skoll


On October 18, 2016 2:27:09 AM EDT, Ruga  wrote:
>Yes, you can prefix a quoted string to the actual address. No, the
>quoted string is not part of the address.

Indeed.

>There are two approaches here: one is to defend the spammer's abuse of
>the standard (intended to trick the average Joe into believing they
>have received mail from someone else), and the other is to read the
>standard

I think you are the one with reading comprehension problems if you are still 
implying my example violates the standard.

Regards,

Dianne.



Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-18 Thread Paul Stead

The following rules look for a From label which looks to have an email address 
looks for this type of spoofed address

The following would be valid, for example:

From: "p...@domain.com" 


http://ruleqa.spamassassin.org/20161017-r1765221-n/T_PDS_FROM_2_EMAILS/detail

http://ruleqa.spamassassin.org/20161017-r1765221-n/T_FROM_2_EMAILS/detail - 
similar to above with less metas

They both seem to hit more ham than spam on the Corpus


Paul

On 18/10/16 07:27, Ruga wrote:
Yes, you can prefix a quoted string to the actual address. No, the quoted 
string is not part of the address.

There are two approaches here: one is to defend the spammer's abuse of the 
standard (intended to trick the average Joe into believing they have received 
mail from someone else), and the other is to read the standard


On Tue, Oct 18, 2016 at 4:02 AM, Dianne Skoll 
<'d...@roaringpenguin.com'> wrote:
On Mon, 17 Oct 2016 19:11:29 -0400
Ruga  wrote:


rfc 822 (the actual standard):


Which as I mentioned is obsolete, but I'll play with you...


authentic = "From" ":" mailbox ; Single author / ...
mailbox = addr-spec ; simple address / phrase route-addr
addr-spec = local-part "@" domain


And you left out the BNF of "phrase", didn't you? Tsk tsk!

You can't pick and choose pieces of RFCs, you know. They come as a package
deal.

TL;DR, the header:

From: "Dianne Skoll " 


is absolutely compliant with RFC-822 and its successors, RFC-2822 and
RFC-5322.

Regards,

Dianne.

--
Paul Stead
Systems Engineer
Zen Internet


Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-18 Thread Ruga
Yes, you can prefix a quoted string to the actual address. No, the quoted 
string is not part of the address.

There are two approaches here: one is to defend the spammer's abuse of the 
standard (intended to trick the average Joe into believing they have received 
mail from someone else), and the other is to read the standard


On Tue, Oct 18, 2016 at 4:02 AM, Dianne Skoll <'d...@roaringpenguin.com'> wrote:
On Mon, 17 Oct 2016 19:11:29 -0400
Ruga  wrote:

> rfc 822 (the actual standard):

Which as I mentioned is obsolete, but I'll play with you...

> authentic = "From" ":" mailbox ; Single author / ...
> mailbox = addr-spec ; simple address / phrase route-addr
> addr-spec = local-part "@" domain

And you left out the BNF of "phrase", didn't you? Tsk tsk!

You can't pick and choose pieces of RFCs, you know. They come as a package
deal.

TL;DR, the header:

From: "Dianne Skoll " 

is absolutely compliant with RFC-822 and its successors, RFC-2822 and
RFC-5322.

Regards,

Dianne.

Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-18 Thread Ruga
RFC 2822 and 5322 are in the "Standards Track".
RFC 822 is still the standard.


On Tue, Oct 18, 2016 at 2:52 AM, Dianne Skoll <'d...@roaringpenguin.com'> wrote:
On October 17, 2016 7:11:29 PM EDT, Ruga  wrote:
>rfc 822 (the actual standard):

Are you serious? RFC 822 is decades obsolete, long since superseded by 2822 and 
then by 5322.

Regards,

Dianne.

Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-18 Thread Ruga
<>


On Tue, Oct 18, 2016 at 1:25 AM, Paul Stead <'paul.st...@zeninternet.co.uk'> 
wrote:




On 17/10/16 23:52, Ruga wrote:

https://tools.ietf.org/html/rfc5322#section-3.6.2
from = "From:" mailbox-list CRLF ... 
https://tools.ietf.org/html/rfc5322#section-3.4 ... ---8<--- mailbox = 
name-addr / addr-spec name-addr = [display-name] angle-addr display-name = 
phrase mailbox-list = (mailbox *("," mailbox)) / obs-mbox-list Normally, a 
mailbox is composed of two parts: (1) an optional display name that indicates 
the name of the recipient (which can be a person or a system) that could be 
displayed to the user of a mail application, and (2) an addr-spec address 
enclosed in angle brackets ("<" and ">"). There is an alternate simple form of 
a mailbox where the addr-spec address appears alone, without the recipient's 
name or the angle brackets. The Internet addr-spec address is described in 
[section 3.4.1](https://tools.ietf.org/html/rfc5322#section-3.4.1).
--
Paul Stead
Systems Engineer
Zen Internet

Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-17 Thread Dianne Skoll
On Mon, 17 Oct 2016 19:11:29 -0400
Ruga  wrote:

> rfc 822 (the actual standard):

Which as I mentioned is obsolete, but I'll play with you...

> authentic = "From" ":" mailbox ; Single author / ...
> mailbox = addr-spec ; simple address  / phrase route-addr
> addr-spec = local-part "@" domain

And you left out the BNF of "phrase", didn't you?  Tsk tsk!

You can't pick and choose pieces of RFCs, you know.  They come as a package
deal.

TL;DR, the header:

   From:  "Dianne Skoll " 

is absolutely compliant with RFC-822 and its successors, RFC-2822 and
RFC-5322.

Regards,

Dianne.


Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-17 Thread Dianne Skoll


On October 17, 2016 7:11:29 PM EDT, Ruga  wrote:
>rfc 822 (the actual standard):

Are you serious?  RFC 822 is decades obsolete, long since superseded by 2822 
and then by 5322.

Regards,

Dianne.



Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-17 Thread Paul Stead



On 17/10/16 23:52, Ruga wrote:

https://tools.ietf.org/html/rfc5322#section-3.6.2



  from=   "From:" mailbox-list CRLF

...
https://tools.ietf.org/html/rfc5322#section-3.4
...

---8<---
  mailbox =   name-addr / addr-spec

  name-addr   =   [display-name] angle-addr

  display-name=   phrase

  mailbox-list=   (mailbox *("," mailbox)) / obs-mbox-list


  Normally, a mailbox is composed of two parts: (1) an optional display
  name that indicates the name of the recipient (which can be a person
  or a system) that could be displayed to the user of a mail
  application, and (2) an addr-spec address enclosed in angle brackets
  ("<" and ">").  There is an alternate simple form of a mailbox where
  the addr-spec address appears alone, without the recipient's name or
  the angle brackets.  The Internet addr-spec address is described in
  section 3.4.1.

--
Paul Stead
Systems Engineer
Zen Internet


Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-17 Thread Ruga
rfc 822 (the actual standard):

authentic = "From" ":" mailbox ; Single author / ...
mailbox = addr-spec ; simple address  / phrase route-addr
addr-spec = local-part "@" domain



On Tue, Oct 18, 2016 at 12:52 AM, Ruga <'r...@protonmail.com'> wrote:

https://tools.ietf.org/html/rfc5322#section-3.6.2




On Mon, Oct 17, 2016 at 2:18 AM, Dianne Skoll <'d...@roaringpenguin.com'> wrote:
On Sun, 16 Oct 2016 18:08:20 -0400
Ruga  wrote:

> In my servers, the above string is not RFC compliant,
> and therefore the whole mail is automatically
> rejected as SPAM.

Your servers fail in RFC comprehension. The message header:

From: "Dianne Skoll " 

is absolutely 100% RFC-compliant.

If you feel it is not, please cite the RFC that's violated, including
the specific section being violated.

Regards,

Dianne.

Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-17 Thread Ruga
https://tools.ietf.org/html/rfc5322#section-3.6.2




On Mon, Oct 17, 2016 at 2:18 AM, Dianne Skoll <'d...@roaringpenguin.com'> wrote:
On Sun, 16 Oct 2016 18:08:20 -0400
Ruga  wrote:

> In my servers, the above string is not RFC compliant,
> and therefore the whole mail is automatically
> rejected as SPAM.

Your servers fail in RFC comprehension. The message header:

From: "Dianne Skoll " 

is absolutely 100% RFC-compliant.

If you feel it is not, please cite the RFC that's violated, including
the specific section being violated.

Regards,

Dianne.

Re: The real spoofing issue

2016-10-17 Thread RW
On Mon, 17 Oct 2016 16:30:43 +0200
Ralph Seichter wrote:

> On 17.10.16 15:45, RW wrote:
> 
> > Most of what SpamAssassin targets is RFC compliant. It would be
> > perfectly legitimate to score bogus addresses in the display name
> > if it proved useful.  
> 
> With "useful" being open to interpretation.

As with everything, "useful" comes from rule QA and feedback.

> Some of my customers
> are willing to accept a much higher degree of potential spam than
> others, to ensure that legitimate mail is less likely to be weeded
> out. Still, as long as the default SA scores are zero (or close to
> zero) it might be feasible to check if the decoded From-Header
> contains mismatching e-mail addresses. It could be a spoof attempt,
> it could be misconfigured software, but it could also be legitimate.

I'm not saying that it should be done, in my experience spammers don't
usually put email addresses there. My point is that RFC compliance is
irrelevant.


Re: The real spoofing issue

2016-10-17 Thread Ralph Seichter
On 17.10.16 15:45, RW wrote:

> Most of what SpamAssassin targets is RFC compliant. It would be
> perfectly legitimate to score bogus addresses in the display name
> if it proved useful.

With "useful" being open to interpretation. ;-) Some of my customers are
willing to accept a much higher degree of potential spam than others, to
ensure that legitimate mail is less likely to be weeded out. Still, as
long as the default SA scores are zero (or close to zero) it might be
feasible to check if the decoded From-Header contains mismatching e-mail
addresses. It could be a spoof attempt, it could be misconfigured
software, but it could also be legitimate.

-Ralph


Re: The real spoofing issue

2016-10-17 Thread Dianne Skoll
On Mon, 17 Oct 2016 14:45:11 +0100
RW  wrote:

> On Mon, 17 Oct 2016 15:20:27 +0200
> Ralph Seichter wrote:

> >   From: "John Doe " 

> > is perfectly legitimate. 

> but an unusual and rather silly thing to do.

As I mentioned, Yahoo Groups did something like this last time I checked.
They did it in order not to break DMARC, but still make the original sender
address visible.

> Most of what SpamAssassin targets is RFC compliant. It would be
> perfectly legitimate to score bogus addresses in the display name if
> it proved useful.

Yes, and spammers would move on to something like:

From: =?UTF-8?Q?John=20Doe=20=3Cjohn=E2=80=8B=40=E2=80=8Bdoe.org=3E?= 


To answer the obvious question, (0xE2 0x80 0x8B) is UTF-8 for a
zero-width space, meaning the mail reader would display an apparent
email address but no sane parser would extract an email address.
Making a parser that could cope with all the tricks in the Unicode
toolbox would be very hard.

Regards,

Dianne.


Re: The real spoofing issue

2016-10-17 Thread RW
On Mon, 17 Oct 2016 15:20:27 +0200
Ralph Seichter wrote:

> On 17.10.16 02:38, Benny Pedersen wrote:
> 
> > one could argue if From:Name and From:Addr have differing domains
> > its forged ?  
> 
> Which RFC defines "From:Name" and "From:Addr" (I don't see the terms
> in RFC5322), and where does it say that domain names must match if
> they are present? The header line
> 
>   From: "John Doe " 
> 
> is perfectly legitimate. 

but an unusual and rather silly thing to do.

> Guessing whether or not domains are related
> is not something I'd like a piece of software to do.

Most of what SpamAssassin targets is RFC compliant. It would be
perfectly legitimate to score bogus addresses in the display name if
it proved useful.


Re: The real spoofing issue

2016-10-17 Thread Ralph Seichter
On 17.10.16 02:38, Benny Pedersen wrote:

> one could argue if From:Name and From:Addr have differing domains its
> forged ?

Which RFC defines "From:Name" and "From:Addr" (I don't see the terms in
RFC5322), and where does it say that domain names must match if they are
present? The header line

  From: "John Doe " 

is perfectly legitimate. Guessing whether or not domains are related is
not something I'd like a piece of software to do.

-Ralph


Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-16 Thread Dianne Skoll
>one could argue if From:Name and From:Addr have differing domains its 
>forged ?

One could argue that, but one could not argue that my sample From: header is 
not RFC-compliant.

Last I checked, Yahoo Groups rewrote the From: header in exactly that manner.

Furthermore, the Quoted-String part of the header can be almost anything you 
like, so there's no concept of From:Name having a domain.  I leave it as an 
exercise for the reader to devise a From:Name that sort of looks like it 
contains an email address but is unparseable as such.

Regards, 

Dianne


Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-16 Thread Benny Pedersen

On 2016-10-17 02:18, Dianne Skoll wrote:


From: "Dianne Skoll " 

is absolutely 100% RFC-compliant.


lets break test it :)


If you feel it is not, please cite the RFC that's violated, including
the specific section being violated.


one could argue if From:Name and From:Addr have differing domains its 
forged ?


Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-16 Thread Bill Cole

On 16 Oct 2016, at 18:08, Ruga wrote:


From: "Dianne Skoll " 


In my servers, the above string is not RFC compliant,


Are you writing your own RFC's? That's cool: the IETF could do with some 
competition. Where are you publishing them and accepting comments?


The IETF's RFC5322 includes this ABNF ultimately specifying the From 
header:


   local-part  =   dot-atom / quoted-string / obs-local-part

   domain  =   dot-atom / domain-literal / obs-domain

   domain-literal  =   [CFWS] "[" *([FWS] dtext) [FWS] "]" [CFWS]

   dtext   =   %d33-90 /  ; Printable US-ASCII
   %d94-126 / ;  characters not including
   obs-dtext  ;  "[", "]", or "\"

   qtext   =   %d33 / ; Printable US-ASCII
   %d35-91 /  ;  characters not including
   %d93-126 / ;  "\" or the quote character
   obs-qtext

   quoted-pair =   ("\" (VCHAR / WSP)) / obs-qp

   qcontent=   qtext / quoted-pair

   quoted-string   =   [CFWS]
   DQUOTE *([FWS] qcontent) [FWS] DQUOTE
   [CFWS]

   word=   atom / quoted-string


   phrase  =   1*word / obs-phrase

   display-name=   phrase

   angle-addr  =   [CFWS] "<" addr-spec ">" [CFWS] /
   obs-angle-addr

   name-addr   =   [display-name] angle-addr

   addr-spec   =   local-part "@" domain

   mailbox =   name-addr / addr-spec

   mailbox-list=   (mailbox *("," mailbox)) / obs-mbox-list

   from=   "From:" mailbox-list CRLF

It seems to me that Dianne's example is entirely legal as a From header, 
essentially because you can put almost anything inside a quoted-string. 
You'll note that I've left out all the still-legal 'obs-*' component 
specs because you do not need them in this case. Note that RFC5322 did 
not expand what is allowed in a From header.


Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-16 Thread Dianne Skoll
On Sun, 16 Oct 2016 18:08:20 -0400
Ruga  wrote:

> In my servers, the above string is not RFC compliant,
> and therefore the whole mail is automatically
> rejected as SPAM.

Your servers fail in RFC comprehension.  The message header:

   From: "Dianne Skoll " 

is absolutely 100% RFC-compliant.

If you feel it is not, please cite the RFC that's violated, including
the specific section being violated.

Regards,

Dianne.



Re: The real spoofing issue

2016-10-16 Thread Antony Stone
On Monday 17 October 2016 at 00:08:20, Ruga wrote:

> > From: "Dianne Skoll " 
> 
> In my servers, the above string is not RFC compliant,
> and therefore the whole mail is automatically
> rejected as SPAM.

I think you misunderstood.

The suggestion was not that email should be sent with this as the sender field.

The suggestion was that email clients should display the sender thus, when 
there is a mismatch between the envelope From field and the email header From 
field.


Antony.

-- 
"I find the whole business of religion profoundly interesting.  But it does 
mystify me that otherwise intelligent people take it seriously."

 - Douglas Adams

   Please reply to the list;
 please *don't* CC me.


Re: The real spoofing issue (was Re: How to get spam assassin to detect spoofed mails as SPF is clearly useless)

2016-10-16 Thread Ruga
> From: "Dianne Skoll " 


In my servers, the above string is not RFC compliant,
and therefore the whole mail is automatically
rejected as SPAM.

Re: The real spoofing issue

2016-10-16 Thread Antony Stone
On Sunday 16 October 2016 at 17:30:19, Dianne Skoll wrote:

> Oh, and one more thing...
> 
> ... every email reader I've ever used shows neither the From: address nor
> the envelope sender by default.  They all default to showing the full name
> on the From: line, which naturally is impossible to protect or verify.

Indeed.  This is bad (but understandable, since (a) companies like Microsoft 
and Apple try to hide "all this technical stuff" from poor bewildered users, 
and (b) the majority of users would not know what to make of it anyway, unless 
the discrepancy were pointed out to them in no uncertain terms).

The growth of email on mobile devices with limited screen pixels doesn't help, 
either, since this encourages developers of web-based or client-based email 
systems to reduce the amount of detail shown to the user.

> I would love for mail readers to display this in the sender column:
> 
>Dianne Skoll  [spammer.org]

Yes.  Users seem (I don't really know, I've not done a survey, but I hope?) to 
have understood the "green locked padlock" / "red unlocked padlock" symbols in 
their browser address lines these days, for good and bad SSL connections, so 
surely something similar should be possible with email addresses?

> However, the DMARC people said UI design was not in DMARC's scope.  Meh.

"Meh" indeed, however I wonder if either:

 - some significant mail client provider (Thunderbird?) could be persuaded to 
implement this as a feature, and see whether others catch on and copy it

 - it could be included in some project such as MailScanner, which has access 
to the entire email content, and can also modify anything it likes, 
specifically including the subject, to highlight suspicious senders of the sort 
you've outlined above ?

I know there would be false positives from such a system (neatly summarised in 
a couple of your earlier postings in this thread), however these could be 
dealt with in the "local configuration" aspect of running any mail server, 
where the "trusted=doubtful" flag could be converted to "trusted=yes" for 
specific cases.

> (And I'm not even convinced that would offer much protection... end-users
> are wonderful at ignoring red flags.)

Yes, well, that's a well-known and well-documented problem outside the scope 
of any RFC, technical solution, system admin, or psychologist :(


Antony.

-- 
Perfection in design is achieved not when there is nothing left to add, but 
rather when there is nothing left to take away.

 - Antoine de Saint-Exupery

   Please reply to the list;
 please *don't* CC me.