Re: Today's Google Docs phish

[Alex]

Hi,

>> >>> I found a local version which maybe did the trick
>> >>>
>> >>> redirector_pattern
>> >>>
>> >>> m'^https?:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/url\?.*?(?<=[?&])q=(.*?)(?:$|[&\#])'i
>> >>
>
>> Yes, but I don't understand how that equates to an eventual score.
>
> I haven't used these, but by the look of it it's trying to identify
> the encoded URI so that other rules can see it.
>
>> Perhaps I don't understand regex's well enough, but I don't understand
>> what it does with the redirector site portion and the target portion.
>
> In rules you often see ?: used in brackets to prevent the matching
> text being captured as this makes it more efficient, e.g. (?:\w+\.)
> in the above. Towards the end of the pattern is  (.*?) which is
> used to capture the target uri.
>
> .*? is like .*, but matches the shortest run of characters it can,
> rather than the longest.

Am I understanding correctly that redirector_pattern breaks up the one
encoded URI into multiple URIs that are available for rules to be
written using them, instead of ?

In other words, if I were to write a uri rule that includes
www.googleapis.com, it would match in this case? How does it differ
had I not had a redirector pattern and just wrote a rule matching the
pattern directly?

May  5 21:32:22.768 [5533] dbg: uri: parsed uri found:
https://www.googleapis.com/auth/contacts=false_granted_scopes=true_type=token_uri=https://googledocs.g-docs.pro/g.php=customparam
in hard-coded redirector

Initially I thought it was hitting the redirector_pattern supplied by
Axb a few days ago, but it looks like it matches one of the patterns
included in 72_scores.cf already.

Re: DKIM_VALID EnvelopeFrom

[RW]

On Fri, 5 May 2017 22:49:43 +
David Jones wrote:

> From: RW 
>     
> >On Fri, 5 May 2017 19:56:27 +
> >David Jones wrote:  
> 
> >> >I don't seen why anyone one would want a form of whitelisting
> >> >where a DKIM pass on a trusted domain would be ignored if there's
> >> >no SPF pass.    
> >> 
> >> Correct.    
> 
> >I don't know why you write  "correct" and then go on to write
> >something contrary.  
> 
> It's not a contradiction.  See below.

If you think it isn't you have read it correctly.
 
> >>This is why I only add envelope-from domains to my
> >> whitelist_auth list that is currently 2,595 entries.  
> 
> 
> >That's not a good idea. When you don't feel you can just put a
> >"header from" domain into whitelist_auth,  you should use one or
> >both of whitelist_from_dkim and whitelist_from_spf instead.  
> 
> Both of those are effectively the same when you carefully add only
> envelope-from domains with specific patterns.

There are only two possibilities either the header and envelope domains
are the same in which case it makes no difference, or they are not,
in which case you are giving up on DKIM and relying only on SPF.

Re: DKIM_VALID EnvelopeFrom

[RW]

On Sat, 6 May 2017 00:32:22 +0200
Reindl Harald wrote:

> Am 06.05.2017 um 00:15 schrieb RW:
> >> This is why I only add envelope-from domains to my
> >> whitelist_auth list that is currently 2,595 entries.  
> > 
> > 
> > That's not a good idea. When you don't feel you can just put a
> > "header from" domain into whitelist_auth,  you should use one or
> > both of whitelist_from_dkim and whitelist_from_spf instead  
> 
> whitelist_auth *IS* one or both



whitelist_auth is not the same as using just whitelist_from_dkim and
it's not the same as using separate  whitelist_from_dkim and
whitelist_from_spf entries when the "envelope from" domain is different
to the "header from" domain in the email you wish to whitelist.

Re: DKIM_VALID EnvelopeFrom

[David Jones]

From: RW 
    
>On Fri, 5 May 2017 19:56:27 +
>David Jones wrote:

>> >I don't seen why anyone one would want a form of whitelisting where a
>> >DKIM pass on a trusted domain would be ignored if there's no SPF
>> >pass.  
>> 
>> Correct.  

>I don't know why you write  "correct" and then go on to write something
>contrary.

It's not a contradiction.  See below.

>>This is why I only add envelope-from domains to my
>> whitelist_auth list that is currently 2,595 entries.


>That's not a good idea. When you don't feel you can just put a "header
>from" domain into whitelist_auth,  you should use one or both of
>whitelist_from_dkim and whitelist_from_spf instead.

Both of those are effectively the same when you carefully add only
envelope-from domains with specific patterns.  If they passed DKIM
signing for these entries I am adding then the domain owner has lost
control of their DNS and some bad guys are adding DKIM records which
would be highly unlikely.  I don't think bad guys are going to setup perfect
DKIM on a highjacked DNS server/hosting.  I guess it's possible if some
DKIM private keys got loose and spammers start using them.

I have been doing this for a couple of years now and it works very well
in my environment.  I have had to remove 1 or 2 entries over the past
few years based on compromised accounts which showed me not to
whitelist_auth certain domains.

Dave

Re: DKIM_VALID EnvelopeFrom

[David Jones]

>From: Matus UHLAR - fantomas 
    
 On Fri, 5 May 2017, David Jones wrote:

> I think I would have to write a simple SA plugin to compare the
> envelope-from with the DKIM signature domain to see if they matched
> then I could use a meta rule to glue all of this together.

>>>John Hardin skrev den 2017-05-05 21:45:
 Or file a bug to get it implemented in the base DKIM plugin. I suspect
 extending that would be easier (and neater in the long run) than a
 parallel plugin for just that one DKIM check.

>>>From: Benny Pedersen 
>>>http://search.cpan.org/dist/Mail-DMARC/
>>
>>>who will make the missing sa plugin to it ?

>On 05.05.17 20:22, David Jones wrote:
>>I just filed a bug per John's recommendation but I think it
>>would be best to put that logic into a DMARC plugin since
>>this is getting into what DMARC does.

>agreed but there's still one thing I don't understand:

>If a mail is DKIM-signed, it means that it's authenticated, including
>headers like From:.

Authentication and authorization are very different things.

>what's the point of checking if SPF and DKIM domains match?
>This way authentic (but forwarded, e.g. through mailing lists) mail will get
>"caught" but what's the poit of it?

DKIM signing only does authentication to prevent tampering with the
body and headers.  It doesn't have to do with authorization that like
SPF does.  Both authentication and authorization are needed to prove
an email is from who it claims to be and not altered.

Of course a compromised mail account can send both an authorized
and authenticated email with malicious content.  You don't want to
whitelist_auth domains with real user accounts that can be compromised.

Re: DKIM_VALID EnvelopeFrom

[RW]

On Fri, 5 May 2017 19:56:27 +
David Jones wrote:


> >Alignment of the two from address is needed in DMARC so that SPF can
> >match on the same domain that the MUA displays (if it even does). It
> >doesn't do anything for DKIM.   
> 
> Did you read that returnpath.com link above about DMARC passing if
> SPF or DKIM passes and are aligned?  They know what they are doing
> and I have seen this to be true in my own inbound mail based on
> OpenDMARC headers.

I don't doubt that *they* know what they are doing. That article gives
reasons to have both on outgoing mail, but has no argument at all in
favour of requiring both to verify incoming mail.


> >I don't seen why anyone one would want a form of whitelisting where a
> >DKIM pass on a trusted domain would be ignored if there's no SPF
> >pass.  
> 
> Correct.  

I don't know why you write  "correct" and then go on to write something
contrary.


>This is why I only add envelope-from domains to my
> whitelist_auth list that is currently 2,595 entries.


That's not a good idea. When you don't feel you can just put a "header
from" domain into whitelist_auth,  you should use one or both of
whitelist_from_dkim and whitelist_from_spf instead.

Re: DKIM_VALID EnvelopeFrom

[Matus UHLAR - fantomas]

On Fri, 5 May 2017, David Jones wrote:


I think I would have to write a simple SA plugin to compare the
envelope-from with the DKIM signature domain to see if they matched
then I could use a meta rule to glue all of this together.



John Hardin skrev den 2017-05-05 21:45:

Or file a bug to get it implemented in the base DKIM plugin. I suspect
extending that would be easier (and neater in the long run) than a
parallel plugin for just that one DKIM check.



From: Benny Pedersen 
http://search.cpan.org/dist/Mail-DMARC/



who will make the missing sa plugin to it ?


On 05.05.17 20:22, David Jones wrote:

I just filed a bug per John's recommendation but I think it
would be best to put that logic into a DMARC plugin since
this is getting into what DMARC does.


agreed but there's still one thing I don't understand:

If a mail is DKIM-signed, it means that it's authenticated, including
headers like From:.

what's the point of checking if SPF and DKIM domains match?
This way authentic (but forwarded, e.g. through mailing lists) mail will get
"caught" but what's the poit of it?

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
   One OS to rule them all, One OS to find them, 
One OS to bring them all and into darkness bind them 

Re: DKIM_VALID EnvelopeFrom

[David Jones]

>From: Benny Pedersen 
    
>John Hardin skrev den 2017-05-05 21:45:
>> On Fri, 5 May 2017, David Jones wrote:
>> 
>>> I think I would have to write a simple SA plugin to compare the 
>>> envelope-from with the DKIM signature domain to see if they matched 
>>> then I could use a meta rule to glue all of this together.
>> 
>> Or file a bug to get it implemented in the base DKIM plugin. I suspect
>> extending that would be easier (and neater in the long run) than a
>> parallel plugin for just that one DKIM check.

>http://search.cpan.org/dist/Mail-DMARC/

>who will make the missing sa plugin to it ?

I just filed a bug per John's recommendation but I think it
would be best to put that logic into a DMARC plugin since
this is getting into what DMARC does.

Dave

Re: DKIM_VALID EnvelopeFrom

[Benny Pedersen]

John Hardin skrev den 2017-05-05 21:45:

On Fri, 5 May 2017, David Jones wrote:

I think I would have to write a simple SA plugin to compare the 
envelope-from with the DKIM signature domain to see if they matched 
then I could use a meta rule to glue all of this together.


Or file a bug to get it implemented in the base DKIM plugin. I suspect
extending that would be easier (and neater in the long run) than a
parallel plugin for just that one DKIM check.


http://search.cpan.org/dist/Mail-DMARC/

who will make the missing sa plugin to it ?

Re: DKIM_VALID EnvelopeFrom

[David Jones]

From: RW 
    
>On Fri, 5 May 2017 17:45:37 +
>David Jones wrote:

>> From: RW 
>>     
>> >On Fri, 5 May 2017 14:51:32 +
>> >David Jones wrote:  
>> 
>> >> >I know. I do not want to validate the envelope from with DKIM. I
>> >> >just want to know if the mail was DKIM-VALID signed by the DOMAIN
>> >> >used in the envelopefrom.    
>> >>   
>> >> >So the only thing I want with the envelop from is to extract the
>> >> >domain and test if the mail was DKIM signed (and valid) by that
>> >> >domain.    
>> >>   
>> >> >This tells me the envelope from is not some random spoofed
>> >> >address, but actually controlled by someone who handled the
>> >> >e-mail before it arrived at our mta.    
>> >> 
>> >> This actually would be a very useful rule/logic to add to SA:
>> >>   
>> >>https://blog.returnpath.com/why-passing-and-aligning-both-spf-and-dkim-is-key-to-email-deliverability/
>> >> 
>> 
>> >So what would be the point in running a separate DKIM test against
>> >the envelope if you are looking for alignment.  
>> 
>> I don't think this would be a separate DKIM test necessarily.  It
>> should be a combination of SPF_PASS + DKIM_VALID_AU + the
>> envelope-from matches the DKIM-signed domain.  This is basically
>> perfect DMARC alignment where the domain has "p=reject" and DMARC
>> would pass meaning the domain was not spoofed.

>Alignment of the two from address is needed in DMARC so that SPF can
>match on the same domain that the MUA displays (if it even does). It
>doesn't do anything for DKIM. 

Did you read that returnpath.com link above about DMARC passing if
SPF or DKIM passes and are aligned?  They know what they are doing
and I have seen this to be true in my own inbound mail based on
OpenDMARC headers.

>I don't seen why anyone one would want a form of whitelisting where a
>DKIM pass on a trusted domain would be ignored if there's no SPF
>pass.

Correct.  This is why I only add envelope-from domains to my
whitelist_auth list that is currently 2,595 entries.

Re: DKIM_VALID EnvelopeFrom

[John Hardin]

On Fri, 5 May 2017, David Jones wrote:

I think I would have to write a simple SA plugin to compare the 
envelope-from with the DKIM signature domain to see if they matched then 
I could use a meta rule to glue all of this together.


Or file a bug to get it implemented in the base DKIM plugin. I suspect 
extending that would be easier (and neater in the long run) than a 
parallel plugin for just that one DKIM check.



--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  We should endeavour to teach our children to be gun-proof
  rather than trying to design our guns to be child-proof
---
 3 days until the 72nd anniversary of VE day

Re: DKIM_VALID EnvelopeFrom

[RW]

On Fri, 5 May 2017 17:45:37 +
David Jones wrote:

> From: RW 
>     
> >On Fri, 5 May 2017 14:51:32 +
> >David Jones wrote:  
> 
> >> >I know. I do not want to validate the envelope from with DKIM. I
> >> >just want to know if the mail was DKIM-VALID signed by the DOMAIN
> >> >used in the envelopefrom.    
> >>   
> >> >So the only thing I want with the envelop from is to extract the
> >> >domain and test if the mail was DKIM signed (and valid) by that
> >> >domain.    
> >>   
> >> >This tells me the envelope from is not some random spoofed
> >> >address, but actually controlled by someone who handled the
> >> >e-mail before it arrived at our mta.    
> >> 
> >> This actually would be a very useful rule/logic to add to SA:
> >>  
> >> https://blog.returnpath.com/why-passing-and-aligning-both-spf-and-dkim-is-key-to-email-deliverability/
> >>   
> 
> >So what would be the point in running a separate DKIM test against
> >the envelope if you are looking for alignment.  
> 
> I don't think this would be a separate DKIM test necessarily.  It
> should be a combination of SPF_PASS + DKIM_VALID_AU + the
> envelope-from matches the DKIM-signed domain.  This is basically
> perfect DMARC alignment where the domain has "p=reject" and DMARC
> would pass meaning the domain was not spoofed.


Alignment of the two from address is needed in DMARC so that SPF can
match on the same domain that the MUA displays (if it even does). It
doesn't do anything for DKIM. 

I don't seen why anyone one would want a form of whitelisting where a
DKIM pass on a trusted domain would be ignored if there's no SPF
pass.  

Re: DKIM_VALID EnvelopeFrom

[RW]

On Fri, 5 May 2017 19:51:23 +0100
RW wrote:

Sorry, that was sent by accident.

Re: DKIM_VALID EnvelopeFrom

[RW]

On Fri, 5 May 2017 17:45:37 +
David Jones wrote:

> From: RW 
>     
> >On Fri, 5 May 2017 14:51:32 +
> >David Jones wrote:  
> 
> >> >I know. I do not want to validate the envelope from with DKIM. I
> >> >just want to know if the mail was DKIM-VALID signed by the DOMAIN
> >> >used in the envelopefrom.    
> >>   
> >> >So the only thing I want with the envelop from is to extract the
> >> >domain and test if the mail was DKIM signed (and valid) by that
> >> >domain.    
> >>   
> >> >This tells me the envelope from is not some random spoofed
> >> >address, but actually controlled by someone who handled the
> >> >e-mail before it arrived at our mta.    
> >> 
> >> This actually would be a very useful rule/logic to add to SA:
> >>  
> >> https://blog.returnpath.com/why-passing-and-aligning-both-spf-and-dkim-is-key-to-email-deliverability/
> >>   
> 
> >So what would be the point in running a separate DKIM test against
> >the envelope if you are looking for alignment.  
> 
> I don't think this would be a separate DKIM test necessarily.  It
> should be a combination of SPF_PASS + DKIM_VALID_AU + the
> envelope-from matches the DKIM-signed domain.  This is basically
> perfect DMARC alignment where the domain has "p=reject" and DMARC
> would pass meaning the domain was not spoofed.
> 
> >> When both align, it should be a very good candidate for
> >> whitelist_auth based on the sender domain reputation.  
> 
> >If it passes DKIM and the domain has a good reputation then what
> >difference would alignment make.  
> 
> Proper security in any context checks both authorization and
> authentication. This is SPF and DKIM respectively in the email
> filtering context.  Spammers can get control of a compromised account
> and send a valid DKIM-signed email through that email server that
> would pass SPF with an envelope-from of example.com and DKIM
> signature of example.net (or some domain they had DNS control of like
> paypa1.com).  If it passed DKIM_VALID_AU then the visible From:
> address in the recipient's mail client would show example.net or
> paypa1.com.
> 
> Would I trust example.com or example.net in the above scenario?  Which
> would be added to whitelist_auth?  The authorized email was from
> example.com but the authenticated email was from example.net.  The
> DMARC standard says that either SPF or DKIM has to pass for a DMARC
> pass based on that link above.  The point of that link is to align
> both for best delivery results.
> 
> I am just saying that it would be nice if SA had a rule that hit when
> both matched which is perfect DMARC alignment.  Today I am able to
> get close to this using OpenDMARC to add headers then with custom
> rules to add DMARC_NONE, DMARC_PASS, or DMARC_FAIL.  I think I would
> have to write a simple SA plugin to compare the envelope-from with
> the DKIM signature domain to see if they matched then I could use a
> meta rule to glue all of this together.
> 
> Dave

Re: DKIM_VALID EnvelopeFrom

[David Jones]

From: RW 
    
>On Fri, 5 May 2017 14:51:32 +
>David Jones wrote:

>> >I know. I do not want to validate the envelope from with DKIM. I
>> >just want to know if the mail was DKIM-VALID signed by the DOMAIN
>> >used in the envelopefrom.  
>> 
>> >So the only thing I want with the envelop from is to extract the
>> >domain and test if the mail was DKIM signed (and valid) by that
>> >domain.  
>> 
>> >This tells me the envelope from is not some random spoofed address,
>> >but actually controlled by someone who handled the e-mail before it
>> >arrived at our mta.  
>> 
>> This actually would be a very useful rule/logic to add to SA:
>>  
>> https://blog.returnpath.com/why-passing-and-aligning-both-spf-and-dkim-is-key-to-email-deliverability/

>So what would be the point in running a separate DKIM test against the
>envelope if you are looking for alignment.

I don't think this would be a separate DKIM test necessarily.  It should be a
combination of SPF_PASS + DKIM_VALID_AU + the envelope-from matches
the DKIM-signed domain.  This is basically perfect DMARC alignment where
the domain has "p=reject" and DMARC would pass meaning the domain was
not spoofed.

>> When both align, it should be a very good candidate for whitelist_auth
>> based on the sender domain reputation.

>If it passes DKIM and the domain has a good reputation then what
>difference would alignment make.

Proper security in any context checks both authorization and authentication.
This is SPF and DKIM respectively in the email filtering context.  Spammers
can get control of a compromised account and send a valid DKIM-signed email
through that email server that would pass SPF with an envelope-from of
example.com and DKIM signature of example.net (or some domain they had
DNS control of like paypa1.com).  If it passed DKIM_VALID_AU then the visible
From: address in the recipient's mail client would show example.net or
paypa1.com.

Would I trust example.com or example.net in the above scenario?  Which
would be added to whitelist_auth?  The authorized email was from
example.com but the authenticated email was from example.net.  The
DMARC standard says that either SPF or DKIM has to pass for a DMARC
pass based on that link above.  The point of that link is to align both for
best delivery results.

I am just saying that it would be nice if SA had a rule that hit when both
matched which is perfect DMARC alignment.  Today I am able to get close
to this using OpenDMARC to add headers then with custom rules to add
DMARC_NONE, DMARC_PASS, or DMARC_FAIL.  I think I would have to
write a simple SA plugin to compare the envelope-from with the DKIM
signature domain to see if they matched then I could use a meta rule
to glue all of this together.

Dave

Re: DKIM_VALID EnvelopeFrom

[RW]

On Fri, 5 May 2017 14:51:32 +
David Jones wrote:

> >I know. I do not want to validate the envelope from with DKIM. I
> >just want to know if the mail was DKIM-VALID signed by the DOMAIN
> >used in the envelopefrom.  
> 
> >So the only thing I want with the envelop from is to extract the
> >domain and test if the mail was DKIM signed (and valid) by that
> >domain.  
> 
> >This tells me the envelope from is not some random spoofed address,
> >but actually controlled by someone who handled the e-mail before it
> >arrived at our mta.  
> 
> This actually would be a very useful rule/logic to add to SA:
> https://blog.returnpath.com/why-passing-and-aligning-both-spf-and-dkim-is-key-to-email-deliverability/

So what would be the point in running a separate DKIM test against the
envelope if you are looking for alignment.

> When both align, it should be a very good candidate for whitelist_auth
> based on the sender domain reputation.

If it passes DKIM and the domain has a good reputation then what
difference would alignment make. 

Re: Razor FP on simple http link (by itself)

[RW]

On Fri, 5 May 2017 11:37:38 -0400
Rob McEwen wrote:


> Does RAZOR extract domains from links and checks them against a bad 
> domain database... sort of how SURBL works... and/or check the IP
> that they resolve to? (I don't think so, but now I have to ask just
> to be sure!)
> 
> If not... this seems to go beyond checksum-checking of parts of a 
> message - this seems much more surgical/specific than that.
> 
> Don't get me wrong... I'm a big fan of razor and of other 
> checksum-technologies. But I'm sort of shaken by this because I
> always thought a FP for razor would be much more difficult due to
> larger portions of a message having to match a checksum match in
> order to have a hit. (sort of like a larger "fingerprint" that is not
> easily duplicated in another innocent message, allegedly making FPs
> practically impossible)

razor2 supports multiple hash engines, but currently only engine 8 is
used. This is based on a hash of URI domain name and message size in
multiples of (I think) 100 bytes.

"Google Docs" message

[Joseph Brennan]

Below, this is the non-standard Received header that was in the message
that caused a
UNPARSEABLE_RELAY match. I am splitting it into 2 lines here for
readability.

Received: from 946634442539 named unknown by
gmailapi.google.com with HTTPREST; Wed, 3 May 2017 11:47:36 -0700

This Received header appears to be the format used in a message sent by an
app using oauth to send as a user. If so, legit use is rare. For a short
time I had Mimedefang logging matches on the pattern  /gmailapi.google.com
with HTTPREST/ and saw only a few.



The link in the html part started as follows:

https://accounts.google.com/o/oauth2/auth?client_id=

I don't have samples of legit mail to compare this to. It strikes me as odd
that mail sent using an app (diagnosed from the Received header) would
contain a link to allow an app to get an oauth token. If that's a red flag
then a meta on these two things will diagnose future attempts.



-- 
Joseph Brennan

Razor FP on simple http link (by itself)

[Rob McEwen]

I use SA as a "helper app" within my custom written spam filter. So I'll 
get SA give me an opinion about certain marginal messages, and then my 
spam filter factors the SA score into my spam filter's scoring.


Recently, a prominent law firm for whom I host mail - was complaining 
about FPs where messages from a prominent real estate company were not 
making it to them. Interestingly, their messages kept hitting RAZOR, 
where SA was giving the following response:


1.7 RAZOR2_CHECK   Listed in Razor2 (http://razor.sf.net/)
0.4 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
   [cf: 100]
2.4 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
   above 50%
   [cf: 100]

In testing, I narrowed it all the way down to simply the following 
(alone!) hitting on razor:


either
http://www.example.com
or
http://example.com

(except with the sender's domain, of course)

...either one was triggering this razor score. I even put that as the 
ONLY body text of another message (so a totally different header) - and 
it still triggered. But either variation WITHOUT the "http://; part did 
not trigger.


Interesting... this domain name happens to resolve to an IP that is 
currently blacklisted on Zen. (I know, that is really really bad!) 
Unfortunately, that confuses issues!


Does RAZOR extract domains from links and checks them against a bad 
domain database... sort of how SURBL works... and/or check the IP that 
they resolve to? (I don't think so, but now I have to ask just to be sure!)


If not... this seems to go beyond checksum-checking of parts of a 
message - this seems much more surgical/specific than that.


Don't get me wrong... I'm a big fan of razor and of other 
checksum-technologies. But I'm sort of shaken by this because I always 
thought a FP for razor would be much more difficult due to larger 
portions of a message having to match a checksum match in order to have 
a hit. (sort of like a larger "fingerprint" that is not easily 
duplicated in another innocent message, allegedly making FPs practically 
impossible)


While this kind of more surgical strike can be beneficial in blocking 
more spam - it seems like it changes the paradigm of what I 
(mistakenly?) thought to be RAZOR's potential for collateral damage.


Is this "extra curricular activity"? or did I misunderstand RAZOR's 
checksum technique?


--
Rob McEwen

Re: DKIM_VALID EnvelopeFrom

[Ian Zimmerman]

On 2017-05-05 16:00, Merijn van den Kroonenberg wrote:

> So the only thing I want with the envelop from is to extract the
> domain and test if the mail was DKIM signed (and valid) by that
> domain.
> 
> This tells me the envelope from is not some random spoofed address,
> but actually controlled by someone who handled the e-mail before it
> arrived at our mta.

Yes, this is a valid thing to do.

I do this check completely in the MTA (Exim).  Even if for some reason
you reallly need to do it in SA, the easiest way to get the envelope
sender in SA is have the MTA insert a header, such as X-Envelope-From.
Exim can do that and I'm guessing other major MTAs such as Postfix can
too.

-- 
Please *no* private Cc: on mailing lists and newsgroups
Personal signed mail: please _encrypt_ and sign
Don't clear-text sign:
http://primate.net/~itz/blog/the-problem-with-gpg-signatures.html

Re: DKIM_VALID EnvelopeFrom

[David Jones]

From: Merijn van den Kroonenberg 
    
>> On 05.05.17 11:37, Merijn van den Kroonenberg wrote:
>>>I want to test in SA if the Envelope From domain is DKIM_VALID.
>>
>> the envelope from can't be DKIM-VALID. DKIM validated message content,
>> including some of its headers, not envelope from address.

>I know. I do not want to validate the envelope from with DKIM. I just want
>to know if the mail was DKIM-VALID signed by the DOMAIN used in the
>envelopefrom.

>So the only thing I want with the envelop from is to extract the domain
>and test if the mail was DKIM signed (and valid) by that domain.

>This tells me the envelope from is not some random spoofed address, but
>actually controlled by someone who handled the e-mail before it arrived at
>our mta.

This actually would be a very useful rule/logic to add to SA:
https://blog.returnpath.com/why-passing-and-aligning-both-spf-and-dkim-is-key-to-email-deliverability/

When both align, it should be a very good candidate for whitelist_auth
based on the sender domain reputation.

Dave

Re: DKIM_VALID EnvelopeFrom

[Merijn van den Kroonenberg]

> On 05.05.17 11:37, Merijn van den Kroonenberg wrote:
>>I want to test in SA if the Envelope From domain is DKIM_VALID.
>
> the envelope from can't be DKIM-VALID. DKIM validated message content,
> including some of its headers, not envelope from address.

I know. I do not want to validate the envelope from with DKIM. I just want
to know if the mail was DKIM-VALID signed by the DOMAIN used in the
envelopefrom.

So the only thing I want with the envelop from is to extract the domain
and test if the mail was DKIM signed (and valid) by that domain.

This tells me the envelope from is not some random spoofed address, but
actually controlled by someone who handled the e-mail before it arrived at
our mta.

> --
> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> They that can give up essential liberty to obtain a little temporary
> safety deserve neither liberty nor safety. -- Benjamin Franklin, 1759
>

Re: DKIM_VALID EnvelopeFrom

[Matus UHLAR - fantomas]

On 05.05.17 11:37, Merijn van den Kroonenberg wrote:

I want to test in SA if the Envelope From domain is DKIM_VALID.


the envelope from can't be DKIM-VALID. DKIM validated message content,
including some of its headers, not envelope from address.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety. -- Benjamin Franklin, 1759

Re: DKIM_VALID EnvelopeFrom

[Merijn van den Kroonenberg]

> Merijn van den Kroonenberg skrev den 2017-05-05 11:37:
>
>> I want to test in SA if the Envelope From domain is DKIM_VALID.
>
> you basicly ask how to use sender-id :(

No, I am not interested in sender-id, which is based on SPF.
I merely want to know if the mail is DKIM signed and valid for a specific
domain, namely the domain from the enevelope sender.

>
>> I do some processing of SA maillogs and they contain the EnvelopeFrom
>> address (and not the From address) and I would like to know if they
>> are DKIM_VALID.
>
> envelopefrom is not part of dkim

Okay, maybe I didn't write it very clearly. I am interested in the
*domain* of the EnvelopeFrom. And DKIM is about signing by a domain, so
the mail *could* be DKIM signed by the domain from the envelopefrom. I
just want to test if it is true.

So basically I want to do eval:check_dkim_valid($envelope_from_domain)

>
>> Till now I have been using DKIM_VALID_AU, but this information is no
>> good if HEADER_FROM_DIFFERENT_DOMAINS is hit (difference between From
>> and EnvelopeFrom).
>
> this is not dkim related, dkim pass is no good ?, how can i post to you
> on maillist with envelopefrom with my dkim domain, you need to
> understand that enveopefrom changes on each mta stage, that should not
> break dkim, but some says it breaks spf, it does not, it will just be
> another spf domain

I am not sure what you mean, but I guess you are thinking of a too complex
situation. I am not interested in situations which are not DKIM signed by
the envelopefrom domain, just the ones who are.

>
>> Is there a way to make a DKIM_VALID_EF rule in SA? Or is the only way
>> modifying/extending the DKIM Plugin?
>
> not without a custom plugin, but its still not dkim

Re: DKIM_VALID EnvelopeFrom

[Benny Pedersen]

Merijn van den Kroonenberg skrev den 2017-05-05 11:37:


I want to test in SA if the Envelope From domain is DKIM_VALID.


you basicly ask how to use sender-id :(


I do some processing of SA maillogs and they contain the EnvelopeFrom
address (and not the From address) and I would like to know if they
are DKIM_VALID.


envelopefrom is not part of dkim


Till now I have been using DKIM_VALID_AU, but this information is no
good if HEADER_FROM_DIFFERENT_DOMAINS is hit (difference between From
and EnvelopeFrom).


this is not dkim related, dkim pass is no good ?, how can i post to you 
on maillist with envelopefrom with my dkim domain, you need to 
understand that enveopefrom changes on each mta stage, that should not 
break dkim, but some says it breaks spf, it does not, it will just be 
another spf domain



Is there a way to make a DKIM_VALID_EF rule in SA? Or is the only way
modifying/extending the DKIM Plugin?


not without a custom plugin, but its still not dkim

DKIM_VALID EnvelopeFrom

[Merijn van den Kroonenberg]

Hi,

I want to test in SA if the Envelope From domain is DKIM_VALID.
I do some processing of SA maillogs and they contain the EnvelopeFrom 
address (and not the From address) and I would like to know if they are 
DKIM_VALID.
Till now I have been using DKIM_VALID_AU, but this information is no good if 
HEADER_FROM_DIFFERENT_DOMAINS is hit (difference between From and 
EnvelopeFrom).


Is there a way to make a DKIM_VALID_EF rule in SA? Or is the only way 
modifying/extending the DKIM Plugin?


Cheers,
Merijn