Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

On 06.04.2016 18:54, Stephen Ulmer wrote:
 
> You ignored my point about being a good citizen. I’m talking about what you 
> PUBLISH, how it is useful to others, and how it will eventually lend 
> protection to your users reputations as the number of DMARC implementations 
> increases.

In my last post, I wrote that we probably will implement DMARC from the 
sender's point of view, mainly due to your hint how easy that is.
 
> You talk about rejecting using DKIM, but don’t publish a domain key yourself.

This is a great misunderstanding. Of course, we are publishing our DKIM key(s), 
are signing all messages we send by DKIM, and have published an SPF record. I 
just have said that we don't use DMARC yet. Please note that DKIM, SPF and 
DMARC are three things which are tied together very loosely. Basically, SPF and 
DKIM exist on their own and are completely independent from each other, while 
DMARC uses both of them.

>> 1) If an administrator is not able to add an SPF record to his DNS (which is 
>> also one TXT record with a syntax which can't be easier), he probably will 
>> as well not be able to add a DMARC record. So, from our point of view, since 
>> we already accept all messages which pass SPF, what would be good in 
>> additionally checking / respecting the DMARC record?
 
> It would lend legitimacy to rejected messages that have incorrect DK or SPF 
> alignment [...]

And this is exactly the thing which should *not* be done. Why on earth should 
we help spammers with delivering their messages by letting a DNS server say: 
"Hi guys, mail messages pretending to be from our domain are likely to be 
actually sent by spammers, but please accept them all nevertheless because our 
admin doesn't have 5 minutes to fix the broken SPF record"?

In other words: If I ever would see a server / domain which is configured like 
that, this wouldn't lend legitimacy to it; instead, I would blacklist it 
immediately for helping the spammers. I don't assume that I will have do do 
this because we won't use DMARC at the receiving side and so I won't get 
noticed about such heavily broken configurations, though.

>> 2) I definitely won't respect DMARC records which tell me to accept messages 
>> although they don't pass SPF or DKIM checks. Setting up such DMARC policy 
>> (until I have heavily misunderstood something) re-enables SPAM: When such a 
>> record exists for a domain, every spammer could send messages in the name of 
>> that domain, the receiving MTAs then would look up the DMARC record of that 
>> domain and would accept the message since the DMARC policy says that the 
>> messages should be accepted regardless of missing or wrong SPF / DKIM. Are 
>> they completely crazy?

> It only "re-enables" spam if you think that the lack of SPF and DKIM records 
> are indicators that the originating domain does not send legitimate messages.

No. It helps spammers get their messages into the receiving MTA in every case. 
If a spammer fakes the sender of a message and thus DKIM and SPF don't pass, 
but then the receiving MTA accepts the message due to DMARC records, the 
spammers have won. That does not make any sense in my eyes.

> Some administrators who have responsibility for many other people’s messages 
> like to actually test things and observe the behavior. There are non-reject 
> options in DMARC so those admins can get reports about activity in their name 
> without disrupting an existing system. It is a polite request for diagnostic 
> data from other system owners. I don’t believe that anyone leaves p=none in 
> place for forever (though I could just be wrong about that). You might be 
> surprised by getting some of those reports.

I agree, and I also think that DMARC's reporting protocols are a good thing. I 
don't want to keep anybody from using DMARC; I just said that we won't use it 
at the receiving side for the reasons I have mentioned (we are a very small 
company and thus don't need to run long tests before doing such things).
  
> My point about reputation-based systems is not about what you consume, which 
> seems to be all you care about. You can publish DMARC records and help other 
> administrators know what email from YOUR domain is legitimate.

Thanks to your hint about how easy that is, we'll probably do this very fast.
  
> Here is where I was talking about what you accept. Rejecting a message based 
> on its spam score does not leave any uncertainty. The acceptance status of 
> the message is by definition no more or less clear than anything you are 
> doing now.

I don't think so. We have our MTA deliver all messages to our users if they 
pass SPF or DKIM (except from blacklisted domains), even if they are SPAM. This 
means that it is a human who finally decides if a message is SPAM and if the 
respective domain should be blacklisted. In contrast, SpamAssassin returns some 
sort of spam score which is computed according to some very intelligent magic, 
to long, long training (hopefully) 

Re: Request: Please sign this list's messages via DKIM or SPF

[Stephen Ulmer via Info-cyrus]

> On Apr 6, 2016, at 11:24 AM, Binarus via Info-cyrus 
>  wrote:
> 
> On 06.04.2016 02:54, Stephen Ulmer wrote: 
>> You make some good arguments, but then you say other things that make me 
>> think you’re not clueful. I think you’ll be more persuasive if you consider 
>> the following:
> 
> Indeed, I'm not clueful regarding DMARC. Regarding SPF, DKIM and SMTP in 
> general, I am considering myself quite clueful.
> 
>> To be a good citizen, you should care about the possibility of others 
>> sending spam in your users’ names — we will *eventually* wind up with a 
>> reputation-based anti spam network of some kind, and you’ll want to have 
>> protected them from before its inception. Ergo, you should care at least a 
>> little about DMARC. It's one TXT record — in an environment where you can 
>> just reject whatever mail you want, clearly you’ve got the authority to 
>> implement such.
> 
> I am all times reading that people believe that SPF or DKIM can prevent 
> spammers from sending messages in their names. As I have already stated 
> multiple times, this is just wrong. Of course, every spammer can always send 
> messages with an envelope-from or from of your_name@your_domain, regardless 
> of SPF or DKIM records for your_domain. SPF and DKIM only help the 
> *receiving* MTA to decide if the message is really coming from the pretended 
> sender('s domain). DMARC does not change that.

You ignored my point about being a good citizen. I’m talking about what you 
PUBLISH, how it is useful to others, and how it will eventually lend protection 
to your users reputations as the number of DMARC implementations increases.

You talk about rejecting using DKIM, but don’t publish a domain key yourself. 
What if one of your users needs to send messages to a server that has your 
attitude, but thinks that SPF is pointless?

DMARC indicates to the receiving server a policy that specifies the sending 
domain owner’s level of confidence in their own SPF and DKIM records. DMARC 
basically tells the receiver that the owner of the sending domain agrees that 
you should reject messages that don’t pass SPF or DKIM.


> Furthermore, I can't see much sense in DMARC (except the reporting protocols) 
> at least for the following reasons:
> 
> 1) If an administrator is not able to add an SPF record to his DNS (which is 
> also one TXT record with a syntax which can't be easier), he probably will as 
> well not be able to add a DMARC record. So, from our point of view, since we 
> already accept all messages which pass SPF, what would be good in 
> additionally checking / respecting the DMARC record?

It would lend legitimacy to rejected messages that have incorrect DK or SPF 
alignment. Again, publishing such a record would help others.

> 2) I definitely won't respect DMARC records which tell me to accept messages 
> although they don't pass SPF or DKIM checks. Setting up such DMARC policy 
> (until I have heavily misunderstood something) re-enables SPAM: When such a 
> record exists for a domain, every spammer could send messages in the name of 
> that domain, the receiving MTAs then would look up the DMARC record of that 
> domain and would accept the message since the DMARC policy says that the 
> messages should be accepted regardless of missing or wrong SPF / DKIM. Are 
> they completely crazy?

It only "re-enables" spam if you think that the lack of SPF and DKIM records 
are indicators that the originating domain does not send legitimate messages. 

Some administrators who have responsibility for many other people’s messages 
like to actually test things and observe the behavior. There are non-reject 
options in DMARC so those admins can get reports about activity in their name 
without disrupting an existing system. It is a polite request for diagnostic 
data from other system owners. I don’t believe that anyone leaves p=none in 
place for forever (though I could just be wrong about that). You might be 
surprised by getting some of those reports.

> [Note: Quarantining a message includes accepting it!]
> 
> Furthermore, I wouldn't respect a DMARC record which told me to quarantine a 
> message because quarantining it means accepting it before, and this again 
> would enable the sender to prove that our MTA has received the message and 
> would force us to check the quarantined messages on a regular basis.
> 
> And, last but not least, we won't use a reputation based anti spam network 
> because we don't need it. Our world is black and white:
> 
> - All messages from blacklisted domains are rejected.
> - All messages which pass DKIM or SPF are accepted (even if they later turn 
> out to be SPAM).
> - All other messages are rejected (without blacklisting the respective 
> domain, of course).
> - If SPAM is sent by a domain (proven by SPF or DKIM), this domain becomes 
> blacklisted until the domain owner calls us and explains what has happened.
> 
> Still not being clueful regarding 

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

On 06.04.2016 02:54, Stephen Ulmer wrote: 
> You make some good arguments, but then you say other things that make me 
> think you’re not clueful. I think you’ll be more persuasive if you consider 
> the following:

Indeed, I'm not clueful regarding DMARC. Regarding SPF, DKIM and SMTP in 
general, I am considering myself quite clueful.

> To be a good citizen, you should care about the possibility of others sending 
> spam in your users’ names — we will *eventually* wind up with a 
> reputation-based anti spam network of some kind, and you’ll want to have 
> protected them from before its inception. Ergo, you should care at least a 
> little about DMARC. It's one TXT record — in an environment where you can 
> just reject whatever mail you want, clearly you’ve got the authority to 
> implement such.

I am all times reading that people believe that SPF or DKIM can prevent 
spammers from sending messages in their names. As I have already stated 
multiple times, this is just wrong. Of course, every spammer can always send 
messages with an envelope-from or from of your_name@your_domain, regardless of 
SPF or DKIM records for your_domain. SPF and DKIM only help the *receiving* MTA 
to decide if the message is really coming from the pretended sender('s domain). 
DMARC does not change that.

Furthermore, I can't see much sense in DMARC (except the reporting protocols) 
at least for the following reasons:

1) If an administrator is not able to add an SPF record to his DNS (which is 
also one TXT record with a syntax which can't be easier), he probably will as 
well not be able to add a DMARC record. So, from our point of view, since we 
already accept all messages which pass SPF, what would be good in additionally 
checking / respecting the DMARC record?

2) I definitely won't respect DMARC records which tell me to accept messages 
although they don't pass SPF or DKIM checks. Setting up such DMARC policy 
(until I have heavily misunderstood something) re-enables SPAM: When such a 
record exists for a domain, every spammer could send messages in the name of 
that domain, the receiving MTAs then would look up the DMARC record of that 
domain and would accept the message since the DMARC policy says that the 
messages should be accepted regardless of missing or wrong SPF / DKIM. Are they 
completely crazy?

[Note: Quarantining a message includes accepting it!]

Furthermore, I wouldn't respect a DMARC record which told me to quarantine a 
message because quarantining it means accepting it before, and this again would 
enable the sender to prove that our MTA has received the message and would 
force us to check the quarantined messages on a regular basis.

And, last but not least, we won't use a reputation based anti spam network 
because we don't need it. Our world is black and white:

- All messages from blacklisted domains are rejected.
- All messages which pass DKIM or SPF are accepted (even if they later turn out 
to be SPAM).
- All other messages are rejected (without blacklisting the respective domain, 
of course).
- If SPAM is sent by a domain (proven by SPF or DKIM), this domain becomes 
blacklisted until the domain owner calls us and explains what has happened.

Still not being clueful regarding DMARC, I didn't know that it also is just one 
TXT record - thanks for the hint. If it's really that easy, we will implement 
this, but only from the sender's viewpoint, i.e. we won't make our MTA check / 
use DMARC records.

> There are PLENTY of anti-spam measures “on the server side”. You could choose 
> to use Spamassassin, and reject the most egregious messages (with whatever 
> explanation you’d like). You could also use any number of RBLs. Greylisting 
> has at least some positive effect. You might choose not to implement any of 
> these for various reasons, but they certainly exist and can be effective.

This is all true, but why should we? Of course, we have thought about 
SpamAssassin and similar solutions. But our solution, as currently implemented, 
does not leave any uncertainty: There is one single clear criterion for 
blacklisting a domain (namely if it is proven that the domain has sent spam), 
and we are sure that the pretended sender of every rejected message gets a DSN, 
so we are on the safe side from the legal point of view.

Our solution has three downsides:

1) Before blacklisting a domain, SPAM mails from that domain get to our users.

2) Some big providers don't implement SPF or DKIM, so we are rejecting messages 
from these.

3) After having received SPAM from a big provider's domain, we usually 
blacklist this domain, thus preventing a large number of users from sending us 
messages.

We can happily live with these downsides, especially when comparing them to the 
downsides of other solutions (due to the nature of our company, we won't 
receive messages from the big freemailers anyway).
 
> If you are only checking for false-positives once a year, you deserve 
> whatever the volume in which 

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

> If you want to see flame wars even more pointless and/or entertaining than 
> this one, check out the mailing lists for DMARC. ;-)  They make these recent 
> exchanges seem quaint by comparison.

I am sorry that this thread is not useful to you. I don't consider it a flame 
war. Every party (except the one who called us "phenomenally stupid") had a 
reasoning which at least is worth thinking about.
 
> FWIW, mailing lists and DMARC make a particularly noxious couple, as almost 
> all mailing lists will break DMARC, and thus lead to all sorts of rejections. 
>  That very subject is the topic of the most vitriolic flame wars on the DMARC 
> lists. 

Maybe. We are currently not interested in DMARC.
  
> At the risk of perpetuating this severely off-topic thread, IMHO if "Binarus" 
> is able to eliminate "90% solely by checking for SPF and DKIM" then one must 
> question just what the rest of their anti-Spam measures were doing?

The answer is quite easy: Until now, there just haven't been any measures 
against SPAM on the server side. Instead, users have used Thunderbird's junk 
filter (which works great IMHO). So, before checking SPF / DKIM, the clients 
actually have received every message which hit the server, except the messages 
which were addressed to non-existent recipients. We know quite well how many 
SPAM got to the clients before and after implementing the SPF / DKIM checks.

The problem with letting the clients doing the SPAM handling (explained by the 
example of my personal account): Once per year, I had to go through the JUNK 
folder to see if there were false positives in that folder. Some weeks ago, 
this ended in manually searching through about 12000 spam message and thereby 
finding about 10 important *ham* messages.

Given some court decisions here in Germany, it could eventually be dangerous to 
not handle a message in a timely manner or even to never know about that 
message if the sender can prove that your server has accepted the message. 
Therefore, there is no way around checking your SPAM folder if you let your MUA 
sort out the SPAM.

Some weeks ago, for a reason I still don't know, the SPAM volume hitting our 
clients suddenly doubled (or tripled, I don't have the exact figures). 
Therefore, we have decided to change our SPAM handling. Nobody is keen on 
scanning 10 SPAM messages at the end of the year (my mailbox is not the 
worst one) ...

By the way, I am now finishing my working day, having got exactly 4 SPAM 
messages (two weeks ago: between 200 und 300 per day).

Regards,

Binarus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Nic Bernstein via Info-cyrus]

On 04/05/2016 11:33 AM, Andrew Morgan via Info-cyrus wrote:

On Tue, 5 Apr 2016, lst_hoe02--- via Info-cyrus wrote:



Zitat von Binarus via Info-cyrus :



Combine SPF / DKIM with domain blacklisting, and then you *have* an 
efficient spam fighting tool.




As stated the spam actually reaching our inboxes after around 90% 
cutoff is valid DKIM/SPF signed as it is mostly from the big free 
providers like Outlook.com, Google and Yahoo. Some other big share is 
from professional spam farms with always alternating IP and Domains 
ranges from all over the world with also valid DKIM/SPF. Next big 
share is from educational servers also mostly valid DKIM/SPF. The 
tiny rest with around 10% is in fact not DKIM/SPF signed.
From the valid e-mail around 20% looks like having a valid SPF/DKIM, 
mostly professional newsletters not personal mail from customers.


So No, SPF/DKIM is no useful spam fighting tool at least not in our 
corner of the world.


Another recent standard, DMARC (https://dmarc.org/) allows the domain 
owner to specify what the recipient should do with messages that fail 
DKIM or SPF checks.


We ran into this recently and discovered that Yahoo's DMARC records 
tell the recipient to REJECT messages that fail DKIM or SPF.  Google 
is honoring that DMARC record by putting the message into the Spam 
folder.


This seems like a pretty effective method to prevent someone from 
spoofing email from your domain.  Of course, it does not prevent an 
actual Yahoo account from sending spam, so you still need traditional 
spam detection tools as well.  However, it is nice that a third-party 
sender cannot harm your domain's reputation through spoofing.


Note: I don't care whether this email list uses SPF or DKIM.

Andy


If you want to see flame wars even more pointless and/or entertaining 
than this one, check out the mailing lists for DMARC. ;-)  They make 
these recent exchanges seem quaint by comparison.


   ___
   dmarc-discuss mailing list
   dmarc-disc...@dmarc.org
   http://www.dmarc.org/mailman/listinfo/dmarc-discuss 

FWIW, mailing lists and DMARC make a particularly noxious couple, as 
almost all mailing lists will break DMARC, and thus lead to all sorts of 
rejections.  That very subject is the topic of the most vitriolic flame 
wars on the DMARC lists.


Tho, to be honest, I had assumed that the recent changes to the From and 
Reply-To headers of this mailing list were undertaken to appease strict 
DMARC requirements.


Yes, Google, Yahoo and most of the rest of the Big Boys(c) have adopted 
DMARC with "p=reject" (or whatever that setting is.


At the risk of perpetuating this severely off-topic thread, IMHO if 
"Binarus" is able to eliminate "90% solely by checking for SPF and DKIM" 
then one must question just what the rest of their anti-Spam measures 
were doing?


Cheers,
-nic

--
Nic Bernstein n...@onlight.com
Onlight Inc.  www.onlight.com
6525 W Bluemound Rd., Ste 24  v. 414.272.4477
Milwaukee, Wisconsin  53213-4073  f. 414.290.0335


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Andrew Morgan via Info-cyrus]

On Tue, 5 Apr 2016, lst_hoe02--- via Info-cyrus wrote:



Zitat von Binarus via Info-cyrus :



Combine SPF / DKIM with domain blacklisting, and then you *have* an 
efficient spam fighting tool.




As stated the spam actually reaching our inboxes after around 90% cutoff is 
valid DKIM/SPF signed as it is mostly from the big free providers like 
Outlook.com, Google and Yahoo. Some other big share is from professional spam 
farms with always alternating IP and Domains ranges from all over the world 
with also valid DKIM/SPF. Next big share is from educational servers also 
mostly valid DKIM/SPF. The tiny rest with around 10% is in fact not DKIM/SPF 
signed.
From the valid e-mail around 20% looks like having a valid SPF/DKIM, mostly 
professional newsletters not personal mail from customers.


So No, SPF/DKIM is no useful spam fighting tool at least not in our corner of 
the world.


Another recent standard, DMARC (https://dmarc.org/) allows the domain 
owner to specify what the recipient should do with messages that fail DKIM 
or SPF checks.


We ran into this recently and discovered that Yahoo's DMARC records tell 
the recipient to REJECT messages that fail DKIM or SPF.  Google is 
honoring that DMARC record by putting the message into the Spam folder.


This seems like a pretty effective method to prevent someone from spoofing 
email from your domain.  Of course, it does not prevent an actual Yahoo 
account from sending spam, so you still need traditional spam detection 
tools as well.  However, it is nice that a third-party sender cannot harm 
your domain's reputation through spoofing.


Note: I don't care whether this email list uses SPF or DKIM.

Andy

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

On 05.04.2016 09:34, lst_hoe02--- via Info-cyrus wrote:
 
> The "we generally have to reject all messages which are not secured by SPF or 
> DKIM" mean you want to force others to use non standard headers so in fact 
> you are breaking SMTP RFC.

I think we don't. At least SPF works without additional headers in the messages.

Furthermore, I still can't see how we would break RFCs even if we would "force" 
people to use the DKIM header (in fact, we are not forcing anybody to do so, 
because we let messages pass which have at least *one* of SPF or DKIM passed): 
The RFCs nowhere say that every MTA MUST accept ANY message regardless of the 
sender, connecting server etc. On the contrary, the RFCs explicitly name 
mechanisms (e.g. DSNs) which should be used if a message cannot be delivered to 
its recipient, and people are rejecting messages (and returning appropriate 
DSNs) according to their own policies for decades now.

If you are saying that not accepting *all* messages means breaking the RFCs, I 
disagree.

What I exceptionally like about the way we have implemented the SPF and DKIM 
checks is that the sender gets informed about the problem because he will 
receive an appropriate DSN containing a polite message which explains the 
problem. In summary, I am convinced that our MTA's behavior conforms with the 
RFCs.

> It is your server so your rules, but don't complain if other do not agree 
> with you.

I promise I won't :-)

Regards,

Binarus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

On 05.04.2016 14:15, Alvin Starr via Info-cyrus wrote:
> 
> I kind of have to agree with Andreas to some extent on this.
> SPF/DKIM does not help on incoming spam filtering all that much just because 
> so few people use it and the default action is to accept mail that has no 
> SPF/DKIM tagging.

Our default action is to reject all messages which do not pass either the SPF 
or the DKIM test.

> 
> It is great however for controlling how other people abuse your email address.
> SPF can stop people from sending mail as you from systems that are not your 
> own.

Not really, AFAIK. Even if you add the SPF record to your domain's DNS, a 
spammer of course can still use @ as envelope 
sender or From: header. It is the receiving part who checks if the connecting 
MTA (i.e. the "sending server") is allowed to send messages for  (the check is done by querying the name server for  for 
the SPF record and then checking if the sending (connecting) server one of the 
servers the SPF record allows).

In other words, if no SPF checks are done by the *receiving* MTAs, fake 
messages will make their way through the net without problems.
 
> I would argue that anybody operating a mail server should use SPF/DKIM just 
> to make sure they are not helping the spammers.

I strongly agree.
 
> Sadly putting these tools in place is not trivial and it will only be when 
> postfix, sendmail, qmail and others include SPF/DKIM setups as part of the 
> default install can things really start to change.

Actually, I have been surprised how ridiculously easy I could setup the 
*sending* part of SPF. Using SPF as a sender means adding one TXT record (whose 
syntax can't be simpler) to your DNS records; this could be done within minutes 
(no more true if you want your MTA to forward messages from other domains; 
that's a special case). DKIM is slightly more complicated since it needs 
additional software which must be interfaced to the MTA. I used opendkim and 
liked it very much, though.

Checking SPF and DKIM (the *receiving* part) was much more complicated in our 
case, though. So I would recommend everybody who wants to improve email 
security to start with the sending part. If you don't forward messages for 
other domains, just start with adding the SPF record to your name server (and 
end that record with "-all" in every case, despite other examples which could 
be found on the net).

Regards,

Binarus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

On 05.04.2016 09:42, lst_hoe02--- via Info-cyrus wrote:
> 
> As stated the spam actually reaching our inboxes after around 90% cutoff is 
> valid DKIM/SPF signed as it is mostly from the big free providers like 
> Outlook.com, Google and Yahoo. Some other big share is from professional spam 
> farms with always alternating IP and Domains ranges from all over the world 
> with also valid DKIM/SPF. Next big share is from educational servers also 
> mostly valid DKIM/SPF. The tiny rest with around 10% is in fact not DKIM/SPF 
> signed.
> From the valid e-mail around 20% looks like having a valid SPF/DKIM, mostly 
> professional newsletters not personal mail from customers.
> 
> So No, SPF/DKIM is no useful spam fighting tool at least not in our corner of 
> the world.
> 

We seem to be located in the same country (Germany), nevertheless the situation 
is completely different for us. As I have already reported, we have cut off 
SPAM by 90% solely by checking for SPF and DKIM, and it looks like we could cut 
down it by another order of magnitude if we are blacklisting domains which have 
sent SPF- or DKIM-"signed" SPAM (doing so for a few days, but no exact figures 
yet).

I admit that our situation is somewhat special because we are purely B2B, and I 
absolutely don't care about a freemail provider being blacklisted. I can't even 
remember the last time when we got a valid message which has been sent from a 
freemailer account.

Actually, if everybody did SPF or DKIM tests, this finally would force the 
providers to implement DKIM or SPF the right way. For example, using an 
individual DKIM signature for every sender of a domain is ridiculously easy (at 
least when using the opendkim daemon). That would be a great progress because 
then you could blacklist individual senders instead of the provider.

Regards,

Binarus


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

On 04.04.2016 23:02, Vincent Fox via Info-cyrus wrote:
> I'll admit I am testing SPF as a greylisting measure.
> Your IP gets hardfail, you get 5min deferral.
> 
> I don't delude myself it does anything other than catch maybe
> 5-10% of spammers that don't bother with retries.  More often it
> seems to catch people like a major network backbone operation
> that OUGHT to know better, that has no SPF and acted like it
> was going to require committees and 2 months for the
> brain surgery.
> 
> YMMV indeed.
> 

Well, that seems to be the case. I have no reason to boast here; it is indeed 
true that we cut down the number of spam messages by 90% solely by rejecting 
all messages without one of SPF or DKIM. Since a few days, we are blacklisting 
the domains which have sent SPAM, and now it looks like we could cut down the 
SPAM an additional order of magnitude by doing so (no exact figures yet).

Regards,

Binarus


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

On 04.04.2016 21:50, Joseph Brennan via Info-cyrus wrote:
> 
>> But with SPF or DKIM, you can immediately blacklist any sender
>> domain after having received SPAM from that domain.
> 
> It would never be a phished stolen account, so that would be safe.
> 

You are right. It is the only logical thing to accept emails from stolen or 
phished accounts for the sole reason that they have been stolen or phished.

Joking apart: After having repaired the problem, the victim (i.e. the 
legitimate, white-hat real owner of the account) hopefully sees the DSNs, and, 
if his message is important, might call us and ask what has happened.

Even more, the DSN from our MTA eventually might let the "real owners" know 
that somebody is doing damage to them. Did you think about that? By the way, 
there are countries where you are liable if you send viruses, and in those 
countries, people might be even more grateful if they receive a DSN after a 
spammer has abused their account.

Regards,

Binarus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Alvin Starr via Info-cyrus]

On 04/05/2016 03:42 AM, lst_hoe02--- via Info-cyrus wrote:


Zitat von Binarus via Info-cyrus :



Combine SPF / DKIM with domain blacklisting, and then you *have* an 
efficient spam fighting tool.




As stated the spam actually reaching our inboxes after around 90% 
cutoff is valid DKIM/SPF signed as it is mostly from the big free 
providers like Outlook.com, Google and Yahoo. Some other big share is 
from professional spam farms with always alternating IP and Domains 
ranges from all over the world with also valid DKIM/SPF. Next big 
share is from educational servers also mostly valid DKIM/SPF. The tiny 
rest with around 10% is in fact not DKIM/SPF signed.
From the valid e-mail around 20% looks like having a valid SPF/DKIM, 
mostly professional newsletters not personal mail from customers.


So No, SPF/DKIM is no useful spam fighting tool at least not in our 
corner of the world.


I kind of have to agree with Andreas to some extent on this.
SPF/DKIM does not help on incoming spam filtering all that much just 
because so few people use it and the default action is to accept mail 
that has no SPF/DKIM tagging.


It is great however for controlling how other people abuse your email 
address.
SPF can stop people from sending mail as you from systems that are not 
your own.
DKIM signs your messages so that you have assurance that they are coming 
from your mail servers.


I would argue that anybody operating a mail server should use SPF/DKIM 
just to make sure they are not helping the spammers.


Sadly putting these tools in place is not trivial and it will only be 
when postfix, sendmail, qmail and others include SPF/DKIM setups as part 
of the default install can things really start to change.



--
Alvin Starr   ||   voice: (905)513-7688
Netvel Inc.   ||   Cell:  (416)806-0133
al...@netvel.net  ||


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[lst_hoe02--- via Info-cyrus]

Zitat von Binarus via Info-cyrus :



Combine SPF / DKIM with domain blacklisting, and then you *have* an  
efficient spam fighting tool.




As stated the spam actually reaching our inboxes after around 90%  
cutoff is valid DKIM/SPF signed as it is mostly from the big free  
providers like Outlook.com, Google and Yahoo. Some other big share is  
from professional spam farms with always alternating IP and Domains  
ranges from all over the world with also valid DKIM/SPF. Next big  
share is from educational servers also mostly valid DKIM/SPF. The tiny  
rest with around 10% is in fact not DKIM/SPF signed.
From the valid e-mail around 20% looks like having a valid SPF/DKIM,  
mostly professional newsletters not personal mail from customers.


So No, SPF/DKIM is no useful spam fighting tool at least not in our  
corner of the world.


Regards

Andreas




smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[lst_hoe02--- via Info-cyrus]

Zitat von Binarus via Info-cyrus :


On 04.04.2016 18:12, Sebastian Hagedorn via Info-cyrus wrote:
Personally, I think that's a phenomenally stupid approach. As long  
as you can't show me an RFC that says you MUST or even SHOULD use  
SPF or DKIM, you're breaking SMTP.


I think it's a phenomenally intelligent approach. I can't see in  
which way SMTP is broken by using DKIM or SPF. The DKIM signature is  
in an additional header (additional headers *are* allowed by the  
RFCs), and signing and checking usually is done by milters (I am  
sure that you know them). If a message is rejected by the receiving  
MTA due to failing SPF or DKIM, the sender will get a DSN (which is  
perfectly in conformance with the RFCs).


By the way, many people use all sorts of mail filtering and DSNs  
(and do so since 20 years and more) without an RFC saying they  
SHOULD or MUST do so. Are all people which use any sort of mail  
filter breaking SMTP as well?


Could you please give an example of an SMTP RFC which is violated by  
SPF or DKIM?


Regards,

Binarus




Due to the exponential increase of spam, we generally have to reject all
messages which are not secured by SPF or DKIM, and we know a lot of other
people who do the same (by the way, this has proven to be extremely
effective in our case). When our MTA encounters such a message, it
rejects it and returns a bounce message to the pretended sender,
notifying him about the problem.


The "we generally have to reject all messages which are not secured by  
SPF or DKIM" mean you want to force others to use non standard headers  
so in fact you are breaking SMTP RFC.


It is your server so your rules, but don't complain if other do not  
agree with you.


Regards

Andreas





smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Vincent Fox via Info-cyrus]

On 04/04/2016 09:43 AM, Binarus via Info-cyrus wrote:

But the spammer then first has to get a domain and then has to set up the DNS 
entries, which obviously is too complicated for most spammers. Furthermore, I 
am constantly seeing messages trying to get into the server which originate 
from dynamic IP addresses.

"Too complicated"?  The people setting up shop in the new ICANN gTLD
zones seem savvy enough to spend an extra minute defining
the TXT record for it.

Pulled several spam domains off my logs, they have 'em

[root@mx1 log]# dig txt +short purning.top
"v=spf1 a mx ip4:216.169.122.0/24 -all"

[root@mx1 log]# dig txt +short whicanion.top
"v=spf1 a mx ip4:216.169.125.0/24 -all"
"v=spf1 redirect=_spf.mailhostbox.com"

I'll admit I am testing SPF as a greylisting measure.
Your IP gets hardfail, you get 5min deferral.

I don't delude myself it does anything other than catch maybe
5-10% of spammers that don't bother with retries.  More often it
seems to catch people like a major network backbone operation
that OUGHT to know better, that has no SPF and acted like it
was going to require committees and 2 months for the
brain surgery.

YMMV indeed.



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Joseph Brennan via Info-cyrus]

Binarus via Info-cyrus  wrote:


But with SPF or DKIM, you can immediately blacklist any sender
domain after having received SPAM from that domain.


It would never be a phished stolen account, so that would be safe.

Joseph Brennan
Columbia University Information Technology




Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

> 
> You are for sure aware that neither SPF nor DKIM are able or designed to 
> fight Spam.

I know that a lot of people are stressing this. But it is not my opinion nor 
experience (see below).

> In fact more than half of the Spam reaching our inboxes are valid according 
> DKIM/SPF so we even might reduce spam by rejecting DKIM/SPF signed mail.

In our case, we have cut down SPAM by approximately 90% alone by doing SPF and 
DKIM checks with incoming messages. For example, my own corporate mailbox 
approximately got over 200 SPAM messages per day before rejecting messages 
without DKIM or SPF, and now I am getting somewhere between 10 and 30. That's 
what I meant by "extremely effective in our case".

> DKIM/SPF does only include that the sending server is mandated by DNS to send 
> mail for the given domain and this is easily done with all modern spammer 
> tools.

Well, most spammer tools might do this. But the spammer then first has to get a 
domain and then has to set up the DNS entries, which obviously is too 
complicated for most spammers. Furthermore, I am constantly seeing messages 
trying to get into the server which originate from dynamic IP addresses. The 
majority of spam messages seems still to be sent directly (i.e. without passing 
a "smarthost") to our receiving MTA by PCs which have been infected with a 
trojan horse; it seems that the sending MTA more often than not is part of this 
trojan and thus sends the messages from a dynamic IP address. I am convinced 
that it is impossible for a spammer to continuously update his SPF entries for 
all devices he has under control with the dynamic IP addresses of these devices.

[N.B. Of course, we are rejecting messages event if they pass SPF, but the SPF 
entry has something like +all or ~all in it.]

Now to the most important part of SPF and DKIM (I am stressing this because I 
am convinced that many people really believe that you can't fight SPAM with SPF 
or DKIM):

As you correctly have stated, if a message passes the SPF or DKIM test, it can 
be taken for sure that the *owner* (or some person which has been authorized by 
the owner) of the (pretended) sender's domain actually has authorized that 
message (at least indirectly). In other words, if a message which passes the 
SPF or DKIM test contains SPAM, the owner of the (pretended) sender's domain 
either has allowed somebody to use the domain for sending spam, or he obviously 
is not in control of his staff or his mail or DNS server. In either case, you 
could (and should) blacklist this sender domain.

This is the key aspect: Without SPF and DKIM, you can *not* blacklist a sender 
domain after receiving SPAM from that domain, because you could be sure that 
the sender domain has been faked by the spammer, and if you would have 
blacklisted it, you would not get legitimate emails from there any more 
(imagine the spammer had used someb...@ibm.com as sender's address).

But with SPF or DKIM, you can immediately blacklist any sender domain after 
having received SPAM from that domain. You now know *for sure* that the spammer 
did not abuse / fake the sender's address (letting apart such things like a 
hacked mail relay etc.), but that the domain owner has authorized the SPAM, 
thus you are sure that you do not want to get any more messages from that 
domain.

Combine SPF / DKIM with domain blacklisting, and then you *have* an efficient 
spam fighting tool.

Regards,

Binarus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

On 04.04.2016 18:12, Sebastian Hagedorn via Info-cyrus wrote:
> Personally, I think that's a phenomenally stupid approach. As long as you 
> can't show me an RFC that says you MUST or even SHOULD use SPF or DKIM, 
> you're breaking SMTP.

I think it's a phenomenally intelligent approach. I can't see in which way SMTP 
is broken by using DKIM or SPF. The DKIM signature is in an additional header 
(additional headers *are* allowed by the RFCs), and signing and checking 
usually is done by milters (I am sure that you know them). If a message is 
rejected by the receiving MTA due to failing SPF or DKIM, the sender will get a 
DSN (which is perfectly in conformance with the RFCs).

By the way, many people use all sorts of mail filtering and DSNs (and do so 
since 20 years and more) without an RFC saying they SHOULD or MUST do so. Are 
all people which use any sort of mail filter breaking SMTP as well?

Could you please give an example of an SMTP RFC which is violated by SPF or 
DKIM?

Regards,

Binarus

> 
>> Due to the exponential increase of spam, we generally have to reject all
>> messages which are not secured by SPF or DKIM, and we know a lot of other
>> people who do the same (by the way, this has proven to be extremely
>> effective in our case). When our MTA encounters such a message, it
>> rejects it and returns a bounce message to the pretended sender,
>> notifying him about the problem.
> -- 
> Sebastian Hagedorn - Weyertal 121, Zimmer 2.02
> Regionales Rechenzentrum (RRZK)
> Universität zu Köln / Cologne University - Tel. +49-221-470-89578
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Sebastian Hagedorn via Info-cyrus]

Personally, I think that's a phenomenally stupid approach. As long as you 
can't show me an RFC that says you MUST or even SHOULD use SPF or DKIM, 
you're breaking SMTP.



Due to the exponential increase of spam, we generally have to reject all
messages which are not secured by SPF or DKIM, and we know a lot of other
people who do the same (by the way, this has proven to be extremely
effective in our case). When our MTA encounters such a message, it
rejects it and returns a bounce message to the pretended sender,
notifying him about the problem.

--
Sebastian Hagedorn - Weyertal 121, Zimmer 2.02
Regionales Rechenzentrum (RRZK)
Universität zu Köln / Cologne University - Tel. +49-221-470-89578

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[lst_hoe02--- via Info-cyrus]

Zitat von Binarus via Info-cyrus :


Dave,

On 04.04.2016 13:22, Dave McMurtrie wrote:
the messages which are being sent from this mailing list's server  
don't seem to be protected by SPF or signed by DKIM. Are there  
plans to implement at least one of these in the near future?




We currently have no plans to implement either, but I can put it on our
list of things to do.



Thank you very much for considering.

Due to the exponential increase of spam, we generally have to reject  
all messages which are not secured by SPF or DKIM, and we know a lot  
of other people who do the same (by the way, this has proven to be  
extremely effective in our case). When our MTA encounters such a  
message, it rejects it and returns a bounce message to the pretended  
sender, notifying him about the problem.


You are for sure aware that neither SPF nor DKIM are able or designed  
to fight Spam. In fact more than half of the Spam reaching our inboxes  
are valid according DKIM/SPF so we even might reduce spam by rejecting  
DKIM/SPF signed mail.
DKIM/SPF does only include that the sending server is mandated by DNS  
to send mail for the given domain and this is easily done with all  
modern spammer tools.


But this is also OT here

Regards

Andreas




smime.p7s
Description: S/MIME Cryptographic Signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

Dave,

On 04.04.2016 16:32, Dave McMurtrie wrote:
> I completely agree.  I'll run this up the management chain and see if I
> can get approval.  Really, the ideal solution would be to set up a list
> server in the cyrusimap.org domain and handle it there because CMU
> management doesn't care what we do in that domain.  I'd love to do that,
> but I'm hesitant to foist that change on Cyrus users since info-cyrus
> has been on lists.andrew for so many years now.
> 
> Your input is appreciated, though.

Well, not being an expert in that area, my 2 cents:

I think I wouldn't move to another server, too (never touch a running system). 
But eventually you could forward all messages from lists.andrew to 
cyrusimap.org which then could sign and send them? That way you could keep the 
current server (nearly unaltered) for mailing list management, processing the 
received messages and sending messages. The only change would be to not 
directly send messages, but to forward them.

Before sending, cyrusimap.org should rewrite the envelope-from and from, making 
them something like "cyrus-imapd-l...@cyrusimap.org". The receiving MTAs could 
then get the public DKIM key from cyrusimap.org and check if the signature is 
valid, i.e. if the message actually has been sent by cyrusimap.org.

Or, even easier: Just add an appropriate SPF record to the DNS configuration of 
andrew.cmu.edu, and we could test what happens. Adding such record should get 
immediate approval by your management since it does not affect other DNS 
records or the mailing list server in any way. In other words, you would just 
have one more TXT record in your DNS which will not interfere with any other 
system component in any way. I strongly assume that this already would be 
sufficient.

Regards,

Binarus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Binarus via Info-cyrus]

Dave,

On 04.04.2016 13:22, Dave McMurtrie wrote:
>> the messages which are being sent from this mailing list's server don't seem 
>> to be protected by SPF or signed by DKIM. Are there plans to implement at 
>> least one of these in the near future?
>>
> 
> We currently have no plans to implement either, but I can put it on our
> list of things to do.
> 

Thank you very much for considering.

Due to the exponential increase of spam, we generally have to reject all 
messages which are not secured by SPF or DKIM, and we know a lot of other 
people who do the same (by the way, this has proven to be extremely effective 
in our case). When our MTA encounters such a message, it rejects it and returns 
a bounce message to the pretended sender, notifying him about the problem.

Of course, we absolutely do not want those DSNs to get onto the mailing lists, 
so we have to implement exception rules for every mailing list which does not 
use SPF or DKIM. This generally works, but it requires some manual effort, and 
it only works until a mailing list server changes ...

Given that, I think that other people would be grateful if you would implement 
one of those techniques, too, and implementing is very easy :-)

Regards,

Binarus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Request: Please sign this list's messages via DKIM or SPF

[Dave McMurtrie via Info-cyrus]

On Fri, 2016-04-01 at 15:38 +0200, Binarus via Info-cyrus wrote:
> Dear list administrator,
> 
> the messages which are being sent from this mailing list's server don't seem 
> to be protected by SPF or signed by DKIM. Are there plans to implement at 
> least one of these in the near future?
> 

We currently have no plans to implement either, but I can put it on our
list of things to do.

Thanks!

Dave

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus