Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
For anyone who wants to play around with this, the DNS service has been posted. You can test the existence of a website on a domain or any of its parent domains by making DNS queries as follows: subdomain.domain.com.httpcheck.singulink.com So, if you wanted to check if mail1.mx.google.com or

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
You'll be able to decide how you want to prioritize the fields - I've implemented it as a DNS server, so which domain you decide to send to the DNS server is entirely up to you. On 2/28/2019 10:23 PM, Grant Taylor wrote: On 2/28/19 9:33 AM, Mike Marynowski wrote: I'm doing grabs the first

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
I modified it so it checks the root domain and all subdomains up to the email domain. As for your question - if afraid.org has a website then you are correct, all subdomains of afraid.org will not flag this rule, but if lots of afraid.org subdomains are sending spam then I imagine other spam

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Grant Taylor
On 2/28/19 1:24 PM, Luis E. Muñoz wrote: I suggest you look at the Mozilla Public Suffix List at https://publicsuffix.org/ — it was created for different purposes, but I believe it maps well enough to my understanding of your use case. You'll be able to pad the gaps using a custom list. +1

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Grant Taylor
On 2/28/19 12:33 PM, Mike Marynowski wrote: This method checks the *root* domain, not the subdomain. What about domains that have many client subdomains? afraid.org (et al) come to mind. You might end up allowing email from spammer.afraid.org who doesn't have a website because the parent

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Grant Taylor
On 2/28/19 9:33 AM, Mike Marynowski wrote: I'm doing grabs the first available address in this order: reply-to, from, sender. That sounds like it might be possible to game things by playing with the order. I'm not sure what sorts of validations are applied to the Sender: header. (I don't

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Bill Cole
On 28 Feb 2019, at 21:10, Mike Marynowski wrote: Thunderbird normally shows reply-to in normal messages...is this something that some MUAs ignore just on mailing list emails or all emails? I cannot keep track all of the irrational things done by all MUAs. I'm not even surprised by anything

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
I'm pretty sure the way I ended up implementing it everything is working fine and it's nice and simple and clean but maybe there's some edge case that doesn't work properly. If there is I haven't found it yet, so if you can think of one let me know. Since I'm sending an HTTP request to all

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
Thunderbird normally shows reply-to in normal messages...is this something that some MUAs ignore just on mailing list emails or all emails? Because I see reply-to on plenty of other emails. On 2/28/2019 3:44 PM, Bill Cole wrote: On 28 Feb 2019, at 14:29, Mike Marynowski wrote: Unfortunately

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Bill Cole
On 28 Feb 2019, at 14:39, Antony Stone wrote: > On Thursday 28 February 2019 at 20:33:42, Mike Marynowski wrote: > >> But scconsult.com does in fact have a website so I'm not sure what you >> mean. This method checks the *root* domain, not the subdomain. > > How do you identify the root domain,

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Bill Cole
On 28 Feb 2019, at 14:33, Mike Marynowski wrote: But scconsult.com does in fact have a website so I'm not sure what you mean. This method checks the *root* domain, not the subdomain. Ah, I see. I had missed that detail. That's likely to have fewer issues, as long as you get the registry

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Bill Cole
On 28 Feb 2019, at 14:29, Mike Marynowski wrote: Unfortunately I don't see a reply-to header on your messages. What do you have it set to? I thought mailing lists see who is in the "to" section of a reply so that 2 copies aren't sent out. The "mailing list ethics" guide I read said to always

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Luis E. Muñoz
On 28 Feb 2019, at 11:53, Mike Marynowski wrote: There are many ways to determine what the root domain is. One way is analyzing the DNS response from the query to realize it's actually a root domain, or you can just grab the ICANN TLD list and use that to make a determination. What I'm

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
There are many ways to determine what the root domain is. One way is analyzing the DNS response from the query to realize it's actually a root domain, or you can just grab the ICANN TLD list and use that to make a determination. What I'm probably going to do now that I'm building this as a

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Antony Stone
On Thursday 28 February 2019 at 20:33:42, Mike Marynowski wrote: > But scconsult.com does in fact have a website so I'm not sure what you > mean. This method checks the *root* domain, not the subdomain. How do you identify the root domain, given an email address? For example, for many years in

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
But scconsult.com does in fact have a website so I'm not sure what you mean. This method checks the *root* domain, not the subdomain. Even if this wasn't the case well, it is what it is. Emails from this mailing list (and most well configured lists) come in at a spam score of -6, so they are

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Antony Stone
On Thursday 28 February 2019 at 20:25:36, Bill Cole wrote: > On 28 Feb 2019, at 13:43, Mike Marynowski wrote: > > On 2/28/2019 12:41 PM, Bill Cole wrote: > >> You should probably put the envelope sender (i.e. the SA > >> "EnvelopeFrom" pseudo-header) into that list, maybe even first. That > >>

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
Unfortunately I don't see a reply-to header on your messages. What do you have it set to? I thought mailing lists see who is in the "to" section of a reply so that 2 copies aren't sent out. The "mailing list ethics" guide I read said to always use "reply all" and the mailing list system takes

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Bill Cole
On 28 Feb 2019, at 13:43, Mike Marynowski wrote: On 2/28/2019 12:41 PM, Bill Cole wrote: You should probably put the envelope sender (i.e. the SA "EnvelopeFrom" pseudo-header) into that list, maybe even first. That will make many messages sent via discussion mailing lists (such as this one)

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Bill Cole
Please respect my consciously set Reply-To header. I don't ever need 2 copies of a message posted to a mailing list, and ignoring that header is rude. On 28 Feb 2019, at 13:28, Mike Marynowski wrote: On 2/28/2019 12:41 PM, Bill Cole wrote: You should probably put the envelope sender (i.e.

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
On 2/28/2019 12:41 PM, Bill Cole wrote: You should probably put the envelope sender (i.e. the SA "EnvelopeFrom" pseudo-header) into that list, maybe even first. That will make many messages sent via discussion mailing lists (such as this one) pass your test where a test of real header domains

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
On 2/28/2019 12:41 PM, Bill Cole wrote: You should probably put the envelope sender (i.e. the SA "EnvelopeFrom" pseudo-header) into that list, maybe even first. That will make many messages sent via discussion mailing lists (such as this one) pass your test where a test of real header domains

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Benny Pedersen
Ralph Seichter skrev den 2019-02-28 18:53: By the way, are you aware of https://www.dnswl.org ? https://www.mywot.com https://www.trustpilot.com

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Ralph Seichter
* Mike Marynowski: > Question though - what is your reply-to address set to in the emails > coming from your email-only domain? We very rarely inject Reply-To, because this might interfere with what the original sender intended. -Ralph

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Ralph Seichter
* Mike Marynowski: > You know what I mean. That's quite an assumption to make, in a mailing list. ;-) > I could just not publish this and keep it for myself and I'm sure that > would make it more effective long term for me, but I figured I would > contribute it so that others can gain some

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Ralph Seichter
* David Jones: > I would like to see an Open Mail Reputation System setup by a working > group of big companies so it would have some weight behind it. Running a smaller business, I have no interest whatsoever in a "group of big companies" having any say in our mail reputation, as you can surely

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Bill Cole
On 28 Feb 2019, at 11:33, Mike Marynowski wrote: Question though - what is your reply-to address set to in the emails coming from your email-only domain? I can't answer for Ralph, but in my case I use a mail-only domain in From for most of my personal mail, and while I usually set Reply-To

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread David Jones
On 2/28/19 10:50 AM, Ralph Seichter wrote: > * Mike Marynowski: > >> And the cat and mouse game continues :) > > It sure does, and that's what sticks in my craw here: For a pro spammer, > it is easy to set up websites in an automated fashion. If I was such a > naughty person, I'd just add one

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
You know what I mean. *Many (not all) of the rules (rDNS verification, hostname check, SPF records, etc) are easy to circumvent but we still check all that. Those simple checks still manage to catch a surprising amount of spam. I could just not publish this and keep it for myself and I'm sure

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Ralph Seichter
* Mike Marynowski: > Everything we test for is easily compromised on its own. That's quite a sweeping statement, and I disagree. IP-based real time blacklists, anyone? Also, "we" is too unspecific. In addition to the stock rules, I happen to maintain a set of custom tests which are neither

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
Why even use a test for something that is so easily compromised? -Ralph Everything we test for is easily compromised on its own.

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Ralph Seichter
* Mike Marynowski: > And the cat and mouse game continues :) It sure does, and that's what sticks in my craw here: For a pro spammer, it is easy to set up websites in an automated fashion. If I was such a naughty person, I'd just add one tiny service that answers "all is well" for every incoming

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
And the cat and mouse game continues :) That said, all the big obvious "email-only domains" that send out newsletters and notifications and such that I've come across in my sampling already have placeholder websites or redirects to their main websites configured. I'm sure that's not always

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Ralph Seichter
* Antony Stone: > Each to their own. Of course. Alas, if this gets widely adopted, we'll probably have to set up placeholder websites (as will spammers, I'm sure). -Ralph

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
I would not do it at all, caching or no caching. Personally, I don't see a benefit trying to correlate email with a website, as mentioned before, based on how we utilise email-only-domains. -Ralph Fair enough. Based on the sampling I've done and the way I intend to use this, I still see

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
Question though - what is your reply-to address set to in the emails coming from your email-only domain? The domain checking I'm doing grabs the first available address in this order: reply-to, from, sender. It's not using the domain of the SMTP server. I did come across some email-only

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Antony Stone
On Thursday 28 February 2019 at 17:14:04, Ralph Seichter wrote: > * Grant Taylor: > > Why would you do it per email? I would think that you would do the > > test and cache the results for some amount of time. > > I would not do it at all, caching or no caching. Personally, I don't see > a

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Ralph Seichter
* Grant Taylor: > Why would you do it per email? I would think that you would do the > test and cache the results for some amount of time. I would not do it at all, caching or no caching. Personally, I don't see a benefit trying to correlate email with a website, as mentioned before, based on

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Andrea Venturoli
On 2/28/19 3:40 PM, Mike Marynowski wrote: Right now the test plugin I've built makes a single HTTP request for each email while I evaluate this but I'll be building a DNS query endpoint or a local domain cache to make it more efficient before putting it into production. Please keep us

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
Just one more note - I've excluded .email domains from the check as I've noticed several organizations using that as email only domains. Right now the test plugin I've built makes a single HTTP request for each email while I evaluate this but I'll be building a DNS query endpoint or a local

Re: spamassassin trunk

2019-02-28 Thread Antony Stone
On Thursday 28 February 2019 at 15:26:57, Benny Pedersen wrote: > Antony Stone skrev den 2019-02-28 14:56: > > On Thursday 28 February 2019 at 14:44:05, Benny Pedersen wrote: > >> where is it ? > > > > A Google search for "spamassassin trunk" gives me > >

Re: spamassassin trunk

2019-02-28 Thread Kevin A. McGrail
On 2/28/2019 9:26 AM, Benny Pedersen wrote: > Antony Stone skrev den 2019-02-28 14:56: >> On Thursday 28 February 2019 at 14:44:05, Benny Pedersen wrote: >> >>> where is it ? >> >> A Google search for "spamassassin trunk" gives me >> https://wiki.apache.org/spamassassin/DownloadFromSvn as the

Re: spamassassin trunk

2019-02-28 Thread Benny Pedersen
Antony Stone skrev den 2019-02-28 14:56: On Thursday 28 February 2019 at 14:44:05, Benny Pedersen wrote: where is it ? A Google search for "spamassassin trunk" gives me https://wiki.apache.org/spamassassin/DownloadFromSvn as the first result, and following the "Trunk" link there takes me

Re: Spam rule for HTTP/HTTPS request to sender's root domain

2019-02-28 Thread Mike Marynowski
I've tested this with good results and I'm actually not creating any HTTPS connections - what I've found is a single HTTP request with zero redirections is enough. If it returns a status code >= 400 then you treat it like no valid website, and if you get a < 400 result (i.e. a 301/302 redirect

Re: spamassassin trunk

2019-02-28 Thread Antony Stone
On Thursday 28 February 2019 at 14:44:05, Benny Pedersen wrote: > where is it ? A Google search for "spamassassin trunk" gives me https://wiki.apache.org/spamassassin/DownloadFromSvn as the first result, and following the "Trunk" link there takes me to

spamassassin trunk

2019-02-28 Thread Benny Pedersen
where is it ?

whitelist_from_rcvd hits only sometimes

2019-02-28 Thread Helmut Schneider
Hi, I'm trying to find out why a message sometimes hits whitelist_from_rcvd and sometimes does not. I checked the headers again and again but cannot see the difference. whitelist_from_rcvd quarant...@eu.quarantine.symantec.com messagelabs.com whitelist_from_rcvd