Re: MS-relayed spam

[David Jones via users]

I would report this to Microsoft Abuse and setup local rules that add a point 
or two something like this:

header BAD_O365_SENDER  X-OriginatorOrg =~ /.*\.onmicrosoft\.com$/

With a threshold of 6.2, you might want to consider either lowering that a 
little or bumping up some default scores for some of the "worse" rules.

Most legit senders should not be using their onmicrosoft.com for their primary 
address but there are a few that I have seen over the years so I also have a 
counter rule to subtract a point or two for specific onmicrosoft.com 
subdomains. 

On 1/1/24, 3:29 PM, "Charles Sprickman" mailto:sp...@bway.net>> wrote:


EXTERNAL EMAIL: This message originated outside of ENA. Use caution when 
clicking links, opening attachments, or complying with requests. Click the 
"Phish Alert Report" button above the email, or contact MIS, regarding any 
suspicious message.

Hi all,

Full headers are here as well: https://pastebin.com/wHNmnvtE 


I'm not really following what's going on here - a few things confuse me...

- the empty from envelope, which I thought was more of a "bounce" thing
- that it does seem formatted like a bounce
- across multiple servers I'm seeing a ton more spam just like this the past 
few weeks coming in via MS
- I had assumed that MS (or gmail, or any large provider) would be a bit more 
tuned to this kind of abuse

Anyone else seeing this and if so, what mitigations are you doing in SA?

To me, it appears that a company with some kind of on-prem email server is 
using MS' inbound/outbound filtering/relaying for their email, and I'm assuming 
that the company (acquiretm dot com) has compromised account(s) being used for 
spam, and that this type of account is valuable since it's relayed through a 
somewhat "trusted" entity (MS). Stumped on the empty envelope from though...

Thanks,
Charles

Full headers inline:


Return-Path: 
Delivered-To: myem...@mydomain.com 
Received: from mail.MYDOMAIN.COM (mail.MYDOMAIN.COM [207.99.1.2])
by mail.MYDOMAIN.COM (Postfix) with ESMTP id 62E4ACCE44
for mailto:myem...@mydomain.com>>; Mon, 1 Jan 2024 
14:23:33 -0500 (EST)
X-Virus-Scanned: amavisd-new at MYDOMAIN.COM
X-Spam-Flag: NO
X-Spam-Score: 3.971
X-Spam-Level: ***
X-Spam-Status: No, score=3.971 tagged_above=-100 required=6.2
tests=[ARC_SIGNED=0.001, ARC_VALID=0.001, BAYES_00=-1.9,
DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
FORGED_SPF_HELO=1, FREEMAIL_FROM=0.001, FROM_LOCAL_NOVOWEL=0.5,
HK_RANDOM_FROM=0.001, HTML_IMAGE_ONLY_24=1.618, HTML_MESSAGE=0.001,
MIME_HTML_ONLY=0.1, RCVD_IN_DNSWL_NONE=-0.0001,
RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_VALIDITY_RPBL=1.31,
SCC_BODY_URI_ONLY=1.44, SPF_HELO_PASS=-0.001, T_REMOTE_IMAGE=0.01,
T_SCC_BODY_TEXT_LINE=-0.01] autolearn=no autolearn_force=no
Received: from mail.MYDOMAIN.COM ([207.99.1.2])
by mail.MYDOMAIN.COM (mail.MYDOMAIN.COM [207.99.1.]) (amavisd-new, port 10024)
with ESMTP id y8UwjrBjDDCO for mailto:myem...@mydomain.com>>;
Mon, 1 Jan 2024 14:23:31 -0500 (EST)
Received: from NAM11-DM6-obe.outbound.protection.outlook.com 
(mail-dm6nam11hn2245.outbound.protection.outlook.com [52.100.172.245])
(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by mail.MYDOMAIN.COM (Postfix) with ESMTPS id 731A6CCE43
for mailto:myem...@mydomain.com>>; Mon, 1 Jan 2024 
14:23:31 -0500 (EST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=Icl1NbdVBzy5nVKV4XGHyD5lhcUdtzirTQuOX40QfE0Qb4eogob5tBOWT7T7oxZ6O7oogwqarlyCmJXZfKwxDknw8W/1q9UzYGmNu0vt9l/C/TAQGHd2qdDo7k/S5rA/VkvSbwsWsPlPzHM5gpPvERtV1AwGRibQFb7IAJkW1bL6aTyG8R2JHPyDtSE5hG+0/XFuct7sSqoyr8J1hv7cOP6ZsOmlfLFuKxYoAEqFdi0qCsQD/CjfFzFNcaj9Sas09hbA1E/lEU5lf43EJFPOUX9ieGQA292aleu0PO2lqaU+TOwrr9UdnSHPyo89vQUHCiMd9+4ZMb51dxkvx6dLWQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=cMMl8FFbE2iyyDXVN5kGmj7djfYu1Ef14DADjnKqLVc=;
b=gBRRLW2K0klYaRjOr+bNZO7zS3m+Kb+mkggilqYBqELoa12h3G5gwGFye+aLoJjtPSDnS1d0/GUkPYWm2/JlQZtoKmq4YAqwA4tnT2HYRcckobGDbhOcaop7wKmcQutiBxdr2iG8Hjmbvkf6jkP2AHL9kVqZv73Byv60sg1djmVaNHR+2qJd3vyQ3kepYsngd9QtdsyjjFBb+VjyItwaijKmjO4IBSIr4X5i5CmK+v67YoalMVjoXnKaMEpK/4Qh3Eh5zyzGHjdT7+QzK/T4cDSu+1XA+rHcK7G4/BTwLRs+NBTOYMT52Zr4eo5462nuo/ITG3+SjPM9g8QXkfJ06Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none (sender ip is
193.176.158.140) smtp.rcpttodomain=MYDOMAIN.COM smtp.helo=mail.acquiretm.com;
dmarc=none action=none header.from=x1r862t.onmicrosoft.com; dkim=none
(message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=x1r862t.onmicrosoft.com; s=selector1-x1r862t-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;

Re: Spamassassin always says DKIM_INVALID

[David Jones]

Do you have anything modifying the Subject or altering the message body (like a 
signature/disclaimer or external email warning) after opendkim and before the 
spamass-milter?

From: Alex Woick 
Date: Tuesday, January 14, 2020 at 7:38 AM
To: "users@spamassassin.apache.org" 
Subject: Spamassassin always says DKIM_INVALID

Spamassassin (3.4.3, the same with previous) declares all or almost all the 
incoming DKIM-signed messages as DKIM_INVALID, and I'm not understanding why.
I'm running opendkim on the mail server as milter with Postfix, and the 
opendkim headers say the same dkim signatures are all valid.

Example headers of some mail from this list.
Opendkim says ok:

Authentication-Results: mail.wombaz.de;

dkim=pass (2048-bit key) header.d=linkcheck.co.uk 
header.i=@linkcheck.co.uk header.b="PXrrNHdB"


But Spamassassin says it's invalid:

X-Spam-Status: No, score=-15.5 required=5.0 tests=BAYES_00,DKIM_ADSP_ALL,

DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,

MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS,TXREP,

USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.3


Link to complete message:
https://pastebin.com/raw/1DLtnuRX

Spamassassin is running as spamc/spamd, and is embedded in Postfix with 
spamass-milter. System is running on CentOS 7.

Postfix milter config is this:

smtpd_milters =

  unix:/var/run/opendkim-postfix/sock,

  unix:/var/run/opendmarc-postfix/sock,

  unix:/var/run/clamav-milter/clamav-milter.socket,

  unix:/run/spamass-milter/postfix/sock


Any idea how to find out why Spamassassin isn't able to successfully verify 
dkim sigs, while at the same time Opendkim says it's valid? I just activated 
the dkim plugin of Spamassassin but didn't configure anything dkim-related, 
since there is nothing specific to do.

Alex

Re: DMARC_REJECT?

[David Jones]

On 11/16/19 12:19 AM, Dominic Raferd wrote:
> 
> 
> On Fri, 15 Nov 2019 at 21:17, Kevin A. McGrail  <mailto:kmcgr...@apache.org>> wrote:
> 
> Good idea.  This is done.
> 
> On 11/15/2019 11:49 AM, David Jones wrote:
>  > Perhaps it needs to be named KAM_DMARC_REJECT to make it obvious
> that it
>  > came from the KAM.cf and have a default score of 0.001?
> 
> 
> I believe only the renaming has been done, the default score remains 10; 
> so anyone overriding the default score (that would be, er, me) needs to 
> update their local settings for the new name.

Yes the rename was the only thing done.  The default score should be 
0.001 in KAM.cf then local overrides and meta rules could be used to 
bump up the score as needed.

The only way to get complete/true DMARC support in SA is to install 
OpenDMARC as a milter and then setup local rules to use the headers it 
adds that are specific to the AuthservID value in the 
/etc/opendmarc/opendmarc.conf.

We should add default rules to the SA ruleset that would utilize 
OpenDMARC headers if they were present similar to how SPF checks can use 
Received-SPF and Authentication-Results headers on internal headers.

Any perl people out there want to take a shot at a DMARC plugin that 
would use Authentication-Results nternal headers?


Examples:

Authentication-Results: smtp.ena.net; dkim=none
Authentication-Results: smtp.ena.net; dmarc=pass (p=none dis=none) 
header.from=dmarc.org
Authentication-Results: smtp.ena.net;
  dkim=pass (1024-bit key) header.d=dmarc.org header.i=@dmarc.org
Authentication-Results: smtp.ena.net; spf=none (mailfrom)
Authentication-Results: smtp.ena.net;
  dkim=pass (2048-bit key) header.d=dmarc.org header.i=@dmarc.org
Authentication-Results: smtp.ena.net; spf=pass (mailfrom) 
smtp.mailfrom=ncas.us-cert.gov (client-ip=208.42.190.161; 
helo=mailer190161.service.govdelivery.com; 
envelope-from=messa...@ncas.us-cert.gov; receiver=b...@example.com)
Authentication-Results: smtp.ena.net; dmarc=pass (p=reject dis=none) 
header.from=ncas.us-cert.gov
Authentication-Results: smtp.ena.net;
  dkim=pass (2048-bit key) header.d=ncas.us-cert.gov 
header.i=@ncas.us-cert.gov

-- 
David Jones

Re: DMARC_REJECT?

[David Jones]

On 11/15/19 12:35 AM, Kevin A. McGrail wrote:
> The DMARC Reject rule is about whether a domain has failed DKIM and has
> a DMARC reject policy.  I will add descriptions to these rules ASAP.
> Thanks.
> 
> We have encapsulated the rules in a check for DKIM and SPF.
> 
> Best to report issues with KAM.cf as noted in the file :-)  Happy to
> look at samples.  I would imagine you might have something breaking DKIM
> in your environment with FPs as my first guess.  We've had this in
> production and so does another ISP with no other FPs reported.
> 

While I am for this rule helping all SA instances with KAM.cf added, 
it's pretty risky to put this rule in with a default score higher than 
1.0 as there are so many ways that SA can be launched/integrated.

Perhaps it needs to be named KAM_DMARC_REJECT to make it obvious that it 
came from the KAM.cf and have a default score of 0.001?

I have my own rule for DMARC_REJECT that is tied closely to the headers 
added by OpenDMARC which is going to be more reliable / less risky due 
to it being linked to the MTA as a milter.

If SA is being run post MTA (i.e. inside Thunderbird) then any filtering 
can change the content to remove potentially bad attachments, add an 
"EXTERNAL" warning to the Subject or body, etc. which will break DKIM 
signing.

-- 
David Jones

Re: 3.4.3 release Re: DMARC_REJECT?

[David Jones]

On 11/15/19 1:02 AM, Kevin A. McGrail wrote:
> 
>> how more longer will it take to make 3.4.3 realease by looking ? :=)
> 
> Right now, I'm working through bugs found by myself and my staff to make
> sure the release is up to snuff.
> 
> The biggest delay is a lack of feedback and testing.  Please grab rc6
> and give feedback.
> 

I have been running 3.4.3 rc6 for a few days in production and no 
problems so far on my cluster of 12 SA servers with a pretty good volume 
of emails (about 600,000 per day hit SA).

-- 
David Jones

Re: Advanced Computer System Repair

[David Jones]

On 9/2/19 7:39 PM, Loren Wilton wrote:
>> Hi Loren
>> If you could add the source of the mail you get, the SA devs could 
>> take a look
>> at it as well and provide a better answer for you.
> 
> Ok, here is one from today with a few fields edited for slight privacy 
> on my part.
> Note that they have their own address they use for the replies, which I 
> think (without looking) is pretty common in most of them.
> 
>     Loren
> 
> Return-Path: 
> Received: from mail.earthlink.net [209.86.93.211]
> for  (single-drop); Sun, 01 Sep 2019 22:56:19 -0700 (PDT)
> Received: from noehlo.host ([209.86.89.133])
> by mdl-afraid.atl.sa.earthlink.net (EarthLink SMTP Server) with SMTP id 
> 1I4FiW4fu3Nl36X0; Mon, 2 Sep 2019 01:55:07 -0400 (EDT)
> Received-SPF: pass (ibscan-saipan.atl.sa.earthlink.net: domain of 
> computer-news.pro designates 158.69.197.183 as permitted sender) 
> client-ip=158.69.197.183; envelope-from=cont...@computer-news.pro; 
> helo=notifications;Return-Path: 
> Received: from notifications ([158.69.197.183])
> by ibscan-saipan.atl.sa.earthlink.net (EarthLink SMTP Server) with ESMTP 
> id 1I4FiW3sU3PGoUl1
> for ; Mon, 2 Sep 2019 01:55:06 -0400 (EDT)
> Received: by notifications (Postfix, from userid 0)
> id 3A5A0116157; Mon,  2 Sep 2019 01:55:06 -0400 (EDT)
> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=computer-news.pro;
> s=mail; t=1567403706;
> bh=FoloiwIHxTIWA6HaVG4htahENDXhoOLNOUYPE6kCu6c=;
> h=To:Subject:Date:From:Reply-To:List-Unsubscribe:From;
> b=CQ6a0sxkuAl8aIkdJyXfphLmVFeYZKBGrja9mDcm3zyM/VWyCPmA17IeB6bii5+Rm
>   xOc9UDKb+9iqAMGlTrunvVtG+e/11hhKdc7/Z8pNVwpK++7YyB3BowlcG5tWxKlPzS
>   in3cbt+KrpwzsxViU2dyz+yS4Ns3nF6PUuhPAtVE=
> To: Loren Wilton 
> Subject: Notice of cancellation
> Date: Sun, 1 Sep 2019 22:55:06 -0700
> From: Advanced Computer System Repair 
> Reply-To: Advanced Computer System Repair 
> Message-ID: <92506301f766fca993db51db72155...@computer-news.pro>
> List-Unsubscribe: 
> <mailto:cont...@computer-news.pro?subject=Unsubscribe>, 
> <http://computer-news.pro/u.php?param=xxx>
> MIME-Version: 1.0
> Content-Type: text/html; charset=iso-8859-1
> Content-Transfer-Encoding: 8bit
> X-Authentication-Results: dkim="pass"; (0:DKIM_STAT_OK: function 
> completed successfully); dmarc="none"; (1); dwl="miss"; den="not exempt"
> X-ELNK-SMM: -+-+105-55-74-70hefd50hedl55
> X-ELNK-AV: 0
> X-ELNK-Info: sbv=0; sbrc=.0; sbf=0b; sbw=000;
> X-NKVIR: Scanned

NOTE: the preferred method to report this to the SA mailing list is to 
post the original with minimal redaction to a service like pastebin.com 
to keep the original formatting intact.

Here's how my SA filters scored it (I think the score is a little high 
because of some formatting issues that would be resolved with pastebin.com):

X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on smtp2n.ena.net
X-Spam-Flag: YES
X-Spam-Level: *
X-Spam-ASN: AS7029 209.86.0.0/16
X-Spam-Status: Yes, score=25.8 required=6.0 tests=BAYES_00,BODY_8BITS,
DCC_CHECK,DKIM_INVALID,DKIM_SIGNED,ENA_BAD_OPTOUT,ENA_BAD_OPTOUT1,
ENA_BAD_OPTOUT2,ENA_BAD_OPTOUT3,ENA_BAD_SPAM,ENA_BAD_SPAM_FREEMAIL,
ENA_BAD_SPAM_FREEMAIL_BAYES_OFFSET,ENA_BAYES_00_OFFSET,
ENA_BAYES_OFFSET,ENA_FREEMAIL,ENA_FREEMAIL_BAD_OPTOUT,
ENA_FREEMAIL_DIGEST,ENA_NO_TO_CC,MISSING_DATE,MISSING_FROM,
MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,PP_MIME_FAKE_ASCII_TEXT,
    SPF_HELO_SOFTFAIL,UNPARSEABLE_RELAY shortcircuit=no autolearn=no
autolearn_force=no version=3.4.2

This would solve the problem locally if you want to put this in your 
local.cf:

blacklist_from *@computer-news.pro

-- 
David Jones

Re: Fake EHLO triggering ALL_TRUSTED

[David Jones]

On 7/5/19 11:30 AM, Henrik K wrote:
> On Fri, Jul 05, 2019 at 03:59:41PM +0000, David Jones wrote:
>> My understanding of the proposed X-Relay-Countries-MUA would be
>> identical to the current X-Relay-Countries except when there is an
>> authenticated MSA, then it would show the country code.
> 
> I've never even thought of this, since it doesn't make sense in my mind.
> Either there's MUA (Auth) or there isn't.
> 
>> If you have
>> written the code (I haven't looked yet) for X-Relay-Countries-MUA to be
>> blank when the MUA is blank then I agree with you and I will have to
>> manage multiple sets of the same rules/scores checking each header.
> 
> Such is life.  There are many many more scenarios where one has to maintain
> many rules and scores.  Things can be automated, documented, copypasted,
> etc.  Normal stuff for operators.  :-)
> 
>> This logic could be designed to provide individual headers and other
>> headers for layers of boundaries.  The layer approach could be very
>> useful for scoring differences using multipliers for higher, less
>> trusted sources.
> 
> This is the latest documentation.  If anyone wants to chime in, there's
> still little time, I don't want to debate after 3.4.3-rc4.
> 
>   X-Relay-Countries   _RELAYCOUNTRY_
> All untrusted relays. Contains all relays starting from the
> trusted_networks border. This method has been used by default since
> early SA versions.
> 
>   X-Relay-Countries-External  _RELAYCOUNTRYEXT_
> All external relays. Contains all relays starting from the
> internal_networks border. Could be useful in some cases when
> trusted/msa_networks extend beyond the internal border and those
> need to be checked too.
> 
>   X-Relay-Countries-All   _RELAYCOUNTRYALL_
> All possible relays (internal + external).
> 

Not sure how this would be helpful since it mixes everything together. 
I guess it could be used as a positive indicator when certain countries 
are not in the list but we kinda have this already.

>   X-Relay-Countries-Auth  _RELAYCOUNTRYAUTH_
> Auth will contain all relays starting from the first relay that used
> authentication. For example, this could be used to check for hacked
> local users coming in from unexpected countries.
> 

Perhaps we need something added like a 3rd option like boundary_networks 
as an authentication boundary beyond trusted_networks?

internal_networks = in our admin control and won't forge headers
trusted_networks = trust to not forge headers (no change)
boundary_networks = works like trusted_networks except:

1. X-Relay-Countries will be populated
2. ESMTPA IP not included in ALL_TRUSTED

Then we don't need to add new headers and no new rules to manage.

-- 
David Jones

Re: Fake EHLO triggering ALL_TRUSTED

[David Jones]

On 7/5/19 9:55 AM, Bill Cole wrote:
> On 5 Jul 2019, at 10:30, David Jones wrote:
> 
>> On 7/5/19 9:09 AM, Bill Cole wrote:
>>> On 5 Jul 2019, at 9:37, David Jones wrote:
>>>
>>
>> I believe the only change would be the Relay-Countries value would have
>> country codes in it.
> 
> Yes, which it shouldn't.
> 
> It may sound weird, but it is true that I work with 2 mostly unrelated 
> mail systems where mail comes in via MSAs whose authentication is 
> trustworthy from end-users who live and/or travel in places that send 
> those systems very little legitimate mail via untrusted/unauthenticated 
> sources.
> 

This could be handled pretty easily with a local meta rule if one wanted 
to subtract points for a subset of senders that also hit untrusted 
country codes to offset the additional score from that country hit.

Those are mail servers that you actually manage so are they in the 
internal_networks?  This proposal is only intended to change the way 
trusted_networks are handled.

It seems like the internal_networks and trusted_networks are being used 
for two different purposes that don't completely overlap.

1. Trusted to not forge the Received or X-Originating-IP headers.
2. Both make up the ALL_TRUSTED rule hit and include the authenticated 
ESMTPA which can include spam from a compromised account.

My servers are my problem to detect and handle compromised accounts.  I 
can control these with local SA rules because I have many options at my 
disposal.  For example, using swatch on my own logs to trigger country 
code lookups to detect the same user logging in from two different 
untrusted countries within X minutes which would be unlikely.

Perhaps we need something added like a 3rd option like boundary_networks?

internal_networks = in our admin control and won't forge headers
trusted_networks = trust to not forge headers (no change)
boundary_networks = works just like trusted_networks but 
X-Relay-Countries will fire.

-- 
David Jones

Re: Fake EHLO triggering ALL_TRUSTED

[David Jones]

On 7/5/19 9:51 AM, Henrik K wrote:
> On Fri, Jul 05, 2019 at 02:46:16PM +0000, David Jones wrote:
>>
>> I am completely OK with switching to a new X-Relay-Countries-MUA header
>> as long as it works just like the current X-Relay-Countries when there
>> is no MUA.  If it's differnt logic or an extra header to check, then
>> that would mean duplicating and managing another set of rules and scores
>> for dozens/hundreds of country codes.
>>
>> In other words, could the new X-Relay-Countries rules be ordered so the
>> each one also includes the lower ones like syslog works?
> 
> I don't like it.  There's specific function for the headers.  Someone might
> want to check specific countries for authenticated users, which might not be
> the same countries than for generic relay checks.
> 

If there truly are specific functions for each header with different 
logic then I agree that they should be separate.

My understanding of the proposed X-Relay-Countries-MUA would be 
identical to the current X-Relay-Countries except when there is an 
authenticated MSA, then it would show the country code.  If you have 
written the code (I haven't looked yet) for X-Relay-Countries-MUA to be 
blank when the MUA is blank then I agree with you and I will have to 
manage multiple sets of the same rules/scores checking each header.

This logic could be designed to provide individual headers and other 
headers for layers of boundaries.  The layer approach could be very 
useful for scoring differences using multipliers for higher, less 
trusted sources.

-- 
David Jones

Re: Fake EHLO triggering ALL_TRUSTED

[David Jones]

On 7/5/19 9:36 AM, Henrik K wrote:
> On Fri, Jul 05, 2019 at 02:32:42PM +0000, David Jones wrote:
>> On 7/5/19 9:03 AM, Henrik K wrote:
>>> On Fri, Jul 05, 2019 at 01:37:50PM +, David Jones wrote:
>>>>
>>>> For the sake of others, it would be beneficial if the default behavior
>>>> of X-Relay-Countries changed to the X-Relay-Countries-MSA.
>>>
>>> I renamed it X-Relay-Countries-MUA since it's more describing.  It lists all
>>> after the MSA itself.
>>>
>>> What you suggest is not possible, since -MUA will be empty if there isn't
>>> any MSA (and MUA after that) in the received chain.  If this is unclear,
>>> please let me know how to document it better.  :-)
>>>
>>
>> If the -MUA is empty, wouldn't it simply work like it does today and
>> RelayCountry would skip all trusted relays?  In this example it would
>> not fire at all just like it already did.
> 
> That's still breaking backwards compatibility.  People should expect
> X-Relay-Countries to be empty, even when there is MSA used.
> 
> It doesn't really cost anything to create a new header for this.  Those who
> want to use it for MSA/MUA identification, can do so with the new header.
> 
> PS. Check out the bug, I'm still contemplating a few things...
> 

I am completely OK with switching to a new X-Relay-Countries-MUA header 
as long as it works just like the current X-Relay-Countries when there 
is no MUA.  If it's differnt logic or an extra header to check, then 
that would mean duplicating and managing another set of rules and scores 
for dozens/hundreds of country codes.

In other words, could the new X-Relay-Countries rules be ordered so the 
each one also includes the lower ones like syslog works?

-- 
David Jones

Re: Fake EHLO triggering ALL_TRUSTED

[David Jones]

On 7/5/19 9:03 AM, Henrik K wrote:
> On Fri, Jul 05, 2019 at 01:37:50PM +0000, David Jones wrote:
>>
>> For the sake of others, it would be beneficial if the default behavior
>> of X-Relay-Countries changed to the X-Relay-Countries-MSA.
> 
> I renamed it X-Relay-Countries-MUA since it's more describing.  It lists all
> after the MSA itself.
> 
> What you suggest is not possible, since -MUA will be empty if there isn't
> any MSA (and MUA after that) in the received chain.  If this is unclear,
> please let me know how to document it better.  :-)
> 

If the -MUA is empty, wouldn't it simply work like it does today and 
RelayCountry would skip all trusted relays?  In this example it would 
not fire at all just like it already did.

-- 
David Jones

Re: Fake EHLO triggering ALL_TRUSTED

[David Jones]

On 7/5/19 9:09 AM, Bill Cole wrote:
> On 5 Jul 2019, at 9:37, David Jones wrote:
> 
>> For the sake of others, it would be beneficial if the default behavior
>> of X-Relay-Countries changed to the X-Relay-Countries-MSA.
> 
> Definitely not for 3.4.3. Preferably not at all. While I agree in 
> principle with having some way to trust machines as honest without 
> trusting their authentication systems to be bulletproof, that shouldn't 
> involve changing a useful stable feature in a way that will break 
> reasonable configurations. That change would cause substantial false 
> positives at some sites if deployed without carefully considered 
> preparation. It would be a poison pill for packagers who value stability.
> 
> 

I believe the only change would be the Relay-Countries value would have 
country codes in it.  We aren't suggesting changing any other logic so 
the ALL_TRUSTED would still hit and RBLs would not be check on 
authenticated IPs.

Is your concern the RBL checks on those authenticated IPs?

-- 
David Jones

Re: Fake EHLO triggering ALL_TRUSTED

[David Jones]

On 7/5/19 1:54 AM, Henrik K wrote:
> On Fri, Jul 05, 2019 at 09:50:35AM +0300, Henrik K wrote:
>> On Fri, Jul 05, 2019 at 02:42:28AM +0000, David Jones wrote:
>>> Maybe allow the RelayCountry check to happen on the msa network or the
>>> first relay?
>>>
>>> Or something like trusted_countries that could provide a limit/boundary
>>> to the trust of trusted_networks?
>>>
>>> Compromised accounts often get abused from foreign/unusual countries.  I
>>> have meta rules and DWL/DBL for emails combined with RelayCountry but
>>> these are useless in this situation.
>>
>> Perhaps adding new datadata X-Relay-Countries-External would be enough, it
>> would check all external IPs (vs untrusted for the default
>> X-Relay-Countries).  I think it could use useful in this and other
>> situations when there are lots of additional trusted networks.
>>
>> Maybe also the X-Relay-Countries-MSA to check client IPs from msa_networks.
>>
>> Might even make it to 3.4.3 if KAM wants to delay rc4 just a little bit more.
>> :-D
> 
> See https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7731
> 

Thank you Henrik!  Very nice and quick work.

Depending on how this sorts out, I may have to duplicate all of my 
existing X-Relay-Countries rules and add these to other meta rules with 
an "or" but that would be fine.

For the sake of others, it would be beneficial if the default behavior 
of X-Relay-Countries changed to the X-Relay-Countries-MSA.  That 
shouldn't be too much of a change to cause adverse effects since it 
would still be hitting ALL_TRUSTED and no RBL checks happen.  If the 
RelayCountry.pm plugin was not enabled (default?) then this would result 
in no change for the majority of SA instances and only enhance those 
that have enabled it.

-- 
David Jones

Re: Fake EHLO triggering ALL_TRUSTED

[David Jones]

On 7/4/19 6:35 PM, Bill Cole wrote:
> On 4 Jul 2019, at 16:59, David Jones wrote:
>
>> It seems like authenticated mail submission should only apply to
>> internal_networks and not extend out to the trusted_networks.
>
> No. See https://wiki.apache.org/spamassassin/DynablockIssues.
>
> In short: if you don't trust the authentication of your 
> trusted_networks, you will end up rejecting their legitimately 
> authenticated users for being on PBL-like DNSBLs.
>
> Arguably the trusted/internal/msa network model could use a rethink 
> since today we see so many cracked accounts. If you have specific 
> ideas other than redefining an existing client class back to a broken 
> former handling, I'd be interested in that discussion.
>
Maybe allow the RelayCountry check to happen on the msa network or the 
first relay?

Or something like trusted_countries that could provide a limit/boundary 
to the trust of trusted_networks?

Compromised accounts often get abused from foreign/unusual countries.  I 
have meta rules and DWL/DBL for emails combined with RelayCountry but 
these are useless in this situation.

>> I trust the 96.4.165.21 mail server to not forge the Received headers
>> but compromised accounts happen.
>>
>> Is there another way to accomplish checking the that 88.233 IP as the
>> last-external without stripping off the "A" in ESMTPA at the MTA before
>> SA sees it?
>
> Not without checking the IP of every authenticated client of 
> trusted_networks. I think you can do it with the '-firsttrusted' 
> suffix for DNSBL rules so that you could define a rule that looks for 
> SBL & XBL hits but not PBL hits in Zen.
>
I was thinking about stripping the "A" off othe ESMTPA just for this 
email server since I know that customer's mail flow should never have 
email originate from Turkey and most other countries outside of the US.  
I don't care so much about RBLs but the country is a big indicator of a 
compromised account.

Re: Fake EHLO triggering ALL_TRUSTED

[David Jones]

On 7/4/19 2:28 PM, RW wrote:
> On Thu, 4 Jul 2019 19:11:43 +
> David Jones wrote:
> 
>> Just had a compromised account on one of my customer's mail servers
>> (96.4.156.21) try to blast out phishing email.  This 96.4 IP is our
>> customer space so it's in my trusted_networks since it will not forge
>> the Received header.
> 
> This is nothing to do with ehlo, it hit ALL_TRUSTED because it's
> authenticated mail submission into the trusted network.
> 

Thank you for this information.

It seems like authenticated mail submission should only apply to 
internal_networks and not extend out to the trusted_networks.

I trust the 96.4.165.21 mail server to not forge the Received headers 
but compromised accounts happen.

Is there another way to accomplish checking the that 88.233 IP as the 
last-external without stripping off the "A" in ESMTPA at the MTA before 
SA sees it?

>> The 88.233 IP is from Turkey (88.233.47.16.dynamic.ttnet.com.tr) and
>> should have triggered a number of rules based on the RelayCountry
>> plugin.
>>
>> This email should not have hit ALL_TRUSTED and should have done
>> RelayCountry and ASN lookups on 88.233.47.16.
>>
>>
>> Received: from mail.lced.net (mail.lced.net [96.4.156.2])
>>by smtp5i.ena.net (Postfix) with ESMTP id DF9421480F90
>>for ; Thu, 4 Jul 2019 12:56:42 -0500
>> (CDT) Received: from 192.168.1.2 (unknown [88.233.47.16])
>>    by mail.lced.net (Postfix) with ESMTPA id 8F22630961D6D
>>for ; Thu, 4 Jul 2019 12:56:40 -0500
>> (CDT)
>>
>>

-- 
David Jones

Fake EHLO triggering ALL_TRUSTED

[David Jones]

Just had a compromised account on one of my customer's mail servers 
(96.4.156.21) try to blast out phishing email.  This 96.4 IP is our 
customer space so it's in my trusted_networks since it will not forge 
the Received header.

The 88.233 IP is from Turkey (88.233.47.16.dynamic.ttnet.com.tr) and 
should have triggered a number of rules based on the RelayCountry plugin.

This email should not have hit ALL_TRUSTED and should have done 
RelayCountry and ASN lookups on 88.233.47.16.


Received: from mail.lced.net (mail.lced.net [96.4.156.2])
  by smtp5i.ena.net (Postfix) with ESMTP id DF9421480F90
  for ; Thu, 4 Jul 2019 12:56:42 -0500 (CDT)
Received: from 192.168.1.2 (unknown [88.233.47.16])
  by mail.lced.net (Postfix) with ESMTPA id 8F22630961D6D
  for ; Thu, 4 Jul 2019 12:56:40 -0500 (CDT)


-- 
David Jones

Re: How to create my personal RBL

[David Jones]

aling with a standalone
> list,
>  >but the  OP did ask specifically about using database queries to
>  >implement a blacklist, so I thought it was worthwhile to tell him
> what's
>  >involved in doing that.
> 
> No. The OP wanted to store data in DB to avoid restarting SA, not
> mentioning
> any other specific reason to use DB.
> 
> using DNSBL does avoid restarting SA and does not require any
> plugin, which
> is a great advantage.
> 
> we are trying to provide described requirements, while avoiding proposed
> complicated solutions.
> 
>  >For all I know the OP either has a similar archive or is intending to
>  >implement one: searching for a specific message with a database
> tool is
>  >a *lot* faster than ferreting through a set of very large mail folders
>  >with your MUA, though of course the effort of creating and maintaining
>  >the database, mail loader, query tools and SA plugin is non trivial.
> 
> well, if THIS is the real reason...
> 
> -- 
> Matus UHLAR - fantomas, uh...@fantomas.sk <mailto:uh...@fantomas.sk>
> ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Saving Private Ryan...
> Private Ryan exists. Overwrite? (Y/N)
> 


-- 
David Jones

Re: How to create my personal RBL

[David Jones]

On 6/25/19 10:20 AM, Martin Gregorie wrote:
> On Tue, 2019-06-25 at 16:11 +0200, hg user wrote:
>> I'd like to create my own RBL that answers queries about IP, domain or
>> address reputation.
>> Data should be stored in a database (mysql, postgres, redis, etc) so
>> that information can be added/modified/removed without the need to
>> restart spamassassin (I think the simpler solution would be a list in
>> SA...)
>>
>> How can I create this setup?
>>
> You need to build a Perl plugin for Spamassassin that connects to, and
> queries the database together with at least one SA rule that triggers
> the plugin via an eval:plugin_query() call where plugin_query() is a
> plugin function that runs the database query using data extracted from
> the message by SA and returns either 1 (the query found a match in the
> database) or zero (no matches found).
> 
> 
> Martin
> 
> 

Actually the SA part is very simple.  Use the AskDNS SA plugin to do the 
DNS lookup:

askdns  MYRBL_ENV   _SENDERDOMAIN_.dbl.example.com A 
/^127\.0\.0\.2$/
tflags  MYRBL_ENV   nice net
describeMYRBL_ENV   Sender's envelope domain listed in my RBL.
score   MYRBL_ENV

askdns  MYRBL_FROM  _SENDERDOMAIN_.dbl.example.com A 
/^127\.0\.0\.2$/
tflags  MYRBL_FROM  nice net
describeMYRBL_FROM  Sender's From domain listed in my RBL.
score   MYRBL_FROM  0.001


The trickier part is to setup the DNS side.  If you have a single SA 
host, you should already have a local caching DNS server and the 
/etc/resolv.conf and/or the SA DNS setting pointed to 127.0.0.1.

I use PowerDNS Recursor but Unbound or BIND would work fine.

Install rbldnsd for your distro and get it listening on an alternate 
port like 127.0.0.1:530.

https://rbldnsd.io/

Create a text file with domains to block.  This can come from a database 
with a web front-end or whatever you want.  I have a database that I 
push records into from sources of spam and entries by a web interface. 
Then a script does a simple SELECT of the domains to a text file, then 
rsync's it to my 2 DNS servers that my 8 SA servers point to.  Cron this 
for every 2-3 minutes and rbldnsd will gladly detect changes to the 
files without needing to be restarted/reloaded.

I recommend putting a "test" entry at the top of the rbldnsd file so you 
can query test.dbl.example.com from a monitoring system to make sure it 
answers with the expected value.

Then you setup your local caching DNS server to forward the 
dbl.example.com to 127.0.0.1:530.  Note that this "dbl.example.com" 
doesn't have to be a real DNS zone.  It could be "dbl.local" or whatever 
since it's only known by the local DNS server(s) that your SA server(s) 
are pointed to.  These DNS servers should not be accessible by the 
Internet so they should be separate DNS caches dedicated to the SA 
server(s).  If it's only one, then it could all be setup on 127.0.0.1. 
If it's a few, you could put rbldnsd on all of them and still use 
127.0.0.1 and rsync the rbldnsd files to all of them locally.

-- 
David Jones

Re: Mail to local users

[David Jones]

On 6/16/19 4:41 PM, @lbutlr wrote:
> When I send an mail from my home machine to a user who is local to my mail 
> server, SpamAssassin (via spmass-milter) tags the mail as spam entirely 
> because my home IP is in the PBL blacklist. Which of course, it is and it 
> should be.
>
> However, since the mail is actually originating on my server with an 
> authenticated connection, it seems the RBL check should be able to be avoided?
>
> Received: from darth.lan (c-73-14.161.160.hsd1.co.comcast.net [73.14.161.160])
>  by mail.covisp.net(Postfix 3.4.5/8.13.0) with SMTP id unknown;
>  Sun, 16 Jun 2019 15:26:32 -0600
>  (envelope-from )
>
> X-Spam-Status: Yes, score=7.3 required=5.0 tests=BAYES_50,NO_FM_NAME_IP_HOSTN,
>  RCVD_IN_PBL,RCVD_IN_SORBS_DUL,RDNS_DYNAMIC,TO_NO_BRKTS_DYNIP
>  autolearn=no autolearn_force=no version=3.4.2
>
> Obviously I do not want to lower the scores for the three criteria that put 
> me over the limit on these emails, but similarly I do not want my mail to 
> other people on the server getting tagged as spam (nor there mail to me). I 
> have my current IP in the SpamAssassin-milter with the -I flag, but the IP 
> changes and this isn’t a solution for others sending mail to local accounts 
> from local accounts.
>
> Postfix main.cf:
> smtpd_milters =
>  unix:/var/run/spamass-milter.sock,
> milter_connect_macros = j {daemon_name} v {if_name} _
> #milter_default_action = tempfail
> milter_default_action = accept
>
> /usr/local/sbin/spamass-milter -f -p /var/run/spamass-milter.sock -u spamd -e 
> -i 65.121.55.40/29 -i 73.14.161.160 -i 127.0.0.1 -r 10
>
> Seems like the -I fall should be taking care of this for me, at present. But 
> how do I tell spamass-milter not to check for PBL and other similar tests on 
> mails from local users to local users?
>
>
Find the header being added by your Postfix MTA and add a rule to 
subtract points for authenticated senders.  It should have "ESMTPSA" 
when sent by an authenticated account.

header   LOCAL AUTH_SENDER Received =~ /\.example\.com \(Postfix\) 
with ESMTPSA/
score    LOCAL_AUTH_SENDER -4.0

Also, try adding that IP to your internal_networks and see how the 
scoring changes:

internal_networks 73.14.161.160

You may want to meta rule that with something else unique in the headers 
to prevent

Dave

Re: recent update to __STYLE_GIBBERISH_1 leads to 100% CPU usage

[David Jones]

On 5/30/19 8:10 AM, Kevin A. McGrail wrote:
> I would check your spamd logs and see the time each message scan is 
> taking.  My servers can keep tons of spamd running at 100% just because 
> of constant mail.
> 
> In other words, just having 5 processes at full load really doesn't mean 
> much.
> 
> On Thu, May 30, 2019, 09:06 Larry Nedry  <mailto:spamassas...@bluestreak.net>> wrote:
> 
> On 5/29/19 7:32 AM, Karsten Bräckelmann wrote:
>  > That's a non-scoring sub-rule, setting its score to 0 has no effect.
>  > Redefining the rule to disable it is the way to go:
>  >
>  >    meta __STYLE_GIBBERISH_1  0
> 
> FWIW, I've added this to local.cf <http://local.cf>, recompiled, and
> restarted spamd but
> am still seeing CPUs hanging at 100%.
> 
> Larry
> 

I am using MailScanner which launches spamassassin on batches of emails 
and the past couple of weeks I have been getting SA timeouts and being 
killed by MailScanner.  I have never found a way to troubleshoot this as 
I can't reproduce it consistently.  I am using KAM.cf which has the:

meta __STYLE_GIBBERISH_1  0

line but does that need to be:

score __STYLE_GIBBERISH_1  0

to completely disable it?

-- 
David Jones

Re: my spamassassin has serious config problems

[David Jones]

On 5/27/19 5:13 PM, hg user wrote:
> The server was installed and configured by a "zimbra man", a person I 
> fully trust. Since I manage a commercial antivirus/antispam solution 
> that is not properly working for the italian language, I was tasked to 
> join the project in order to understand if we could switch from the 
> proprietary solution to spamassassin.
> 
> I'm now in the process of double-checking the configuration of 
> spamassassin and feeding the bayes engine...
> 
> Testing the system I noticed that spamassassin logged the internal MTAs 
> (including the antivirus server) as external and I asked *the zimbra 
> man* to correct the configuration. He replied it was not necessary. 
> Sorry I didn't specify I asked the person in charge of the system.
> 
> In the end, I need to think about the answer of RW: spamassassin is run 
> by amavis but with no internal servers defined, it uses my internal one 
> as the external. Received header needs some more care, and probably also 
> the list of stop words should be expanded. Probably there is a ratio 
> behind some decisions taken by the developers, but I can't fully 
> understand how the destination address can help on whether a message is 
> spam or not, at least not 6 times.

The internal_networks and trusted_networks are _very important_ to be 
set correctly for a number of reasons, not just Bayes.  This gives SA 
the proper "view" to the outside/Internet.  Keep in mind 
internal_networks is not literally your RFC 1918 internal_networks and 
the trusted_networks are not only ones that you managed/control.

Internal_networks is any public or private IP space that you trust to 
not forge the Received or synthetic received headers like X-Originating-IP.

Trusted_networks can be external/public networks that you know won't 
change or forge the Received or synthetic received headers.

I have recently added all Google and Office 365 IP blocks to my 
trusted_networks to better detect last-external client IPs.  This allows 
for deep Received header inspection since I know that Google and 
Microsoft aren't going to forge those headers.  Very interesting 
information comes out into the open as a result of this.

P.S. To implement/try this extended trusted_networks, set the score for 
ALL_TRUSTED to -0.001 and disable it from shortcircuit'ing.

score   ALL_TRUSTED -0.001
shortcircuit ALL_TRUSTED off

-- 
David Jones

Re: Rule for non-DKIM-signed messages

[David Jones]

On 5/12/19 9:29 PM, Kurt Fitzner wrote:
> On 2019-05-11 23:25, David Jones wrote:
> 
> I don't have anything nearly so elaborate.  But then I don't have the 
> spam volume either.
> 

That's fine.  Just wanted to point out that "one size doesn't fit all" 
for other readers on this list.  The default SA rules have to follow 
RFCs and standard practices in a general way so SA can start out working 
for all then allow each admin to tune it toward the mail flow they have.

> 
> Thanks,
> 
>    Kurt
> -- 
David Jones

Re: Rule for non-DKIM-signed messages

[David Jones]

On 5/10/19 1:16 PM, Kurt Fitzner wrote:
> On 2019-05-10 12:42, Matus UHLAR - fantomas wrote:
> 
>> I wanted to comment OP's mail, but since I don't have DKIM set up, I 
>> wasn't
>> sure it would pass  :-)
> 
> I actually didn't have DKIM signing set up myself until a couple weeks 
> ago.  I had been lazy in setting it for a while, but I had to because 
> the first time I would email anyone on gmail it was going directly to 
> their spam folder.  Hotmail too, to a lesser extent.  But Google is 
> really aggressive with unsigned mail, and they have a strong "it's our 
> way or the highway" policy.
> 
> On 10.05.19 14:48, David Jones wrote:
> 
>>> I caution against this since non-DKIM signed email has no relation to
>>> spam or ham.  How did you come up with the "about 90%" number?  Did you
>>> grep logs to get real numbers over a couple of months?
> 
> I should clarify.  I do get DKIM-signed spam.  I just don't get any 
> non-DKIM-signed ham.  Going back and looking at my archived mail and 
> logs I can see that a) all legitimate emails were DKIM-signed, and b) 
> virtually every message that was not DKIM-signed was spam.  So I intend 
> to assign no ham scoring weight to a message having a DKIM signature, 
> but I do feel pretty safe in assigning a heavy penalty to those mails 
> without it.
> 

Is this for a single mailbox?  If that is the case, then it's fine to 
make a decision like that for a single mailbox.  For those of us running 
mail filtering plaforms for customers, this would be a very bad rule.

I filter for about 60,000 to 80,000 mailboxes (can't tell for sure with 
Exchange accepting everything and bouncing later) and use DKIM_VALID_AU 
heavily with thousands of subdomain entries like:

whitelist_auth *@*.joann.com
whitelist_auth *@*.potterybarn.com
whitelist_auth *@*.aa.com
whitelist_auth *@*.saks.com
whitelist_auth *@*.dominos.com
whitelist_auth *@*.fandango.com

I know for sure that these emails are:

1. System generated and not from user accounts that can be compromised
2. Generated by a mail server under the control or authorized by their 
respective domain owners.

I have an automated system that finds these candidates every week and 
adds them automatically to my SA config file.  This is a whole category 
of email that I don't have to worry about false positives allowing me to 
increase the sensitivity of scores and meta rules to help block 
compromised accounts and zero-hour spam.

My SA servers see millions of emails each week and they handle a lot of 
non-DKIM signed ham.

-- 
David Jones

Re: Rule for non-DKIM-signed messages

[David Jones]

On 5/10/19 1:52 AM, Pedro David Marco wrote:
> Hi Kurt,
> 
> 
> On the contrary, most spam i see is valid DKIM signed...   tons of 
> hacked sites... tons of emails from free trials of big-cheeses...
> 
> Nevertheless...
> 
> meta    NO_DKIM_SIGNED    ! DKIM_SIGNED
> score NO_DKIM_SIGNED        2
> describe NO_DKIM_SIGNED        Email does not have DKIM signature
> 

That alone is too risky to score alone and should be used in a meta rule 
like this:

metaSPAM_NOT_DKIM_SIGNED!DKIM_SIGNED && (MISSING_HEADERS || 
FSL_BULK_SIG || RDNS_DYNAMIC || OTHER_RULE_COMMONLY_SEEN_AS_SPAM)
score   SPAM_NOT_DKIM_SIGNED2
describe SPAM_NOT_DKIM_SIGNED   Spammy characteristics and not DKIM signed


> Pedro.
> 
> 
>  >
>  >On Friday, May 10, 2019, 4:26:46 AM GMT+2, Kurt Fitzner 
>  wrote:
>  >
>  >I've noticed on my mail server that DKIM signing is almost diagnostic of
>  >spam.  Almost no legitimate sender is without DKIM, and about 90% of my
>  >spam is unsigned, so I want to bias non-DKIM-signed heavily towards
>  >spam.  To that end I was wondering if there are any built-in rules I can
>  >activate to score emails that are not DKIM-signed? I'd rather use a
>  >built-in rule than roll my own.

I caution against this since non-DKIM signed email has no relation to 
spam or ham.  How did you come up with the "about 90%" number?  Did you 
grep logs to get real numbers over a couple of months?

Any compromised account from Office 365 (and there are a lot) is going 
to have DKIM_SIGNED by Microsoft's "tenant.onmicrosoft.com" domain which 
means absolutely nothing when determining ham/spam.  All that means is 
it was signed by Microsoft mail servers on the way out.  If DKIM_VALID 
was hit, then it means the spam wasn't modified.

-- 
David Jones

Re: GeoIP2 packages

[David Jones]

On 5/6/19 10:20 AM, Alex wrote:
> Hi all,
> 
> I'm looking for the GeoIP2 and IP Country packages for fedora/CentOS
> needed for the RelayCountry plugin. I believe there were some license
> changes recently that prevent them from being included with the latest
> distributions?
> 
> I believe they also have a ton of dependencies based on the last time
> I tried to build them on my own, including a bunch of maxmind modules.
> 
> Of course I could build them from CPAN, but would obviously prefer the
> ease of a package manager. The best approach for getting this all
> working would be greatly appreciated.
> 

I have found that it's best to do it either all RPM-based or all 
CPAN-based installs.  It's possible to do as many RPMs that exist then 
use CPAN for the rest but mixing is not fun with all of the different 
paths everything get's put into.  As RPMs get older and perl moves on, 
it makes CPAN installs look better and better.

I just went through this over the weekend.  I decided to download perl 
5.28.2 and compile it on my old CentOS 6 boxes that I don't have time to 
rebuild.  Honestly it's not as hard as I thought and now that I have 
compiled it with a prefix of '/usr/local' all I have to do is distribute 
a few directories to rsync the complete config from a master to about a 
dozen secondary SA boxes.

Now I can run 'cpan install GeoIP2' and as long as I have the proper 
devel packages on the master.  Everything is installed on the master and 
then is rsync'd to the secondaries (all running the same version of 
CentOS 6) along with the SA, postfix, opendmarc, opendkim, 
python-policyd-spf, sqlgrey, DCC, fail2ban, ClamAV Unofficial sigs, etc. 
  Then monit running on each box detects the change and reloads/restarts 
services after doing a config check first for that component (postfix, 
clamd, MailScanner, etc.).

FYI, I am doing my GeoIP2 updates monthly from my master and rsync'ing 
the /usr/share/GeoIP out.  Same for ClamAV Unofficial signatures, 
KAM.cf, and SA rule updates.

Dirs:
/usr/local/lib/perl5
/usr/local/lib/perl5/site_perl
/usr/local/share/man
/usr/share/spamassassin  (needed for the TextRep.pm defaults)
/root/.cpan

Then the standard SA dirs:
/var/lib/spamassassin
/etc/mail/spamassassin

Plus any others you need to throw in for add-ons.
/usr/share/GeoIP
/var/clamav
/var/dcc/whiteclnt


1. Make a backup of /etc/mail/spamassassin.
2. Stop any daemons are using spamassasssin.
3. Uninstall the spamassassin RPM if it's installed.
4. Download the perl tgz, extract, and compile it
 ./Configure -des -Dprefix=/usr/local
5. cpan install CPAN
6. cpan install all other modules including Mail::SpamAssassin
7. I copied the following files into /usr/bin from /usr/local/bin to 
overwrite the original distro location.  CentOS 7 will be /bin but it's 
symlinked to /usr/bin so it's essentially the same.

perl
perldoc
cpan
spamassassin
spamd
spamc

I tried symlinks first but some apps didn't like it.  I could have tried 
a hard link but I didn't as they can be confusing and bite you later.

-- 
David Jones

Re: SPF

[David Jones]

On 5/3/19 6:26 PM, Grant Taylor wrote:
> On 5/3/19 5:10 PM, Kevin A. McGrail wrote:
>> I guess if you lose control of your keys and/or your DNS is 
>> compromised, then yes, you have a DKIM issue.
> 
> This brings up a non-repudiation issue introduced by DKIM.
> 

This is similar to saying not to use HTTPS because there's a possibility 
that the web server's certificate private keys and DNS might be 
compromised so I am safer with plain HTTP.  If the DKIM signature aligns 
with the From: header domain and is valid, DKIM_VALID_AU means you know 
the source of the email and that is was not modified.  If the DKIM 
signature is any other domain, then it doesn't mean much from a spam/ham 
perspecitive which is why DKIM_VALID has a tiny default value.

> How can you successfully refute a DKIM-Signature if someone has your 
> signing keys.
> 

DKIM is only one of dozens of tools in the spam-fighting tool bag.  I 
use it in very specific cases when DKIM_VALID_AU rules hit or I use it 
in meta rules so that other conditions have to also be true.


> My quick skim of parts of RFC 6376 makes me think that it is dangerous 
> and discouraged to associate authentication based on DKIM-Signature, 
> even when the d= SDID (?) matches the From: header.
> 

I have been using SPF, DKIM, and DMARC for years now in specific meta 
rules based on patterns noticed in the mail flow through smtp.ena.net 
and it has worked pretty well.

1. Carefully whitelist_auth primary domains with user/human mailboxes 
since they can be compromised.  Some cases are worth the risk for 
difficult partners like legal, HR, financial, etc. with content that 
looks spammy and can hit high Bayes scores.

2. Whitelist_auth subdomains of primary domains are usually safe for 
systems-generated emails and mailing lists (whitelist_auth *.apache.org)


> Yet even more reason to reread RFC 6376 before replying to Bill's email.
> 
> Presently I'm comfortable in thinking that DKIM-Signature validation 
> meaning that the message has not changed in transit.  I'm not (yet) 
> comfortable drawing any conclusions about authentication.
> 

Analyze your email based on DKIM_VALID_AU hits and look for patterns. 
Based on your definition of spam vs UCE vs ham.  If there is enough 
volume, you should see how DKIM_VALID_AU and DMARC can enhance/extend 
SPF accuracy which was your original question.

-- 
David Jones

Re: SPF

[David Jones]

On 5/3/19 11:30 AM, Grant Taylor wrote:
> On 5/3/19 9:48 AM, Bill Cole wrote:
>> An entirely different mechanism (DKIM) exists to verify From headers.
> 
> DKIM is only positive confirmation that the (signed) headers (and body 
> content) has not changed since the signature was applied.
> 

Not completely true as long as domain/DNS control is not compromised.

> DKIM does nothing to verify the authenticity of what was signed (at the 
> time it was signed).
> 

Technically this is correct but the fact that it's signed and matches 
the author's domain in the From: header provides authenticity of the 
origin of the email.  In SA rules this would be DKIM_VALID_AU hits.

For example, Microsoft signs customer emails using the tenant's 
subdomain under onmicrosoft.com.  All this confirms is the email came 
from the Office 365 platform with the original content unmodified. 
Since it doesn't align with the From: domain, DKIM really means nothing 
from a forged/spoofed (negative) perspective.  DKIM can prove the 
positive that it was not forged/spoofed when it aligns and hits 
DKIM_VALID_AU.

> ARC (not DMARC) is a similar signature of what comes in to detect 
> modification down stream.
> 

I am not completely clear on ARC but I though it's objective is to 
provide a "chain of custody" as email goes through mail servers so 
receiving mail servers can authenticate the origin.  I was thinking it's 
something like a combination of SPF (validation) and DKIM (authentication).

-- 
David Jones

Re: SPF

[David Jones]

On 5/3/19 9:47 AM, RW wrote:
> On Fri, 3 May 2019 06:55:40 -0700 (MST)
> user321 wrote:
> 
>> Any reason why SA is checking for SPF against envelope from not the
>> header from?
> 

See the SPF link on this page:

https://blog.returnpath.com/how-to-explain-dmarc-in-plain-english/

> Because that's how SPF works.
> 
>> I am rejecting the SPF_FAIL e-mails on Postfix (-all only), but still
>> spammers can forge the header from field.
>> Can I change SPF plugin to work with header from? If yes how?
>> What are the pros and cons of that?
> 
> Generally you don't want to enforcing a modified standard that no one
> else knows about.
> 
> DMARC solves the problem by requiring that for a DMARC pass from SPF
> the envelope address has to be aligned with the from header address.
> 

FYI,

If opendmarc is setup in your MTA with local SA rules...

DMARC_PASS = SPF_PASS and From: domain aligns with envelope-domain
  _OR_  DKIM_VALID_AU (DKIM_SIGNED and aligns with the From: domain)

I am rejecting DMARC failures of incoming domains with p=reject within 
opendmarc then have these rules in SA:

(change the rule below to match your header in opendmarc.conf)

headerDMARC_PASS Authentication-Results =~ /smtp\.ena\.net; dmarc=pass/
describe  DMARC_PASS DMARC check passed
score DMARC_PASS 0.001

headerDMARC_FAIL Authentication-Results =~ /smtp\.ena\.net; dmarc=fail/
describe  DMARC_FAIL DMARC check failed
score DMARC_FAIL 0.001

headerDMARC_NONE Authentication-Results =~ /smtp\.ena\.net; dmarc=none/
describe  DMARC_NONE DMARC check neutral
score DMARC_NONE 0.001

-- 
David Jones

Re: Spammer in white list aka USER_IN_DEF_SPF_WL

[David Jones]

On 5/2/19 3:11 PM, RW wrote:
> 
> That just means it's a known source of email and not a zombie or IP
> address controlled by an outright spammer. The level of trust is
> described as 'none', that's a lower level than some freemail servers.
> 
> The DKIM signing domain isn't listed at all  on dkimwl.
> 
> 
>>   More importantly, it's [not] listed in Invaluement (IVM or IVM24):
> 
> The Invaluement lists are marketed as low-FP. There's a huge difference
> between being good enough to stay out of Invaluement and being good
> enough for whitelisting at the -15 point level.

Default score for def_whitelist_auth is -7.5.

> 
> 
>>   Every platform has the occassional bad customer that needs
>> to be kicked off
> 
>  From it's website simpliv.com appears to be a company that markets
> online training provided by third-party trainers. In that situation
> simpliv should be managing the lists and enforcing opt-in.
> 
> 
> 
> 

It's removed in SVN so it should get taken out tomorrow night as long as 
the rules promotion is working.

-- 
David Jones

Re: Spammer in white list aka USER_IN_DEF_SPF_WL

[David Jones]

On 5/1/19 10:15 PM, David Jones wrote:
> On 5/1/19 6:04 PM, RW wrote:
>> On Wed, 1 May 2019 10:39:08 -0700 (MST)
>> jandev wrote:
>>
>>> David,
>>>
>>> I tried to send the original email to the email address you
>>> requested. But your mail hoster blocks (554 5.7.1) my TLDs.
>>
>> I doesn't really matter, you posted a link to pastebin on the list.
>>
>> It passed SPF with the envelope domain bounce.comm06.simpliv.com
>> which matches:
>>
>> def_whitelist_auth *@*.simpliv.com
>>
> 
> 129.41.222.236 has a senderscore.org score of 94 currently and is listed
> in dnswl.org as score but do not block outright.  More importantly, it's

I meant to say "it's NOT listed" in IVM which is a very accurate RBL.

> listed in Invaluement (IVM or IVM24):
> 
> http://multirbl.valli.org/lookup/129.41.222.236.html
> 
> The email headers that were posted in pastebein.com are from mass
> marketer that has a valid unsubscribe header/link.
> 
> I wouldn't classify that email as spam unless there were multiple
> reports of them not honoring the unsubscribe or not handling abuse
> reports.  Every platform has the occassional bad customer that needs to
> be kicked off so most RBLs (good ones anyway) will allow for a small
> amount of UCE before hitting the threshold to be listed/blocked.
> 

-- 
David Jones

Re: Spammer in white list aka USER_IN_DEF_SPF_WL

[David Jones]

On 5/1/19 6:04 PM, RW wrote:
> On Wed, 1 May 2019 10:39:08 -0700 (MST)
> jandev wrote:
> 
>> David,
>>
>> I tried to send the original email to the email address you
>> requested. But your mail hoster blocks (554 5.7.1) my TLDs.
> 
> I doesn't really matter, you posted a link to pastebin on the list.
> 
> It passed SPF with the envelope domain bounce.comm06.simpliv.com
> which matches:
> 
> def_whitelist_auth *@*.simpliv.com
> 

129.41.222.236 has a senderscore.org score of 94 currently and is listed 
in dnswl.org as score but do not block outright.  More importantly, it's 
listed in Invaluement (IVM or IVM24):

http://multirbl.valli.org/lookup/129.41.222.236.html

The email headers that were posted in pastebein.com are from mass 
marketer that has a valid unsubscribe header/link.

I wouldn't classify that email as spam unless there were multiple 
reports of them not honoring the unsubscribe or not handling abuse 
reports.  Every platform has the occassional bad customer that needs to 
be kicked off so most RBLs (good ones anyway) will allow for a small 
amount of UCE before hitting the threshold to be listed/blocked.

-- 
David Jones

Re: running a private SA-Mirror

[David Jones]

> W dniu 2019-05-01 o 10:05, A. Schulze pisze:
>> Hello,
>>
>> we've a number of SA instances that need rule updates. For now we configured 
>> them to use a proxy. Works...
>> But there are also instances that can't us a proxy at all.
>>
>> My idea was to setup a private SA-Mirror (apache+rsync) but, I've to manage
>> DNS-Data for mirrors.spamassassin-mirror.example and 
>> 2.3.4.spamassassin-mirror.example.
>> :-/
>>
>> Are there other methods to distribute current ruleset to SA-instances using 
>> sa-update?

There are many ways to accomplish this.  The best one will depend on 
your network layout and what tools you have available.

I suspect you already manage your SA local/custom 
rules/plugins/add-ons/etc from a central location and sync them out from 
there.  This would work very similar to that.

Pick one central server that will be your sa-update "master" then rsync 
the /var/lib/spamassassin//updates_spamassassin_org from it 
using native rsyncd or rsync over SSH with empty passphrase keys.

If you are using spamd, amavis-new, etc. then setup monit to detect a 
checksum/timestamp change on the updates_spamassassin_org directory then 
reload/restart the daemon that is the "glue" to SpamAssassin.

https://mmonit.com/monit/

-- 
David Jones

Re: SA shortcircuit

[David Jones]

On 4/23/19 12:57 AM, Brent Clark wrote:
> Good day David
> 
> Looking at what you got going, Im glad I asked this request.
> Thanks very much for sharing.
> 
> Kind Regards
> Brent Clark
> 

This was just an example based on my mail flow and very customized/tuned 
SA rules for my mail filters.  I am sure that others on this list may 
not agree with my settings below based on different philosophies / 
strategies of spam filtering.

I try to find patterns to group mail into major categories then handle 
each category differently.  For example, system-generated emails using a 
subdomain that aren't real human / user accounts with passwords that can 
be compromised are a category.  By the time this category makes it past 
the MTA checks (postscreen, RBLs, DNS checks, TLD checks, HELO checks, 
etc.) then the majority of them can be safely added to whitelist_auth 
entries and SA doesn't need to evaluate them for content filtering.

Content filtering is the hardest to do as the spammers are constantly 
changing the strategies / campaigns to get around content filtering. 
The best way to filter for content is with very good Bayes training but 
this usually doesn't help with zero day / hour spam from compromised 
accounts.

Another major category of email is certain sources of emails from large 
MSPs that generally do not get listed on RBLs because there are good 
email senders mixed in with the spammers / compromised accounts.

\.(secureserver\.net|web-hosting\.com|websitewelcome\.com|inmotionhosting\.com|unifiedlayer\.com|ezhostingserver\.com|siteprotect\.com|internetbilisim\.net|privateemail\.com|registrar-servers\.com|registeredsite\.com|reflexion\.net|sparkpostmail\.com|siteground\.us|dreamhost\.com|yourhostingaccount\.com|maropost\.com|myregisteredsite\.com)

That is my current regex for "notrust" known sources of spam with legit 
emails mixed in.  Here I maintain a private DWL of senders to trust 
(subtract a few points) while a set of meta rules amplify anything 
suspicious sent from those untrusted above.  In then end, I am able to 
query a MySQL database to find historical scores to add to the DWL using 
a script so this is mostly automated.

Office 365 is another category of email that is tough to filter properly 
because it's so large of an MSP and regulary / frequently has 
compromised accounts that send spam for 30 minutes or so before 
Microsoft seems to shut them down.  Here I wanted to setup something 
like greylisting to handle new senders with a delay.  I am using 
traditional greylisting but it has to be disabled for 
outbound.protection.microsoft.com and google.com email servers for 
technical reasons.  Recently I started another private DWL to handle 
o365 senders similarly to the "notrust" list above and so far it seems 
to be working out.

Hope this helps,
Dave


> On 2019/04/18 15:52, David Jones wrote:
>> On 4/18/19 1:55 AM, Brent Clark wrote:
>>> Good day Guys
>>>
>>> Would anyone be willing to share their shortcircuiting list.
>>>
>>> Currently I am just shortcircuiting CLAMAV, Im looking to improve SA.
>>>
>>> Many thanks.
>>>
>>> Regards
>>> Brent
>>
>> shortcircuit ALL_TRUSTED off
>> shortcircuit USER_IN_WHITELIST on
>> shortcircuit USER_IN_DEF_WHITELIST on
>> shortcircuit USER_IN_BLACKLIST on
>> shortcircuit USER_IN_DKIM_WHITELIST on
>> shortcircuit USER_IN_SPF_WHITELIST on
>> shortcircuit USER_IN_DEF_DKIM_WL off
>> shortcircuit USER_IN_DEF_SPF_WL off
>> shortcircuit RCVD_IN_RP_CERTIFIED off
>> shortcircuit RCVD_IN_RP_SAFE off
>>
>> You will need to set the priority lower than the default to hit before
>> some of the entries above.  Run some messages manually with
>> "spamassassin -D < email.msg" to see the priority if your shortcircuit
>> rule isn't getting hit because a lower priority shortcircuit hit first.
>>
>> I also have some outbound rules that shortcircuit unique emails like
>> those from scanner/copiers that often have missing headers like no
>> Message-ID, bad HELO, etc.
>>
>> Here's an example of a useful one that we all have problems with if we
>> are filtering outbound email:
>>
>> meta    ENA_COPIER  ALL_TRUSTED && (__SUBJ_COPIER ||
>> __MAILER_COPIER || __MSGID_COPIER || __MIME_COPIER || __FROM_COPIER ||
>> __RCVD_COPIER)
>> priority    ENA_COPIER    -500
>> describe    ENA_COPIER  Sent from a copier on network.
>> score   ENA_COPIER  -0.001
>> priority    ENA_COPIER  -500
>> shortcircuit    ENA_COPIER  ham
>> tflags      ENA_COPIER  noautolearn nice
>>
>> I am not publishing the details of those header rules in the meta above
>> on purpose so this rule could be exploited by a compromised account from
>> our network through our mail relays.  These should be fairly obvious
>> based on their names as to what they do.
>>
>> Hope this helps,
>>


-- 
David Jones

Re: SA shortcircuit

[David Jones]

On 4/18/19 1:55 AM, Brent Clark wrote:
> Good day Guys
> 
> Would anyone be willing to share their shortcircuiting list.
> 
> Currently I am just shortcircuiting CLAMAV, Im looking to improve SA.
> 
> Many thanks.
> 
> Regards
> Brent

shortcircuit ALL_TRUSTED off
shortcircuit USER_IN_WHITELIST on
shortcircuit USER_IN_DEF_WHITELIST on
shortcircuit USER_IN_BLACKLIST on
shortcircuit USER_IN_DKIM_WHITELIST on
shortcircuit USER_IN_SPF_WHITELIST on
shortcircuit USER_IN_DEF_DKIM_WL off
shortcircuit USER_IN_DEF_SPF_WL off
shortcircuit RCVD_IN_RP_CERTIFIED off
shortcircuit RCVD_IN_RP_SAFE off

You will need to set the priority lower than the default to hit before 
some of the entries above.  Run some messages manually with 
"spamassassin -D < email.msg" to see the priority if your shortcircuit 
rule isn't getting hit because a lower priority shortcircuit hit first.

I also have some outbound rules that shortcircuit unique emails like 
those from scanner/copiers that often have missing headers like no 
Message-ID, bad HELO, etc.

Here's an example of a useful one that we all have problems with if we 
are filtering outbound email:

metaENA_COPIER  ALL_TRUSTED && (__SUBJ_COPIER || 
__MAILER_COPIER || __MSGID_COPIER || __MIME_COPIER || __FROM_COPIER || 
__RCVD_COPIER)
priorityENA_COPIER  -500
describeENA_COPIER  Sent from a copier on network.
score   ENA_COPIER  -0.001
priorityENA_COPIER  -500
shortcircuitENA_COPIER  ham
tflags  ENA_COPIER  noautolearn nice

I am not publishing the details of those header rules in the meta above 
on purpose so this rule could be exploited by a compromised account from 
our network through our mail relays.  These should be fairly obvious 
based on their names as to what they do.

Hope this helps,

-- 
David Jones

Re: Spammer in white list aka USER_IN_DEF_SPF_WL

[David Jones]

On 4/17/19 4:16 PM, jandev wrote:
> Hi all
> 
> Yesterday our mail server received unwanted email from simpliv.com. It was
> valid DKIM signed for mail.simpliv.com
> Despite the sender ip was listed at Sorbs the email even passed the bayesian
> filter:
>   
> 
> Surprisingly the ip/domain is part of a SA shipped white list: Rule
> USER_IN_DEF_SPF_WL gave it -7.5!
> 
> simpliv.com sent the spam to an email address which was used solely for
> registering an account with slack.com. It seems that simpliv.com
> bought/stole/harvested email addresses in shady ways and uses the email
> database as spam to advertise its courses.
> 
> /var/lib/spamassassin/3.004002/updates_spamassassin_org/60_whitelist_auth.cf
> where the simpliv.com is added says: "These senders should be considered
> trusted following proper opt-in and opt-out practices,..."
> 
> There was no proper opt-in, even Sorbs list them now, probably because they
> hit a honey pot, hence I request simpliv.com to be removed from this white
> list.
> Otherwise having spammers in this SA shipped white list makes the list
> useless.
> 

Please post a lightly redacted version in pastebin.com so I can see what 
went wrong.  That seems odd to hit USER_IN_DEF_SPF_WL when it was DKIM 
signed for mail.simpliv.com.  The envelope-from domain would have been 
mail.simpliv.com and I can't find that in my database going back 6+ months.

The def_whitelist_auth entries are only supposed to hit when SPF_PASS or 
DKIM_VALID_AU are hit.  I need to see the original headers to learn what 
happened and possibly adjust the logic in the determination of 
trustworthy senders.

I have no problem with removing this entry if this sender is no longer 
trustworthy. They were at the time it was added but things do change 
over time.  This would be the second entry in a couple of years to be 
removed out of the hundreds of entries.

P.S. blacklist_from entries should override any whitelist_* entry, if I 
remember correctly.

-- 
David Jones

Office 365 Org tag

[David Jones]

I would like to use the AskDNS plugin to query a private DBL that I can 
populate/manage.  The idea is to subtract a few points for inbound O365 domains 
that have been seen before in an effort to help block compromised O365 accounts 
from domains that have never been seen before.

Ideally a new tag would be created when the last external relay is an 
outbound.protection.microsoft.com host and the X-Originating-Org header value 
(which should match the EnvelopeFrom domain) is used to make a new tag like 
_O365ORG_ for a simple rule like this:

ifplugin Mail::SpamAssassin::Plugin::AskDNS
askdnsO365_ORG_SEEN_BEFORE _O365ORG_.o365seen.example.com A 
/^127\.\d+\.\d+\.2$/
scoreO365_ORG_SEEN_BEFORE-2.0
endif

BTW, how can I find a list of all existing tags available for use?  I tried a 
number of greps and Google searches with no luck.

Thanks,
Dave

Re: Hive Mind: postfix prescreen and SA ruleqa

[David Jones]

On 4/14/19 3:03 AM, Jari Fredriksson wrote:
> We have had some discussions of this in the past. But now I became 
> worried that all SA users do not have access to their border smtp and 
> are NOT configuring postfix with this: https://pastebin.com/LGkdi7NM
> 
> Now, I am part of RuleQA. Should I accept everything and pass it so 
> SpamAssassin and to my corpus or not? Reindl Harald may have his say as 
> a corporate maintainer or something but the SpamAssassin user base is more.
> 
> How can I best support SpamAssassin besides having a mass check 
> automation and mirrors for the sa-update?

I have a secondary iredmail server hosting a few decommissioned domains 
receiving both ham and spam, but mostly spam.  This iredmail server has 
been stripped down to a very basic/stock Postfix and amavis-new configs 
to allow SA to score everything.  I sync over my /etc/mail/spamassassin 
custom config files and point to the same Bayes DB in redis so it scores 
similarly.  Dovecot Sieve rules put mail into the ham and spam folders 
for my masscheck processing based on very low scores for ham and very 
high scores for spam.  Messages that score in the middle are checked and 
manually moved into the ham or spam folder.  I also have a Spamcop 
folder that a script runs every 5 minutes to submit to Spamcop.  If I am 
able to catch a compromised account quickly, then I release that email 
to spamcop+postmas...@sa.ena.net which automatically goes into the 
Spamcop folder.

Note the sa.ena.net is an internal-only domain that is used to route 
mail to my iredmail server for the SA masscheck corpus.

Once you get this type of platform setup, it can be used for other spam 
fighting techniques on the primary mail filters like:

- train your shared redis Bayes DB with the ham and spam folder
- fail2ban - run a custom script on the secondary server to block IPs on 
the primary filters
- swatch - run custom scripts when certain rules are hit:
- add entries to your own private RBL to catch zero hour spam
- auto release certain messages from quarantine as an attachment
    - etc...

-- 
David Jones

Re: more spam is getting through :-(

[David Jones]

On 3/17/19 4:03 PM, James wrote:
> I've been getting a lot of spam so I'm thinking of lowering the 
> "required" number.
> 
> About 50 % spam gets a 4.4 so my required=4.5 is a tiny bit high.
> 
> I run sa-learn --ham on my inboxes.
> Is there a way to whitelist all the email addresses in my inboxes?
> 

Have you done any tuning of SA or just the stock installation?  There 
are many ways to make SA more accurate that can't be included by default 
for various reasons.  One reason is that everyone's mail flow is 
different.  Another reason is there are many different ways to use SA 
and you didn't say how you were using SA so we can't give helpful advice 
on tuning SA.

-- 
David Jones

Re: Whitelist_from??

[David Jones]

On 3/14/19 5:50 PM, @lbutlr wrote:
> I've been having a lot of problems with emails from comixology getting tagged 
> as spam and then the message attachment is often, but not always, corrupt.
> 
> Content analysis details:   (6.8 points, 5.0 required)
> 
> pts rule name  description
>  -- --
> -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
>  no trust
> [54.240.13.78 listed in list.dnswl.org]
> 0.2 BAYES_999  BODY: Bayes spam probability is 99.9 to 100%
> [score: 1.]
> 3.5 BAYES_99   BODY: Bayes spam probability is 99 to 100%
> [score: 1.]
> 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level
> mail domains are different
> 0.8 MPART_ALT_DIFF BODY: HTML and text parts are different
> 0.0 HTML_MESSAGE   BODY: HTML included in message
> 0.4 MIME_HTML_MOSTLY   BODY: Multipart message mostly text/html MIME
> 0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not necessarily
> valid
> 0.7 MIME_HEADER_CTYPE_ONLY 'Content-Type' found without required
> MIME headers
> 0.1 DKIM_INVALID   DKIM or DK signature exists, but is not valid
> 1.0 BODY_URI_ONLY  Message body is only a URI in one line of text or
> for an image
> 0.0 T_REMOTE_IMAGE Message contains an external image
> 
> The attached message when I open it starts:
> 
> =23outlook A =7B  PADDING-BOTTOM: 0px; PADDING-LEFT: 0px; PADDING-RIGHT: 
> 0px=
> ; PADDING-TOP: 0px =7D
> BODY =7BPADDING-BOTTOM: 0px; MARGIN: 0px; PADDING-LEFT: 0px; WIDTH: 100% =
> =21important; PADDING-RIGHT: 0px; PADDING-TOP: 0px; -webkit-text-size-adjus=
> t: 100%; -ms-text-size-adjust: 100%
> =7D
> =7D =20
> 
> 
> I added whitelist_auth comixology.com to local.cf and still had issues, so I 
> also added whitelist_from comixology.com, but messages are still tagged as 
> spam.
> 
> From: Comics by comiXology 
> 
> But the message are actually coming from amazon.com. I have these references 
> to amazon in local.cf
> 
> adsp_override amazon.com custom_high
> adsp_override amazon.com
> whitelist_auth *@amazon.com
> 
> (not sure about the first two lines, don't recall those settings)
> 
> 
> 

I would recommend using this if they hit SPF_PASS or DKIM_VALID_AU

whitelist_auth *@*.comixology.com

If they don't have good SPF or DKIM like this one, then use:

whitelist_from_rcvd *@*.comixology.com amazonses.com

The "amazonses.com" would be the part of the sending mail server's name 
when it has good FCrDNS.  If that mail server doesn't have good FCrDNS, 
then use:

whitelist_from_rcvd *@*.comixology.com [ip.ad.dr.ess]


whitelist_from should be the last option and I only use it on a full 
email address that is very unique so spammers won't be able to match 
that by accident from any source server or IP address.

-- 
David Jones

Re: Spam rule for HTTP/HTTPS request to sender's root domain

[David Jones]

On 2/28/19 10:50 AM, Ralph Seichter wrote:
> * Mike Marynowski:
> 
>> And the cat and mouse game continues :)
> 
> It sure does, and that's what sticks in my craw here: For a pro spammer,
> it is easy to set up websites in an automated fashion. If I was such a
> naughty person, I'd just add one tiny service that answers "all is well"
> for every incoming HTTP request.
> 
> Why even use a test for something that is so easily compromised?
> 
> -Ralph
> 

I would like to see an Open Mail Reputation System setup by a working 
group of big companies so it would have some weight behind it.  Setup 
some sort of scale like 0 to 100 for reputation that starts established 
domains older than X days at 50 (in the middle) and then have a clearing 
house for spam reports where it takes several different reports from 
different sources to lower a domain's score.  I am sure some smart 
Google engineers or SpamCop.net could do this their spare time in a way 
that can't be abused or poisoned.

Newly registered domains that are less than X days old would start at 
zero or 25 and have to earn their increase in score over time.  Maybe 
every week without a report of spam the score goes up by some increment.

Domains would have to implement good SPF, DKIM, and DMARC to participate 
in this reputation system.  A postmaster address (maybe the DMARC 
reporting email address) would be required with a mail loop verification.

Bounce messages would have a clear/plain message with a link explaining 
why the message was bounced (because of a sender problem, not the 
recipient mail server problem).  Default to opt-in sending copies of the 
bounce message to the postmaster address and require mail admins to 
opt-out if they don't want it.  (A major problem in email support today 
is not having good contacts of admins on the other end.  End users don't 
know what to do with bounce messages and mail admins can't easily get 
together to work on delivery problems.)

-- 
David Jones

Re: using existing score value in new rule's score

[David Jones]

On 2/22/19 4:37 PM, David B Funk wrote:
> Is there a rule "score" syntax that allows you to use the score assigned 
> to an existing rule to calculate the value assigned to another rule?
> 
> Specifically what I'm trying to do is to negate the "damage" a 
> particular rule does for messages that meet particular local criteria.
> 
> For example: "HTML_IMAGE_ONLY_28" is a rule that will assign a modest 
> number of points to a message that contains a small amount of HTML and 
> an image.
> 
> What I want to do is to create a local rule:
> 
> meta L_HTML_IMAGE_ONLY_28_FIX  ( HTML_IMAGE_ONLY_28 && L_O365_USER )
> describe L_HTML_IMAGE_ONLY_28_FIX   Fix damage from 
> HTML_IMAGE_ONLY_28 for local O-365 users
> score L_HTML_IMAGE_ONLY_28_FIX ( -1.0 * HTML_IMAGE_ONLY_28 )
> 
> Where if HTML_IMAGE_ONLY_28 fires and another rule which detects that 
> the message was generated by a local Office-365 user, negate the score 
> from the HTML_IMAGE_ONLY_28 rule.
> 
> My problem is that our campus has switched the bulk of our user 
> population to Office-365 and many outlook users like to "decorate" their 
> messages with images (wall-paper, deparmental logos, etc).
> When one of these people sends a short message (1~5 lines of text) in 
> their outlook, it's not unusual for several of SA's rules to fire (EG 
> DC_GIF_UNO_LARGO, HTML_IMAGE_ONLY_28, SARE_GIF_STOX, etc) which pushes 
> the messages into spam score range.
> 
> I'd like to automate the un-doing of this damage w/o having to 
> continually chase after changes in the scoring.
> Thus the desire for syntax to calculate the score value. It doesn't have 
> to be evaluated dynamically, just calculate the score at reload time.
> 
> Thanks.
> 

I use the X-OriginatorOrg header in a meta rule with other headers to 
subtract a few points (trust) certain Office 365 senders.  Otherwise, I 
treat Office 365 like other "FREEMAIL" sources that are mostly untrusted 
(add a point or two).  You don't have to do the later but the former 
might be helpful.

-- 
David Jones

Re: Spam : You have 5 Incoming messages

[David Jones]

On 1/30/19 2:03 AM, Brent Clark wrote:
> Good day Guys
> 
> We are seeing quite a few of the following spam, been delivered to our 
> users.
> 
> https://pastebin.com/raw/43VqDPTy
> 
> Notice the:
> 
> You have 5 Incoming messages t=
> hat could not be delivered to eunice@REMOVED
> Retrieve Messages and reconfigure SMTP server to avoid losing important 
> fil=
> es and messages.
> 
> Then at the bottom, see the URL try and catch the recipient.
> 
> This email it to serve as a FYI to the community and maybe a global rule 
> can pushed out, and secondly to ask if someone can please peer review my 
> below ruleset. It works, I am just wondering if it can be done better.
> 
> header    HTEST Subject =~ 
> /[0-9]?\s?(Underliverable|Incoming)?\sMessages\s(for|failed)?\s?(for)?/i
> score HTEST 0.01
> describe  HTEST Testing new rule
> 
> Many thanks
> Brent Clark

I think you redacted/changed too much for us to be able to help without 
guessing.

1. Did the original email subject have "Spam: " at the front or did your 
system add that?

2. Please leave the original Received: header IPs since that doesn't 
give away any sensitive information.  We need those to check for RBLs.

3.  Please leave any sender information like the envelope-from address 
and the From: header address.

4.  Only redact your recipient's address and name.  Replace the 
recipient's domain with something like example.com or redacted.com so it 
looks like a real domain format.  Otherwise, it may hit SA rules that 
wouldn't trigger on the original email like TO_MALFORMED.

Here's what my SA platform scored it as but it's not going to be 
accurate enough with that first redacted spample.  Please send us 
another one minimally redacted.

X-Spam-Status: Yes, score=5.6 required=5.0 tests=BAYES_50,HTML_MESSAGE,
TO_IN_SUBJ,TO_MALFORMED,TVD_RCVD_SINGLE,UNPARSEABLE_RELAY 
shortcircuit=no
autolearn=no autolearn_force=no version=3.4.1
X-Spam-Report:
*  2.2 TVD_RCVD_SINGLE Message was received from localhost
*  0.0 HTML_MESSAGE BODY: HTML included in message
*  1.2 BAYES_50 BODY: Bayes spam probability is 40 to 60%
*  [score: 0.4993]
*  2.1 TO_MALFORMED To: has a malformed address
*  0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay 
lines
*  0.1 TO_IN_SUBJ To address is in Subject

-- 
David Jones

Re: SpamSender with 2 @-signs in the address

[David Jones]

On 12/4/18 9:09 AM, Benny Pedersen wrote:
> Bill Cole skrev den 2018-12-04 03:58:
> 
>> DKIM and DMARC *ONLY* operate on headers, *NEVER* on the envelope.
> 
> SPF is part of DMARC so not correct

I think he meant that DKIM related to DMARC means the DKIM signature has 
to align/match the From: header domain to pass which is DKIM_VALID_AU in SA.

In the case of SPF, DMARC will pass if the envelope-from domain check 
hits SPF_PASS in SA.

DMARC_PASS = SPF_PASS || DKIM_VALID_AU
DMARC_FAIL = !SPF_PASS && !DKIM_VALID_AU
DMARC_REJECT = DMARC_FAIL && DMARC record contains p=reject

-- 
David Jones

Re: spoofing mail

[David Jones]

On 12/1/18 8:31 AM, Matus UHLAR - fantomas wrote:
>> El vie., 30 nov. 2018 a las 3:06, Matus UHLAR - fantomas
>> () escribió:
>>> And, yes, there could be rule that catches message-id added by internal
>>> server. Note that:
>>> - Message-ID is not required (has SHOULD in RFC)
>>> - many mailservers add message-id if it doesn't exist.
> 
>>> >> https://pastebin.com/ktMUDLps
> 
>>> not available anymore :-(
> 
> On 30.11.18 10:55, Rick Gutierrez wrote:
>> Hi , here it is https://pastebin.com/3TtsjXSX
>>
>> last trace ,  after my gateway analyzes it
>>
>> https://pastebin.com/76rNVnnp
> 
> - is "mydomain.com" your real domain?
> 
> - funny that Message-Id is signed in DKIM and DKIM is valid.
> 
> hmmm more to think about later.
> 

DKIM_VALID only confirms it was signed correctly by any domain.  Anyone 
can generate keys and DNS records to sign an email with a domain for 
which they control/manage the DNS.  I can sign all emails leaving my 
edge mail servers with an ena.net or ena.com key.  That only means you 
can be sure it is authentic (unmodified) and came from my servers.  It 
doesn't mean I am allowed to send for that domain.

DKIM_VALID_AU confirms the DKIM signature aligned with the author's 
From: header domain and is authentic (unmodified).  This means something 
but is still not an indicator of ham or spam -- just that it came from 
that domain unmodified.  If you trust the domain like paypal.com to not 
send UCE or spam from compromised accounts, then you can whitelist_auth 
that domain.

-- 
David Jones

Re: spoofing mail

[David Jones]

On 11/29/18 9:44 AM, Paul Stead wrote:
> I can't find MSGID_BELONGS_RECIPIENT in the standard distribution - I think 
> this might be because my Plugin is installed.
> 
> Another to get into branch?
> 

I think this one is worthy of consideration to be included in the core 
SA ruleset.

https://github.com/fmbla

[root@server spamassassin]# pwd
/etc/mail/spamassassin
[root@server spamassassin]# cat 99_recipient_msgid.cf
ifplugin Mail::SpamAssassin::Plugin::RecipientMsgID

   meta __PDS_MAILING_SOFTWARE (__VIA_ML || __DOS_HAS_MAILING_LIST || 
__DOS_HAS_LIST_UNSUB || __HAS_LIST_ID || __DOS_HAS_LIST_ID || 
__HAS_X_MAILING_LIST)

   meta MSGID_BELONGS_RECIPIENT __MSGID_BELONGS_RECIPIENT && 
!__PDS_MAILING_SOFTWARE && !ENA_TRUSTED_LIST
   describe MSGID_BELONGS_RECIPIENT Message-ID domain belongs to recipient
   score MSGID_BELONGS_RECIPIENT 2.2

   meta MSGID_FAKE_FROM_2_EMAILS (__PLUGIN_FROMNAME_SPOOF && 
__MSGID_BELONGS_RECIPIENT)
   describe MSGID_FAKE_FROM_2_EMAILS MSGID belongs to recipient and 
faked froms
   score MSGID_FAKE_FROM_2_EMAILS 4.2

   full __FROM_NAME_LAST_THING 
/From:\W*([\w+.-]+\@[\w.-]+\.\w\w++).*\1(?:\s*|<\/\w+>|--[\w_\-\.\=]{2,}--)+$/s

   meta SPOOF_NAME_LAST_THING (__PLUGIN_FROMNAME_SPOOF && 
__FROM_NAME_LAST_THING)
   describe SPOOF_NAME_LAST_THING From 2 emails and fake from name as 
last thing
   score SPOOF_NAME_LAST_THING 2.2

endif

-- 
David Jones

Re: spoofing mail

[David Jones]

On 11/29/18 3:30 AM, Rupert Gallagher wrote:
> Message-ID and To have the same domain, but From does not. You should 
> have never received that mail.
> 

Here's what my mail filters say.  You can ignore the DKIM_INVALID 
because the body was intentionally modified (redacted) to post to pastbin.

X-Spam-Status: Yes, score=11.0 required=5.0 tests=BAYES_99,DKIM_INVALID,
DKIM_SIGNED,ENA_BAD_SPAM,ENA_RELAY_NOT_US,MSGID_BELONGS_RECIPIENT,
RCVD_IN_IVMBL,UNPARSEABLE_RELAY shortcircuit=no autolearn=no
autolearn_force=no version=3.4.1
X-Spam-Report:
*  5.2 BAYES_99 BODY: Bayes spam probability is 99 to 100%
*  [score: 0.9980]
*  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
*  valid
*  1.2 RCVD_IN_IVMBL No description available.
*  0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay 
lines
*  0.1 DKIM_INVALID DKIM or DK signature exists, but is not valid
*  2.2 ENA_RELAY_NOT_US Relayed from outside the US and not on 
whitelists
*  2.2 MSGID_BELONGS_RECIPIENT Message-ID domain belongs to recipient
*  0.0 ENA_BAD_SPAM Spam hitting really bad rules.

A well-trained Bayes helps a lot.

You could/should increase the score on MSGID_BELONGS_RECIPIENT in your 
/etc/mail/spamassassin local scores file.

Local overrides of scores and settings is typically done in 
/etc/mail/spamassassin/local.cf but feel free to make your own *.cf 
files in /etc/mail/spamassassin.  Amavis can create it's own files to 
customize settings in /etc/mail/spamassassin so compare a vanilla SA 
installation to what you have to find the best place to put your local 
settings.

-- 
David Jones

Re: spoofing mail

[David Jones]

On 11/27/18 11:22 AM, Rick Gutierrez wrote:
> El mar., 27 nov. 2018 a las 11:14, Alan Hodgson
> () escribió:
> 
>>
>> Wow, that's hard to read.
>>
>> It was close to being tagged because of the Pakistan relay. Just add a few 
>> points for Word docs and you should be good. Word docs from spammy countries 
>> should really get a lot of points.
> 
> Hi Alan , I think it's a valid point, except for one thing, what
> happens if you do not attach a document?
> 
> Something I want to ask you, where can I increase this score or in what rules?
> 
> 

Can you send a copy of the original email lightly redacted via pastebin 
so I can run it through my filters to give some pointers?

-- 
David Jones

Re: Custom DMARC_FAIL rule

[David Jones]

On 11/27/18 7:46 AM, RW wrote:
> On Mon, 26 Nov 2018 20:13:12 -0500
> Robert Fitzpatrick wrote:
> 
>> I have the following custom rules working pretty well in testing, but
>> ran into this message with two "Authentication-Results" headers:
>>
>>> Authentication-Results: mx3.webtent.org; dmarc=none (p=none
>>> dis=none) header.from=email.monoprice.com
>>> Authentication-Results: mx3.webtent.org;
>>> dkim=fail reason="signature verification failed" (2048-bit
>>> key; unprotected) header.d=email.monoprice.com
>>> header.i=@email.monoprice.com header.b=JvTxQQIc
>>
>> This triggers DMARC_FAIL in my custom rules below, but all I want to
>> pick up on is 'header.from' failures. What do I need to change the
>> regular expression to also pick up on header.from in the header?
>> Would I just add '.*header.form' after =fail?
>>
>>> # DMARC rules
>>> header __DMARC_FAIL Authentication-Results =~ /webtent.org;
>>> (dmarc|dkim)=fail /
> 
> 
> dkim=fail doesn't imply the email failed DMARC. Just look for
> dmarc=fail. Using header.from is just a roundabout way of eliminating
> the unneccessary dkim=fail matches.
> 
> 

Correct.  For DMARC to pass _either_ SPF_PASS and aligns with the 
envelope-from domain _OR_ DKIM_VALID_AU which is a pass and alignment 
with the From: header domain.  If both pass and align then that is even 
better.

Keep it simple.  (Adjust the "smtp.ena.net" for your own OpenDMARC 
AuthservID value.)


header  DMARC_PASS  Authentication-Results =~ /smtp\.ena\.net; 
dmarc=pass/
describeDMARC_PASS  DMARC check passed
score   DMARC_PASS  -0.01

header  DMARC_FAIL  Authentication-Results =~ /smtp\.ena\.net; 
dmarc=fail/
describeDMARC_FAIL  DMARC check failed
score   DMARC_FAIL  0.01

header  DMARC_NONE  Authentication-Results =~ /smtp\.ena\.net; 
dmarc=none/
describeDMARC_NONE  DMARC check neutral
score   DMARC_NONE  0.01

header  __DMARC_FAIL_REJECT Authentication-Results =~ 
/smtp\.ena\.net; 
dmarc=fail \(p=reject/
metaDMARC_FAIL_REJECT   __DMARC_FAIL_REJECT && !ENA_TRUSTED_LIST
describeDMARC_FAIL_REJECT   DMARC check failed and the sending 
domains 
says to reject this message
score   DMARC_FAIL_REJECT   8.2


Adjust the ENA_TRUSTED_LIST above to whatever you want to do to exclude 
certain senders or mailing lists from DMARC checks.

-- 
David Jones

Re: DKIMWL_WL_MED spams

[David Jones]

On 11/21/18 12:13 PM, Matus UHLAR - fantomas wrote:
> Hello,
> 
> I have recently noticed spams spreading via amasonses.com and outlook.com.
> hitting DKIMWL_WL_MED that pushed score below threshold.
> 
> especially amazonses.com mail seemed to be amazon cloud servers.
> 
> Has anyone noticed this too?
> 
> I have disabled DKIMWL_WL_MED for now.
> 

Amazon has either loosened up their security or they have some customers 
that weren't properly vetted.  I have noticed an uptick in SES spam 
lately too.  I report them to SpamCop which reports them to Amazon's 
email abuse.  Please report them to Amazon to help all of us out. Just 
blocking them locally doesn't really help anyone because the spammer 
will use another throwaway domain via SES to spam again soon.
The fact that Amazon will handle abuse reports properly means they 
should get some trust points subtracted.  Users should be able to 
unsubscribe and give feedback that "I never signed up for this email" 
for the ones that get through.
The type of spam that is coming from Amazon SES lately is mostly people 
trying to sell contact lists.  I take it as a challenge to enhance my 
regex that blocks these types of emails not just from SES.

-- 
David Jones

Re: Forgery with SPF/DKIM/DMARC

[David Jones]

On 11/17/18 9:52 AM, John Hardin wrote:
>> From: John D. Smith  
>> To: kdeu...@vianet.ca
>> Message-ID: <35706717752563516902.8f4660866aa84...@vianet.ca>
>>
>> Couple of things:
>> 1. Recent discussions on this mailing list showed me that the Message-ID
>> should never have the recipient's domain in it
> 
> That's not 100% true.
> 
> There is no requirement that the sender put a Message-ID on the message. 
> It is valid for your MTA to add a Message-ID onto a message that was 
> received without one. That is likely going to be done using your domain.
> 

Sure.  Real MTA/MUA's should be setting a Message-ID header else it will 
look very spammy.  Copiers, scanners, and other basic SMTP-enabled 
devices often don't put all of the "standard" headers in their messages 
so they have to be whitelisted safely.

My Postfix MTA doesn't add the Message-ID header and even if it did, it 
would be something like "ena.net" that is not going to match the dozens 
of domains that I filter.

This strategy seems to have helped stop this type of spam so far without 
over blocking.

> I'd suggest a filter on Message-ID domain would be more appropriate at 
> the MTA level than in SA - if a message is received from outside with a 
> Message-ID having a domain that you control, reject it at that point, 
> before the possibility of adding a local one because it's missing 
> becomes a source of ambiguity.
>
I am using MailScanner so that is a drawback not being able to reject 
during the SMTP conversation.  I do try to reject as much as I can at 
the MTA so MailScanner and SA only have to block the tough ones.  Most 
spam scores above 30 which is not going to be missed by anyone (not 
something they are expecting to receive).

> I do something similar with HELO. You might want to look into that too - 
> check your logs for the HELOs that spammers are using, there are 
> low-hanging fruit there (that I'm reluctant to discuss publicly).
> 

Interesting.  I may have to make some time over the holidays to research 
this a bit more in my logs.  My mail filtering is pretty spot on right 
now but if anything gets through I will check the HELO details.  Most of 
the things that get through now are zero-hour messages from compromised 
accounts so those HELO's are going to be good and everything else 
(FCrDNS, SPF, DKIM, DMARC) will be legit and pass.  I am thinking of 
increasing the time on greylisting to give DCC and RBLs time to catch up 
with compromise accounts.

>> 2. Seems like there should be easy rules to detect more than one pair of
>> angle brackets and more than on at sign to add points to non-standard
>> display names.
> 
> There probably are. A big question is: does that appear enough in the 
> masscheck corpora to be promoted as a useful rule?
> 

I think my ena-week[0-4] (past 5 weeks) masscheckers are still the 
majority of the overall masscheck corpora.  I need some help planting 
email addresses out there that will attract more spam of differing types 
or something.  I definitely need to get more non-English spam in there.

-- 
David Jones

Re: Forgery with SPF/DKIM/DMARC

[David Jones]

On 11/16/18 7:44 AM, Robert Fitzpatrick wrote:
> We're having an issue with spam coming from the same company even though 
> SPF and DKIM is setup with DMARC to reject. Take this forwarded email 
> for instances
> 
>>  Original message  From: User  Date: 
>> 11/15/18 10:42 AM (GMT-07:00) To: Other User  
>> Subject: OVERDUE INVOICE
>> Sorry for the delay…. This is an invoice reminder. The total for your 
>> item is $1,879.17.
>> THX,
>> -
>> User T 123.456.7890 | O 123.456.7891 EMail:u...@company.com
> 
> However, the raw headers show as this...
> 
>> Date: Thu, 15 Nov 2018 18:35:35 +0100
>> From: User 
>> 
>> To: other.u...@company.com
>> Message-ID: <860909106225419267.2007038e08376...@company.com>
>> Subject: OVERDUE INVOICE
> 
> Could someone suggest a rule to match the signature with the last From 
> email or envelope from? Or another suggestion how this could be resolved.
> 
> Thanks!
> 


From: John D. Smith  
To: kdeu...@vianet.ca
Message-ID: <35706717752563516902.8f4660866aa84...@vianet.ca>

Couple of things:
1. Recent discussions on this mailing list showed me that the Message-ID 
should never have the recipient's domain in it so I setup a local meta 
rule to match all of my customer domains that I filter for with an 
inbound rule (like ALL_TRUSTED) to add a bunch of points.
2. Seems like there should be easy rules to detect more than one pair of 
angle brackets and more than on at sign to add points to non-standard 
display names.
3. I add a point or two for invoice-related subjects just because I want 
to lower the bar for them being caught.  Legit invoice senders should 
have other good rules hit that will offset this.  I try to make legit 
invoice senders score just below the block threshold so anything 
suspicious like that From: or Message-ID: header will push it over the 
limit.
You can setup logwatch or grep your mail logs often from cron to alert 
you when your invoice-related rules are hit so you don't cause a problem 
blocking a real invoice in the first month or two as you are tuning your 
rules and scores.

-- 
David Jones

Re: Phishing email or no?

[David Jones]

On 10/12/18 4:12 PM, Pedro David Marco wrote:
> 
> 
>  >On Friday, October 12, 2018, 10:48:21 PM GMT+2, Rupert Gallagher 
>  wrote:
> 
>  >I love outlook.com ...
> 
> i have seen recently an Office365 Phishing campaign coming from 
> Office365 severs...  as good as it gets...
> 
> -
> PedroD

What we need is a milter or SA plugin that keeps track of new Office365 
senders like greylisting just for outbound.protection.outlook.com to 
handle compromised accounts and spammers that Microsoft can't seem to 
detect and lock fast enough.  Maybe they need to start using 
SpamAssassin and hire some of us to do their mail filtering.  :)

-- 
David Jones

Re: Phishing email or no?

[David Jones]

On 10/11/18 7:00 PM, Alex wrote:
> Hi,
> 
> On Thu, Oct 11, 2018 at 5:15 PM David Jones  wrote:
>>
>> On 10/11/18 3:30 PM, Alex wrote:
>>> Hi,
>>>
>>> I'm curious what people think of this:
>>>
>>> https://pastebin.com/1XjwaCY1
>>>
>>> It's unsolicited, so that makes it spam to me, but is it dangerous?
>>> yesinsights.com appears to be a legitimate company, but the sender,
>>> e...@hrteamerus.com, is a registered domain but has no DNS record.
>>>
>>> Is it just a lame attempt to confirm email addresses?
>>>
>>> Outlook just seems to be a non-stop source of spam. I'd report it to
>>> yesinsights, but it appears it's being used exactly as the service
>>> intended?
>>>
>>> Any idea on tips to block it, other than bayes?
>>>
>>
>> Is that the entire email in the pastebin link above?  I ran it through
>> my SA platform and it's missing a few headers.
>>
>>  DKIM_INVALID,DKIM_SIGNED,ENA_NO_TO_CC,MISSING_DATE,MISSING_FROM,
>>  MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT
> 
> Yes, it's the complete email - those missing headers are in the
> pastebin. It also passed DKIM. Send me a message if you want the
> original.
> 
>> Since it doesn't have a valid opt-out, I would report it to SpamCop,
>> report it to yesinsights.com's abuse if SpamCop doesn't already, and add
>> a blacklist_from *@hrteamerus.com entry.
> 
> Yes, we've seen an increase in these types of emails. We've reported
> it to spamcop, but there doesn't appear to be a way to communicate
> abuse to yesinsights.
> 

I checked yesinsights.com site and they don't have a way to contact them 
or report abuse.  They do have a free week trial so you could setup a 
trial to get in touch with someone and tell them they need to have an 
abuse contact setup with Spamcop or they will eventually be listed on 
RBLs if they have enough shady customers sending to recipients that 
haven't opted into these emails.

If I received complaints from my customers about spam from yesinsights, 
I would put a REJECT line in my Postfix config with a details 
explanation as to why they were being blocked to give them feedback in 
their logs in case they actually check them.

Another option you have if you see repeating characteristics is to 
create a local meta rule that combines URLs with yesinsights.com with 
the envelope-from domain of hrteamerus.com or other things you see over 
and over to add some points.

This email came via Office 365 which is a major problem for sorting out 
spam.  They are so large that you can't block them outright so I have 
created a set of meta rules that amplify some spammy scores for O365 and 
add a point or two for all O365 email then put known good O365 senders 
to an exception list.  It has worked pretty well for the past year. 
Takes a little work up front to start the list but I haven't had to do 
much lately.  I mainly had to exclude senders that send odd attachments 
or invoices that trigger suspicious phishing-type rules.

-- 
David Jones

Re: Phishing email or no?

[David Jones]

On 10/11/18 3:30 PM, Alex wrote:
> Hi,
> 
> I'm curious what people think of this:
> 
> https://pastebin.com/1XjwaCY1
> 
> It's unsolicited, so that makes it spam to me, but is it dangerous?
> yesinsights.com appears to be a legitimate company, but the sender,
> e...@hrteamerus.com, is a registered domain but has no DNS record.
> 
> Is it just a lame attempt to confirm email addresses?
> 
> Outlook just seems to be a non-stop source of spam. I'd report it to
> yesinsights, but it appears it's being used exactly as the service
> intended?
> 
> Any idea on tips to block it, other than bayes?
> 

Is that the entire email in the pastebin link above?  I ran it through 
my SA platform and it's missing a few headers.

DKIM_INVALID,DKIM_SIGNED,ENA_NO_TO_CC,MISSING_DATE,MISSING_FROM,
MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT

Since it doesn't have a valid opt-out, I would report it to SpamCop, 
report it to yesinsights.com's abuse if SpamCop doesn't already, and add 
a blacklist_from *@hrteamerus.com entry.

If you start seeing patterns of repeating emails, then a local content 
rule and Bayes training would be the best option.  Maybe get these into 
the nightly masscheck so others can work on some rules to go into the 
default ruleset.

-- 
David Jones

Re: RBL

[David Jones]

On 10/10/18 3:12 PM, Kevin Miller wrote:
> I may be wrong, as I haven't implemented it yet, but postscreen may give you 
> that same functionality at the MTA level.
> 
> ...Kevin
> --
> Kevin Miller
> Network/email Administrator, CBJ MIS Dept.
> 155 South Seward Street
> Juneau, Alaska 99801
> Phone: (907) 586-0242, Fax: (907) 586-4588 Registered Linux User No: 307357
> 
> -Original Message-
> From: Grant Taylor [mailto:gtay...@tnetconsulting.net]
> Sent: Wednesday, October 10, 2018 12:09 PM
> To: users@spamassassin.apache.org
> Subject: Re: RBL
> 
> On 10/10/2018 01:56 PM, Tom Hendrikx wrote:
>> However, in general it's better to use DNSBLs at the MTA level, which
>> uses a lot less resources than implementing them in Spamassassin. So
>> try and set them up in postfix first.
> 
> I conceptually agree.
> 
> However, I prefer to do some RBL testing in SpamAssassin because I can
> easily check multiple RBLs and tag messages as spam, or reject, based on
> spam score.  Conversely, most MTA's implement RBLs as a binary pass /
> fail situation.  Thus SpamAssassin gives more flexibility and provides a
> configurable gray area that MTA's can't do themselves.
> 
> 
> 

Yes.  Search the SA archive lists for postscreen.  There was a thread a 
couple of years ago where we listed a good weighted list to allow 
combining the power of multiple RBLs for better results.

I also mentioned implementing postwhite at the same time to bypass 
postscreen for some senders so you can increase the sensitivity of your 
postscreen_dnsbl_sites safely.

https://github.com/stevejenkins/postwhite

-- 
David Jones

Re: wordpress whitelist entry

[David Jones]

On 10/9/18 2:21 PM, RW wrote:
> 
> I've recently noticed that newsletters from a small wordpress site are
> hitting USER_IN_DEF_SPF_WL.
> 
> The headers are of the form:
> 
>Return-Path: 
>...
>To: m...@example.com
>From: Some Amateur Website 
> 
> and the use of the bounce handling subdomain  b.wordpress.com is
> causing a match on:
> 
>def_whitelist_auth *@*.wordpress.com
> 
> Theses emails are legitimate, and I've not had much wordpress spam, but
> they are essentially freemail bulk mail.
> 

I am not understanding the question or issue.  If they 1) don't send 
spam, 2) only send opt-in email with a valid opt-out option and 3) they 
quickly handle any abuse reports then they should be considered a 
trusted sender.  Since these are system-generated emails and not real 
human mailboxes that can be compromised to send spam, then that 
def_whitelist_auth entry is safe.

Once we find evidence that any def_whitelist_auth sender fails to follow 
all 3 rules above then post an example here via pastebin.com and we will 
take appropriate action.

-- 
David Jones

Re: Bitcoin update

[David Jones]

On 10/5/18 4:38 PM, Antony Stone wrote:
> On Friday 05 October 2018 at 23:26:12, Rupert Gallagher wrote:
> 
>>> https://pastebin.com/TRD7FzRQ
>>>
>>> I have a sample here
>>
>> There are at least three reasons to reject that e-mail upfront, with no
>> need to parse its body.
> 
> Hints might be appreciated for the uninitiated.
> 
> 
> Antony.
> 
> 
> PS: Please do NOT set Reply-To to your own address on list postings.
> 

Are you doing any RBLs at the MTA?  This thing looks really bad and 
would never have made it past my Postfix postscreen_dnsbl_sites list.

 http://multirbl.valli.org/lookup/114.46.223.46.html

If it had made it to SpamAssassin, here's what my rules would have scored:

Content analysis details:   (29.8 points, 5.0 required)

  pts rule name  description
 -- 
--
  5.2 BAYES_99   BODY: Bayes spam probability is 99 to 100%
 [score: 1.]
  3.2 BAYES_999  BODY: Bayes spam probability is 99.9 to 100%
 [score: 1.]
  0.5 FROM_DOMAIN_NOVOWELFrom: domain has series of non-vowel letters
  1.5 CK_HELO_DYNAMIC_SPLIT_IP Relay HELO'd using suspicious hostname
 (Split IP)
  0.2 CK_HELO_GENERICRelay used name indicative of a Dynamic Pool or
 Generic rPTR
  1.9 DATE_IN_FUTURE_06_12   Date: is 6 to 12 hours after Received: date
  3.2 DCC_CHECK  Detected as bulk mail by DCC (dcc-servers.net)
  0.1 FROM_EQUALS_TO From: and To: have the same username
  0.0 KHOP_DYNAMIC   Relay looks like a dynamic address
  3.6 HELO_DYNAMIC_IPADDR2   Relay HELO'd using suspicious hostname (IP addr
 2)
  1.0 RDNS_DYNAMIC   Delivered to internal network by host with
 dynamic-looking rDNS
  2.2 ENA_RELAY_NOT_US   Relayed from outside the US and not on 
whitelists
  0.1 HDR_ORDER_FTSDMCXX_DIRECT Header order similar to spam
 (FTSDMCXX/boundary variant) + direct-to-MX
  2.0 MIMEOLE_DIRECT_TO_MX   MIMEOLE + direct-to-MX
  2.5 DOS_OE_TO_MX   Delivered direct to MX with OE headers
  2.5 NO_FM_NAME_IP_HOSTNNo From name + hostname using IP address
  0.0 ENA_BAD_SPAM   Spam hitting really bad rules.


-- 
David Jones

Re: Re: No rule updates since 1/1/17

[David Jones]

Tom,

Let me know if you are still interested in setting up a masschecker.  
That goes for anyone on this list as well.  I have worked out the 
sorting issue pretty well now and my ena-weekX masscheckers are now the 
largest contributions to the RuleQA corpus keeping the nightly rule 
scoring updating regularly the past year.


http://ruleqa.spamassassin.org/ (see the ena-weekX in the green box)

New/more masscheckers are always welcome and will help you learn the 
best way to tune your SA platform to get every last drop of accuracy 
from your local meta rules.  We could really use masscheckers with 
primary languages not English to add/improve core SA rules.


Here's my setup:

- I have an iRedmail server that I split copies of most of my email to 
an internal-only email domain "sa.ena.net."


- The iRedmail server has Sieve rules (easily managed by RoundCube) 
based on certain rule hits and scores from my main Internet edge 
MailScanner filtering that move them into Ham and Spam folders as 
unread.  Mail scoring in the middle -- not high enough for obvious Spam 
or low enough for obvious Ham are left in the main Inbox.


- I spend a few minutes each day visually scanning the Subjects of the 
unread email then mark them as Read.


- If I find a zero-hour email in the main Inbox, then I move it to a 
SpamCop folder.  A script that runs every 5 minutes to check the SpamCop 
folder, strips of some extra Received headers from my internal hops, 
then submits it as an attachment to my SpamCop account.


- A script moves the Maildir email to 4 other masschecker VMs to split 
out the load so they will be able to submit their results quickly.  
Ena-week0 is the last week of ham/spam that is still on the iRedMail 
server.  Ena-week1-4 are running on the other 4 masschecker VMs to give 
a total of 5 weeks of recent corpus.  I currently have 100,939 Ham and 
292,001 Spam in ena-week0-4.


- I run a local Bayesian train on the ena-week0 Ham and Spam folder to 
my Redis-based Bayes storage shared across my 8 MailScanner nodes and my 
iRedMail/amavis server.  This method has shown to keep my Bayes scores 
very accurate.


Hope someone finds this information helpful.

Dave


On 01/20/2017 01:02 PM, Tom Hendrikx wrote:

On 20-01-17 19:46, David Jones wrote:

From: Kevin Golding 
Sent: Friday, January 20, 2017 11:59 AM
To: users@spamassassin.apache.org
Subject: Re: No rule updates since 1/1/17
 

On Fri, 20 Jan 2017 17:26:01 -, Bill Keenan
 wrote:

What is the fix needed so /usr/bin/sa-update starts getting updates? I
too have not received an update from updates.spamassassin.org
<http://updates.spamassassin.org/> since 1-Jan-17.

Besides updates.spamassassin.org <http://updates.spamassassin.org/>,
what other rule sets are commonly used? Hundreds of spam messages are
getting through with only updates.spamassassin.org
<http://updates.spamassassin.org/> rules.

This seems like a good time to mention
https://wiki.apache.org/spamassassin/NightlyMassCheck
If more people can contribute, even just a small corpora of mail, then
updates will be published more frequently. At the moment a very small
number of people provide data, meaning there is very little margin for
error.

I would like to help with the nightly masscheck but I don't have the
resources to manually check ham and spam.  This also gets into the
grey area of how people define spam.  I also have a very good MTA
setup with RBLs and DNS checks that block most of the spam before
it reaches SA in MailScanner.  My SA only has to block a very small
percentage of my definition of spam so I am not sure how helpful
my mail filtering platform can be even though it's very accurate.

Dave


I think I can say the same about my platform, but since this issue keeps
popping up I just applied for an account just to find out if my
contribution could help. I can't speculate so I'm just gonna try if it
helps :)

Kind regards,
    Tom



--
David Jones

Re: From name containing a spoofed email address

[David Jones]

On 08/24/2018 07:02 PM, Kevin A. McGrail wrote:

On 1/18/2018 6:52 AM, Pedro David Marco wrote:

David,

This rule can do the full job... i have tested it with good results..  
 (Can be tested here: https://regex101.com/r/Vpmhjz/3 )


It checks if the level domain next to the TLD in the From:name matches 
the domain next to the TLD in From:email


header       FROM_DOMAINS_MISMATCHFrom 
!~/(?:[^<].+?)\@(?:.+?\.)*?(.+?\.)(?:.+?).*?<.+?(\@\1|\@.*?\.\1)/

describe    FROM_DOMAINS_MISMATCHDomain name mismatch in From header

Did this ever get considered for a sandbox.

Alan Hodgson also had a good posted on one but not tested.
Regards,
KAM


I am not sure this is going to be worth as sandbox rule.  There are 
going to be a high number of system-generated and mass-marketing emails 
that aren't going to match the From: header.


From my experience, this is a local rule that detects high-value 
display names in phishing attempts.  For example, the C-level 
executive's name as the Display Name when it comes from gmail.com to the 
Finance department to wire money.


From: "CEO Name Here" 

Also, DMARC is supposed to help with this spoofing of the From: header. 
I handle this locally with OpenDMARC adding headers used in an SA meta 
rule.  This is the best way to handle this until SA natively supports DMARC.


--
David Jones

Re: Bayes overtraining

[David Jones]

On 07/25/2018 12:49 PM, Daniele Duca wrote:

Hi,

I'm evaluating incorporating CRM114 in my current setup and I was 
reading the FAQs about training the filter here: 
http://crm114.sourceforge.net/src/FAQ.txt


What made me rethink my actual strategy were the following lines:

...

If you train in only on an error, that's close to the minimal change
necessary to obtain correct behavior from the filter.

If you train in something that would have been classified correctly
anyway, you have now set up a prejudice (an inappropriately strong
reaction) to that particular text.

Now, that prejudice will make it _harder_ to re-learn correct behavior on
the next piece of text that isn't right.  Instead of just learning
the correct behavior, we first have to unlearn the prejudice, and
_then_ learn the correct behavior.
...

In my current SA setup I use bayes_auto_learn along with some custom 
poison pills (autolearn_force on some rules) , and I'm currently 
wondering if over training SA's bayes could lead to the same "prejudice" 
problem as CRM114.


I'm thinking that maybe it would be better to use 
"bayes_auto_learn_on_error 1"


What is your preferred strategy? Train everything you can or train only 
errors?


Daniele



I personally found in our customer mail flow that CRM114 and Bogofilter 
didn't help that much.  We well-trained Bayesian DB with good meta 
rules, RBLs (Invaluement is a must) along with MTA checks/blocks have 
worked out to be spot on for my mail flow.


--
David Jones

Re: Problem with new rules

[David Jones]

On 07/24/2018 11:10 PM, chuckee wrote:

I'm from a reasonably large ESP and we handle all types of emails being sent
via our servers. We've noticed a change with SpamAssassin in the last few
days/weeks which is causing problems.
The following 2 rules are causing these problems:
3.5 HDR_ORDER_FTSDMCXX_DIRECT Header order similar to spam
and
2.0 MIMEOLE_DIRECT_TO_MX MIMEOLE + direct-to-MX



Feel free to set these scores to zero or 0.01 for a quick fix to your 
solution while this gets sorted out.



The high scoring first rule, especially, is causing problems, and is
triggered if a sender is sending emails from Windows Live Mail (it seems to
be just this email client), and their email only has 1 'Received from'
header.
As an ESP we can confirm that it is extremely common for ESP's to strip out
'Received from' headers - if we didn't, then many recipient mail servers
reject emails because they look at the (often bad) reputation of the IP
address of the sender and judge an email on that. For example, if a sender
is sending an email from a hotel, their email would be judged based on the
reputation of the hotel's IP address (which is a ridiculous scenario). We
can confirm that Barracuda is one such email filter that does this.



You shouldn't have to strip out Received headers to prevent legit emails 
from getting rejected.  Either this is from using too aggressive RBLs or 
you need to add to your trusted_networks to look past some Received 
headers.  Another option I use is to make meta rules for certain OK 
Received headers to subtract a few points.



As such, both of the above rules (in our opinion) should not be in place.
ESP's must strip out received headers to prevent legitimate emails from
getting rejected. If we leave them in to cater for those SpamAssassin rules
then many emails will get rejected based on reputation checks of the
sender's own IP address (which is against proper email protocol, but has
been happening ever since email was invented and will continue to happen).

What can be done to influence whoever made these recent rule changes to
revert things back to how they were?



I guess someone needs to look back through the SVN commits to see when 
this was introduced.



Thanks



I filter for about 60,000 mailboxes and I don't see any hits in my mail 
logs for either of those rules in the past 3 days on my production mail 
flow.


--
David Jones

Re: Scans and Invoice spam containg HREF to something bad

[David Jones]

On 06/19/2018 10:38 AM, Andy Smith wrote:
This has literally just come through to me, zero BAYES and got passed my 
custom rule as the HREF URL has changed:



thanks, Andy.


On 19-06-2018 17:33, Kevin A. McGrail wrote:

Well you are welcome to send me new Spamples to look at.  As I noted, 
I've never seen these variants and RBLs aren't hitting them which ALSO 
means you have some new variants.

Regards,
KAM


Content analysis details:   (11.0 points, 5.0 required)

 pts rule name  description
 -- 
--

 1.2 ENA_SUBJ_INVOICE   Subject contains suspicious invoice wording
 0.2 SPF_NONE   SPF: sender does not publish an SPF Record
 0.0 HTML_MESSAGE   BODY: HTML included in message
 1.2 BAYES_50   BODY: Bayes spam probability is 40 to 60%
[score: 0.5037]
 3.2 DCC_CHECK  Detected as bulk mail by DCC (dcc-servers.net)
 0.8 KAM_LAZY_DOMAIN_SECURITY Sending domain does not have any
anti-forgery methods
 2.2 ENA_RELAY_NOT_US   Relayed from outside the US and not on 
whitelists
 2.2 ENA_SPF_NONE   Add points for suspicious emails that don't 
have an SPF

setup.
 0.0 ENA_BAD_SPAM   Spam hitting really bad rules.

I am trying to reply to this thread but these emails look so spammy that 
my outbound filtering is blocking them.


FYI The IVM_URI BL is catching onto these very quickly.  Good job, Rob!

--
David Jones

Re: Scans and Invoice spam containg HREF to something bad

[David Jones]

On 06/19/2018 10:38 AM, Andy Smith wrote:
This has literally just come through to me, zero BAYES and got passed my 
custom rule as the HREF URL has changed:



https://pastebin.com/pBfhXd6B


thanks, Andy.


On 19-06-2018 17:33, Kevin A. McGrail wrote:

Well you are welcome to send me new Spamples to look at.  As I noted, 
I've never seen these variants and RBLs aren't hitting them which ALSO 
means you have some new variants.

Regards,
KAM


Content analysis details:   (11.0 points, 5.0 required)

 pts rule name  description
 -- 
--

 1.2 ENA_SUBJ_INVOICE   Subject contains suspicious invoice wording
 0.2 SPF_NONE   SPF: sender does not publish an SPF Record
 0.0 HTML_MESSAGE   BODY: HTML included in message
 1.2 BAYES_50   BODY: Bayes spam probability is 40 to 60%
[score: 0.5037]
 3.2 DCC_CHECK  Detected as bulk mail by DCC (dcc-servers.net)
 0.8 KAM_LAZY_DOMAIN_SECURITY Sending domain does not have any
anti-forgery methods
 2.2 ENA_RELAY_NOT_US   Relayed from outside the US and not on 
whitelists
 2.2 ENA_SPF_NONE   Add points for suspicious emails that don't 
have an SPF

setup.
 0.0 ENA_BAD_SPAM   Spam hitting really bad rules.

--
David Jones

Re: Question regarding trusted_networks

[David Jones]

On 06/16/2018 09:37 AM, Matus UHLAR - fantomas wrote:

On 06/15/2018 05:44 PM, J Doe wrote:
    Jun 15 18:39:23.422 [8422] dbg: config: trusted_networks are not 
configured; it is recommended that you configure trusted_networks 
manually


My question is:

— Should I manually set trusted_networks to have the IP address of 
the host it is running on and ignore the warning from --lint or …
— Should I not set trusted_networks and ignore the warning from 
--debug ?


On 16.06.18 06:33, David Jones wrote:
internal_networks should be any RFC 1918 networks that your mail 
server sees plus any public networks that are in your control.


no. only servers that deliver mail to you, as your MX servers or other
mailservers directly within your organization should be in
internal_networks.



That is basically the same thing worded a little differently.  If you 
have an internal mail relay and your SA server has a private IP on it, 
then that will be an RFC 1918 IP or range in your internal_networks.


If your SA servers only have public IPs on them, then you won't have any 
RFC 1918 IPs in the internal_networks but you may have 127.0.0.1 and 
fe80::/10 for locally generated email plus any public ranges that 
smarthost to you.


In my case, I have many customers setup to smarthost to smtp.ena.net so 
I have many large CIDRs in my Postfix mynetworks and SA internal_networks.


This can be very different for each use case and probably deserves a 
drawing to explain it better.  Seems like I have seen a graphic or 
something that shows the logic behind this setting.


Mail with all Received headers of IPs within the internal_networks will 
hit the ALL_TRUSTED rule.


trusted_networks should be internal_networks plus any external 
networks that you trust to not send spam -- in other words they are 
known to have their own outbound mail filtering.  This will tell SA to 
go back one more Received: header to test for "last_external" checks 
and RBL checks.


not external networks. only external mail servers you trust not to forge 
e-mail

headers. They may send spam but are not the spam sources.



True.  That is a better way to put it.  I have read that in the SA wiki 
documentation now that you mention it.


--
David Jones

Re: Question regarding trusted_networks

[David Jones]

On 06/16/2018 06:33 AM, David Jones wrote:

On 06/15/2018 05:44 PM, J Doe wrote:

Hello,

I am currently using SpamAssassin 3.4.1 on Ubuntu Linux 16.04.4 LTS.  
I have SA running on a server with Postfix as the MTA on the same server.


I have a question regarding the trusted_networks configuration 
parameter (man Mail::SpamAssassin::Conf).  I manually added this to a 
custom local.cf file and linted it:


 /etc/spamassassin/local.custom.cf:
 trusted_networks 1.2.3.4

 $ spamassassin --lint 
--config-file=/etc/spamassassin/local.custom.cf


This displays:

 Jun 15 18:31:02.893 [8327] warn: netset: cannot include 
1.2.3.4/32 as it has already been included


This lead me to believe that when SpamAssassin loads, it automatically 
adds the IP address of the host it is running on (along with 
localhost, which is mentioned in man).  As a result, I removed the 
trusted_networks entry and a subsequent lint produces no warnings or 
errors.


When I then ran lint and added the --debug flag:

 $ spamassassin --debug --lint 
--config-file=/etc/spamassassin/local.custom.cf


…I see the following in the output:

 Jun 15 18:39:23.422 [8422] dbg: config: trusted_networks are not 
configured; it is recommended that you configure trusted_networks 
manually


My question is:

— Should I manually set trusted_networks to have the IP address of the 
host it is running on and ignore the warning from --lint or …

— Should I not set trusted_networks and ignore the warning from --debug ?

Thanks,

- J



internal_networks should be any RFC 1918 networks that your mail server 
sees plus any public networks that are in your control.


trusted_networks should be internal_networks plus any external networks 
that you trust to not send spam -- in other words they are known to have 
their own outbound mail filtering.  This will tell SA to go back one 
more Received: header to test for "last_external" checks and RBL checks.


For example:

internal_networks 192.168.0.0/16 fe80::/10 96.4.0.0/15 207.191.176.0/20
trusted_networks 192.168.0.0/16 fe80::/10 96.4.0.0/15 207.191.176.0/20 
162.216.126.0/23


My SA servers actually have public IPs on them so technically I don't 
need the 192.168.0.0/16 in the list but I put it in there for reference. 
  If your mail servers are NAT'd and have a private RFC 1918 IP on them 
then you need to include any internal subnet that can send outbound 
through your SA server.




Oh yeh, when using Postfix your internal_networks should basically match 
the Postfix mynetworks value.


# postconf mynetworks

Then copy/paste the internal_networks to trusted_networks and add any 
external networks that can be trusted, if any.  trusted_networks might 
start out identical to internal_networks until you find external sources 
that you want to include later.


--
David Jones

Re: Question regarding trusted_networks

[David Jones]

On 06/15/2018 05:44 PM, J Doe wrote:

Hello,

I am currently using SpamAssassin 3.4.1 on Ubuntu Linux 16.04.4 LTS.  I have SA 
running on a server with Postfix as the MTA on the same server.

I have a question regarding the trusted_networks configuration parameter (man 
Mail::SpamAssassin::Conf).  I manually added this to a custom local.cf file and 
linted it:

 /etc/spamassassin/local.custom.cf:
 trusted_networks 1.2.3.4

 $ spamassassin --lint --config-file=/etc/spamassassin/local.custom.cf

This displays:

 Jun 15 18:31:02.893 [8327] warn: netset: cannot include 1.2.3.4/32 as it 
has already been included

This lead me to believe that when SpamAssassin loads, it automatically adds the 
IP address of the host it is running on (along with localhost, which is 
mentioned in man).  As a result, I removed the trusted_networks entry and a 
subsequent lint produces no warnings or errors.

When I then ran lint and added the --debug flag:

 $ spamassassin --debug --lint 
--config-file=/etc/spamassassin/local.custom.cf

…I see the following in the output:

 Jun 15 18:39:23.422 [8422] dbg: config: trusted_networks are not 
configured; it is recommended that you configure trusted_networks manually

My question is:

— Should I manually set trusted_networks to have the IP address of the host it 
is running on and ignore the warning from --lint or …
— Should I not set trusted_networks and ignore the warning from --debug ?

Thanks,

- J



internal_networks should be any RFC 1918 networks that your mail server 
sees plus any public networks that are in your control.


trusted_networks should be internal_networks plus any external networks 
that you trust to not send spam -- in other words they are known to have 
their own outbound mail filtering.  This will tell SA to go back one 
more Received: header to test for "last_external" checks and RBL checks.


For example:

internal_networks 192.168.0.0/16 fe80::/10 96.4.0.0/15 207.191.176.0/20
trusted_networks 192.168.0.0/16 fe80::/10 96.4.0.0/15 207.191.176.0/20 
162.216.126.0/23


My SA servers actually have public IPs on them so technically I don't 
need the 192.168.0.0/16 in the list but I put it in there for reference. 
 If your mail servers are NAT'd and have a private RFC 1918 IP on them 
then you need to include any internal subnet that can send outbound 
through your SA server.


--
David Jones

Re: Compromised squareup/amazonses account phish

[David Jones]

On 06/13/2018 02:20 PM, Alex wrote:

Hi,

This phish appears to have been routed through Amazon but DKIM signed
by squareup. Is this a compromised squareup.com account?

https://pastebin.com/CxvULHF6

 From 01000163fa173c6b-7d47b00d-af5c-4755-b203-74392b57ec3d-000...@amazonses.com
  Wed Jun 13 13:00:20 2018
From: INVOICE# 
Reply-To: "Advanced Consulting & Treatment, LLC"


Thanks,
Alex



Compromised accounts or bad customers are going to happen with any 
system.  If you see one or two here or there, report them to SpamCop and 
maybe directly to their abuse contact and move on.  If you start seeing 
a pattern of abuse that they are not handling, then start working on a 
way to block the individual sender within that platform.  If it's a 
major platform like amazoneses.com, then it could cause too much 
collateral damage to block the whole platform.


On that particular email in pastebin, my SA would not have blocked it 
either but you may want to bump up the score of DCC_CHECK a bit or make 
a meta rule with DCC_CHECK and DRUGS_DIET to add a point or so.  Also, 
now that IP is not hitting RCVD_IN_HOSTKARMA_W so your scoring may come 
close to blocking that same email today.


I am adding some characteristics of that email to my local rules to 
block them going forward.  For example:


- INVOICE in all caps is suspicious in the Subject -- add 1 point
- INVOICE in the From:name is suspicious -- add 1 point
- Combo of Invoice and DRUGS_DIET should never hit in a real Invoice 
email so adding a couple of points for that.


Now that email is hitting a score of 9.3 on my local SA so thanks for 
the spample.


Invoice phishing emails have become very bad the past 6 months which has 
caused me the most work to stay on top of them.  I have had to take the 
approach of potentially over blocking them to be on the safe side then 
whitelist the good ones since these are causing major economical damage 
in finance departments from social engineering.


--
David Jones

Re: More outlook phish

[David Jones]

On 06/09/2018 01:28 PM, Alex wrote:

Hi,

On a somewhat related note, I just noticed one of our customers have
listed spf.protection.outlook.com in their SPF record:

bestwesternnwcc.com.600 IN  TXT "v=spf1
include:spf.protection.outlook.com -all"

Doesn't this amount to thousands of IP addresses that could
conceivably be used to spoof any other domain that's "hosted" using
one of those IPs?



You are correct.  We saw a spoofed toysrus.com email from a compromised 
account on Office 365 posted on this mailing list in the last year 
sometime.  That means that someone can easily send a fake email from 
O365 and pass SPF checks if Microsoft doesn't properly detect/prevent this.


I recall another thread on this list that said Microsoft forces webmail 
and native Outlook clients to send within their own domain/organization 
but I am pretty sure an AUTH SMTP client (i.e. Thunderbird, Apple Mail, 
etc.) can specify an envelope-from domain outside of their own 
domain/org.  Maybe MS has blocked this recently but I know I have seen 
this in the wild a year or two ago.


The best thing to do is get DKIM signing setup on your own domains and 
try to move toward DMARC p=reject to prevent spoofing.  This primarily 
needs to be done by high profile domains first that are common 
candidates to be spoofed.  I doubt that anyone would really want to 
spoof ena.com on a large scale but bestwesternnwcc.com could be valuable 
to spoof.


--
David Jones

Re: More outlook phish

[David Jones]

On 06/09/2018 07:08 AM, Pedro David Marco wrote:



 >On Saturday, June 9, 2018, 8:03:31 AM GMT+2, Rupert Gallagher 
 wrote:


 >On Fri, Jun 8, 2018 at 23:05, David Jones <mailto:djo...@ena.com>> wrote:



 2.2 MISSING_HEADERS Missing To: header



The fillowing is all one needs.



5.0 MISSING_HEADERS Missing To: header



Remember that e-mail is mail after all.




The To: header may not exist in Outlook if all recipients where in BCC 
and the original To: is company internal...




Email etiquette and best practices recommend putting your own email 
address in the To: if there are no other addresses in the To: when 
Bcc'ing.  This protects the privacy of all recipients and handles the 
Reply-To-All situation well.


Leaving the To: empty looks very spammy to many mail filters so I try to 
educate senders when I come across these situations to help them get the 
best delivery results to the Internet.  Internally that may be fine.


--
David Jones

Re: More outlook phish

[David Jones]

On 06/08/2018 03:17 PM, Alex wrote:

Hi,

Received this one today that was delivered to about 25 recipients,
lacked a To header, routed through outlook.com and contained a link to
a Google Drive doc that's still active.

https://pastebin.com/y1k0LtM1

It was done under the pretense of a ShareFile attachment.

Is a plugin necessary to tag on when the Subject matches content in the From?

I've created some body rules, and tweaked my existing outlook.com
rules, but I thought everyone should see this, and thought others
might have additional ideas for blocking...




Content analysis details:   (5.8 points, 5.0 required)

 pts rule name  description
 -- 
--

-0.0 SPF_HELO_PASS  SPF: HELO matches SPF record
 2.2 MISSING_HEADERSMissing To: header
 0.0 T_KAM_HTML_FONT_INVALID BODY: Test for Invalidly Named or Formatted
Colors in HTML
-3.2 BAYES_00   BODY: Bayes spam probability is 0 to 1%
[score: 0.]
 0.0 HTML_MESSAGE   BODY: HTML included in message
 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to
background
 1.9 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
 0.9 RAZOR2_CHECK   Listed in Razor2 (http://razor.sf.net/)
-0.1 DKIM_VALID Message has at least one valid DKIM or DK 
signature
 0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not 
necessarily valid

 0.2 ENA_NOT_DKIM_VALID_AU  DKIM signed and valid but not from the
originating author
 1.2 KAM_SHORT  Use of a URL Shortener for very short URL
 0.2 ENA_NO_TO_CC   No To: or Cc: so it must have been 
completely Bcc'd

 0.2 ENA_FREEMAIL   No description available.
 2.2 ENA_DIGEST_FREEMAILFreemail account hitting message digest 
spam seen

 by the Internet (DCC, Pyzor, or Razor).


Reminder that I treat all senders on Office 365 as FREEMAIL (commonly 
abused senders) which gets penalized with meta rules to amplify many 
scores.  If something comes from Office 365 with no To: or Cc: header 
with a URL shortener that should be very suspicious.  I need to add 
another meta rule that combines ENA_FREEMAIL and KAM_SHORT to add a 
couple more points.


--
David Jones

Re: Whitelisting envelope-from

[David Jones]

On 06/01/2018 02:37 PM, Alex wrote:

Hi,
I have an email with an address as follows that I'd like to whitelist:

X-Envelope-From: 

Using whitelist_auth doesn't appear to work:

whitelist_auth FredSavage*@cmail19.com

This was really more of an experiment. I'd probably have to generalize
the right side because the mail could come from any of the cmail*.com
systems, and that just looks too dangerous to rely on.

Can I somehow benefit from the DKIM sig? It did not hit DKIM_VALID_AU.

DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=cm; d=example.com;
  
h=Subject:From:To:Reply-To:Date:MIME-Version:Content-Type:List-Unsubscribe:Message-ID;
i=fred.sav...@example.com;
  bh=u6iVMkwz/zWQtjQSeK69bFJbz20=;
  b=CIFpXKQXg1njX8xuPgki5rexAkanCMoaagxZUG431KdyM5r1f2W+pTxkRYmyYtvsNAkO6KLRL
U0n/gGrJi1avhRcxgfe8naqfwyCfjMdOk7bfrsMEzndUoLhVj6nt

Thanks,
Alex



I have a 'whitelist_auth *@cmail19.com' entry and have experienced no 
problems/complaints with createsend.com.  They have a valid unsubscribe 
link and appear to take abuse reports seriously.  Until I have any 
problems from them, I will keep this whitelist_auth entry.


--
David Jones

Re: Huge spam increment in mid-May

[David Jones]

On 06/01/2018 04:47 AM, Pedro David Marco wrote:


 >Do you have any examples?  I have had a quiet past 2 weeks with almost
 >zero reports of junk by my users.  So either my rules are currently
 >tuned well to block the current spam/phishing campaigns or something.  I
 >assumed a botnet had been take down.  I usually have to deal with a few
 >compromised accounts sending spam each week but not lately.  It's been 
nice.

 >I would like to see some examples via pastebin to check my mail
 >filtering logs.


No David, sorry i have no samples... just "numbers" in reports...



Does this mean you are accurately blocking them then if the reports are 
showing them and there are no complaints from users about missing email? 
 I would skim over these subjects to make sure you aren't overblocking. 
 Also I usually check the rule hits to make sure they look consistent 
with spam.


--
David Jones

Re: Huge spam increment in mid-May

[David Jones]

On 06/01/2018 01:46 AM, Pedro David Marco wrote:
Reviewing May reports i see a huge spam increment in mid-May that lasted 
5 days aprox...


Has someone noticed this as well?  maybe a new active bot-net?


Pedro


Do you have any examples?  I have had a quiet past 2 weeks with almost 
zero reports of junk by my users.  So either my rules are currently 
tuned well to block the current spam/phishing campaigns or something.  I 
assumed a botnet had been take down.  I usually have to deal with a few 
compromised accounts sending spam each week but not lately.  It's been nice.


I would like to see some examples via pastebin to check my mail 
filtering logs.


--
David Jones

Re: Invoice phish

[David Jones]

On 05/15/2018 08:26 PM, David B Funk wrote:

On Tue, 15 May 2018, Alex wrote:


Hi,

We received another of those phishes as a result of a compromised O365 
account.


https://pastebin.com/raw/Fv5NKRAP

Anyone able to take a look and provide ideas on how to block them? It
passes with DKIM_VALID_AU, RCVD_IN_SENDERSCORE_90_100 and SPF_PASS.

It's missing headers, and I've written a rule to account for that, but
it would be great to have some other input.

Interestingly, it was passed through a mimecast system first.

The amount of Outlook/O365/Exchange headers in this email is enormous!

Thanks,
Alex


For openers either totally lose "RCVD_IN_HOSTKARMA_W" & 
"RCVD_IN_DNSWL_LOW" rules, or set their score to something minimal (EG 
-0.1 instead of that honking -2.5) or create a rule that detects the 
message being from O365 and meta it with RCVD_IN_HOSTKARMA_W to then add 
an offsetting score to nullify the damage from RCVD_IN_HOSTKARMA_W WRT 
O365.




Due to the junk coming out of O365 lately I have setup OFFSET rules to 
add back any whitelist points for messages received from O365.  I grep'd 
my rules for any rules that subtracted points and made meta rules to add 
back the same amount.  Due to the size of O365 you really can't allow 
any of their IPs to to subtract points.



(Can we get the maintainers of RCVD_IN_HOSTKARMA_W to remove that 
contagion pit called O365 from their list of "good guy" sites?).




Yes, please!  Good RBLs will not list large mail service providers like 
Google and O365 in their whitelists since there is a mixture of ham and 
spam.


I've done a bit of all of the above so an incoming O365 message ends up 
with no "brownie points" at all, so it's only scored on the merits of 
its contents.


Then look for custom anti-phish rulesets. Your example hit a rule 
"RULEGEN_PHISH2" which was in a file 90_rulegen_phish.cf on my server.

(I'm sorry I don't remember where I got that from).

Train bayes, look for custom URIBL lists that might hit that powned 
website.





I have seen quite a bit of junk coming out of mimecast.com's servers in 
recent months.  I am about to add them to my NOTRUST list which puts 
them in the FREEMAIL category of commonly abused mail service providers. 
 Then my meta rules based on ENA_FREEMAIL will bump up points for email 
coming through any NOTRUST servers.


--
David Jones

Re: Invoice phish

[David Jones]

On 05/10/2018 01:32 PM, RW wrote:

On Thu, 10 May 2018 09:55:00 -0500
David Jones wrote:


On 05/10/2018 09:39 AM, RW wrote:



Microsoft has a list of domains it hosts and a list of hosted
domains (and/or its own addresses) tied to each account.  Given how
much reliance MS place on DMARC's preventing spoofing, and how easy
it would be for them to prevent one user spoofing another's domain
on submission, I'd be very surprised if they allow it.
   


They do. I saw an example a few weeks ago.


The very fact that you are citing just one a few week ago strongly
suggests that they don't.



It's possible that it could have been months ago, I guess, so my memory 
could be off.  The fact that someone tested it recently and Microsoft 
blocks it today is encouraging.  Maybe they enabled this logic recently 
to match what Google is doing which is the correct way to handle this 
and prevent "SPF piggy-backing."



Paul Stead claims to have seen it, but it's important to positively
identify it as spoofing and not hacking.
   


Not sure what the difference is from a mail filtering perspective.


The difference is that if domains that include Micrsoft's SPF are as
wide open to spoofing as you suggest, they shouldn't have
def_whitelist_auth entries.



You are correct.  When they were added this issue of "SPF piggy-backing" 
wasn't an issue.  It may have been known to be a potential problem but 
wasn't being actively exploited like the toyrus.com was last year when I 
first noticed it.


It's also possible that those whitelist_* domains have added the 
"include:spf.protection.outlook.com" to their SPF record recently after 
migrating their corporate mail hosting to O365.  We don't have anything 
actively monitoring whitelist entries for SPF record changes so we have 
to rely on abuse reports to this list to remove/change them in SA.


--
David Jones

Re: Invoice phish

[David Jones]

On 05/10/2018 09:39 AM, RW wrote:

On Thu, 10 May 2018 13:49:15 + (UTC)
Pedro David Marco wrote:

  
David Jones wrote:>It's not only compromised well-established

accounts.  Based on the odd

domain names I have seen, I am pretty sure that Microsoft allows
trials of O365 so spammers are signing up and blasting out
junk/phishing emails until they are discovered.  These spammers can
spoof anyone on O365 like toysrus.com and the SPF checks will pass.



I totally agree with David, i have seen trial periods of 45 days for
O365, then spoofingany other O365 customer is trivial with SPF
totally pointless.


But have you actually tried it? I had a concern about  travelodge.co.uk
being whitelisted when its SPF includes gmail, but I tried spoofing it
through smtp.gmail.com and it didn't work.



It was said on this thread that Google rewrites/forces the envelope-from 
address to be the same as the authenticated sender so Google handles 
this properly.  Microsoft does not.  If you can authenticate, then you 
can send as whatever you want with any headers you want to add/spoof 
including the From:.



Microsoft has a list of domains it hosts and a list of hosted domains
(and/or its own addresses) tied to each account.  Given how much
reliance MS place on DMARC's preventing spoofing, and how easy it would
be for them to prevent one user spoofing another's domain on submission,
I'd be very surprised if they allow it.



They do. I saw an example a few weeks ago.


Paul Stead claims to have seen it, but it's important to positively
identify it as spoofing and not hacking.



Not sure what the difference is from a mail filtering perspective.  From 
Microsoft's perspective it is both.  A spammer got someone's password 
and started sending a bunch of invoice phishing emails pretending to be 
a local construction company that happens to host their email on O365 so 
their SPF record is good.


--
David Jones

Re: Invoice phish

[David Jones]

On 05/10/2018 07:37 AM, RW wrote:

On Thu, 10 May 2018 06:50:46 -0500
David Jones wrote:



I am pretty sure that Microsoft allows
trials of O365 so spammers are signing up and blasting out
junk/phishing emails until they are discovered.  These spammers can
spoof anyone on O365 like toysrus.com and the SPF checks will pass.


Do you have a reason to think that that's possible?

It doesn't seem very likely, but there are some default whitelist
entries that should go if it is.





Which part is possible?  The trial accounts blasting spam or the 
toysrus.com SPF matching?  Anyone on O365 not using webmail or Outlook 
can spoof any other O365 customer using authenticated SMTP to 
smtp.office365.com where they can control the envelope-from and From: 
header and the SPF check will pass.  The only thing stopping it is 
Microsoft's ability to detect unusual activity.


--
David Jones

Re: training bayes database

[David Jones]

On 05/10/2018 07:12 AM, Reio Remma wrote:

On 10.05.18 15:08, David Jones wrote:

On 05/10/2018 07:02 AM, Reio Remma wrote:

On 10.05.18 14:58, Matus UHLAR - fantomas wrote:

Am 09.05.2018 um 16:28 schrieb Matthew Broadhead:
i guess my dns is set to use my isp's dns server.  do i need to 
set up dns relay on my machine so it comes from my ip?


there is no way we send more than 500k emails from our domain so 
i should qualify for the free lookup?



On 09/05/18 20:43, David Jones wrote:
Yes.  Setup BIND, unbound, or pdns_recursor on your SA server that 
is not forwarding to another DNS server then set your 
/etc/resolv.conf or SA dns_server to 127.0.0.1.  This will make 
your DNS queries isolated from your IP to stay under their daily 
limit.


Keep in mind that if your SA box is behind NAT that is not 
dedicated to your server then other DNS queries could get combined 
with your shared public IP.  This is not likely since others are 
not going to query RBL/URIBL servers but it's possible.  If your 
SA server is directly on the Internet as an edge mail gateway then 
this won't be a problem.




On 10.05.18 12:15, Matthew Broadhead wrote:
i already had bind handling my dns.  i just had to add to 
/etc/named.conf


allow-query-cache {localhost; any;};


NO!
this way everyone is allowed to use your server as recursive DNS.

only allow "localhost;" it defined all ipv4 and ipv6 address on your 
system.


It's also better to define allow-recursion instead.
While it means something different, they both have same defaults, but
allow-recursion has more clear meaning.


recursion yes;


not needed by default.


and to /etc/resolv.conf

nameserver 127.0.0.1

i cannot believe that is not the default.  i always assumed my dns 
was working correctly.


It's not default to have DNS server on your system. And it's not 
default to
have localhost in resolv.conf - it may be authoritative-only. 


On a slightly related note. We're running a PFSense firewall with DNS 
Forwarder (dnsmasq) in front of our mail server. From what I've 
gleaned from the net is that it caches as well. Should I still 
install a local (BIND) on the mail server?


Thanks!
Reio


YES!  As I was corrected on this mailing list last year, dnsmasq is 
only a forwarding DNS server so it will cause your queries to be 
lumped into whatever it's forwarding to.  Setup a real recursive DNS 
server local on your mail server since it should have it's own 
dedicated NAT or real public IP on your pfSense firewall so your DNS 
queries will be completely isolated. 


There's also the option of DNS Resolver (unbound) on the firewall - 
would that be better?


Reio


No.  Your DNS traffic for your general network served by your firewall 
is much different from your mail server DNS lookup.  You will probably 
want to forward your firewall DNS server to OpenDNS, Google, or even do 
DNS over TLS someday.


https://wiki.apache.org/spamassassin/CachingNameserver

My favorite is PowerDNS Recursor but Unbound is very popular.

--
David Jones

Re: training bayes database

[David Jones]

On 05/10/2018 07:02 AM, Reio Remma wrote:

On 10.05.18 14:58, Matus UHLAR - fantomas wrote:

Am 09.05.2018 um 16:28 schrieb Matthew Broadhead:
i guess my dns is set to use my isp's dns server.  do i need to set 
up dns relay on my machine so it comes from my ip?


there is no way we send more than 500k emails from our domain so i 
should qualify for the free lookup?



On 09/05/18 20:43, David Jones wrote:
Yes.  Setup BIND, unbound, or pdns_recursor on your SA server that 
is not forwarding to another DNS server then set your 
/etc/resolv.conf or SA dns_server to 127.0.0.1.  This will make your 
DNS queries isolated from your IP to stay under their daily limit.


Keep in mind that if your SA box is behind NAT that is not dedicated 
to your server then other DNS queries could get combined with your 
shared public IP.  This is not likely since others are not going to 
query RBL/URIBL servers but it's possible.  If your SA server is 
directly on the Internet as an edge mail gateway then this won't be 
a problem.




On 10.05.18 12:15, Matthew Broadhead wrote:
i already had bind handling my dns.  i just had to add to 
/etc/named.conf


allow-query-cache {localhost; any;};


NO!
this way everyone is allowed to use your server as recursive DNS.

only allow "localhost;" it defined all ipv4 and ipv6 address on your 
system.


It's also better to define allow-recursion instead.
While it means something different, they both have same defaults, but
allow-recursion has more clear meaning.


recursion yes;


not needed by default.


and to /etc/resolv.conf

nameserver 127.0.0.1

i cannot believe that is not the default.  i always assumed my dns 
was working correctly.


It's not default to have DNS server on your system. And it's not 
default to
have localhost in resolv.conf - it may be authoritative-only. 


On a slightly related note. We're running a PFSense firewall with DNS 
Forwarder (dnsmasq) in front of our mail server. From what I've gleaned 
from the net is that it caches as well. Should I still install a local 
(BIND) on the mail server?


Thanks!
Reio


YES!  As I was corrected on this mailing list last year, dnsmasq is only 
a forwarding DNS server so it will cause your queries to be lumped into 
whatever it's forwarding to.  Setup a real recursive DNS server local on 
your mail server since it should have it's own dedicated NAT or real 
public IP on your pfSense firewall so your DNS queries will be 
completely isolated.


--
David Jones

Re: Invoice phish

[David Jones]

On 05/10/2018 05:16 AM, Rupert Gallagher wrote:


On Thu, May 10, 2018 at 00:54, David B Funk 
<dbf...@engineering.uiowa.edu <mailto:dbf...@engineering.uiowa.edu>> wrote:



 4) Less technical sophistication of the server side filtering VS google


Both Google and Microsoft deliver a product for the masses. They are a 
mcdonald after all: you get the quality that you pay for.


Google rejects messages with either failed dmarc or a banned file type, 
which is good, but also accepts advertisements, because it is *free* 
after all. A relative of mine, who insists in using 
gmail, spotted authentic messages to her from IRS and pension fund 
buried in thousands of spam.


O365 is a paid-for service in the sense that one pays to receive spam.

2FA helps against intrusions, but I find people annoyed by the 
technology, so they disable it. Hence the hacked accounts with poor 
passwords.


It's not only compromised well-established accounts.  Based on the odd 
domain names I have seen, I am pretty sure that Microsoft allows trials 
of O365 so spammers are signing up and blasting out junk/phishing emails 
until they are discovered.  These spammers can spoof anyone on O365 like 
toysrus.com and the SPF checks will pass.


They really need to enable rate limiting and unusual GeoIP-usage 
detection.  Maybe they need to setup a well-tuned SpamAssassin platform 
internally to properly detect spam and lock compromised/abusive accounts 
quickly.  :)


--
David Jones

Re: training bayes database

[David Jones]

On 05/09/2018 01:29 PM, Matthew Broadhead wrote:

On 09/05/18 16:37, Reindl Harald wrote:


Am 09.05.2018 um 16:28 schrieb Matthew Broadhead:

it looks like it is working.  so maybe it is just not flagging or moving
the spam?

in a differnt post you showed this status header which *clearly* shows
bayes is working - bayes alone don't flag, the total socre does, moving
don't happen at all on this layer - other software like sieve is
responsible for acting on the headers of a message

quoting URIBL_BLOCKED is a joke - setup a *recursion* *non-forwarding*
nameserver, no dnsmasq or such crap

http://uribl.com/refused.shtml

with your setup you excedd *obviously* rate-limits and have most
DNSBL/URIBL not working and so you can't expect useful results at all

X-Spam-Status: No, score=-18.15 tagged_above=-999 required=6.2
 tests=[AM.WBL=-3, BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25,
 MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001,
 URIBL_BLOCKED=0.001, USER_IN_DEF_SPF_WL=-7.5]
 autolearn=ham autolearn_force=no


i followed the guidance at that url and it gave me
[root@ns1 ~]# host -tTXT 2.0.0.127.multi.uribl.com
2.0.0.127.multi.uribl.com descriptive text "127.0.0.1 -> Query Refused. 
See http://uribl.com/refused.shtml for more information [Your DNS IP: 
213.171.193.134]"


i guess my dns is set to use my isp's dns server.  do i need to set up 
dns relay on my machine so it comes from my ip?


there is no way we send more than 500k emails from our domain so i 
should qualify for the free lookup?


Yes.  Setup BIND, unbound, or pdns_recursor on your SA server that is 
not forwarding to another DNS server then set your /etc/resolv.conf or 
SA dns_server to 127.0.0.1.  This will make your DNS queries isolated 
from your IP to stay under their daily limit.


Keep in mind that if your SA box is behind NAT that is not dedicated to 
your server then other DNS queries could get combined with your shared 
public IP.  This is not likely since others are not going to query 
RBL/URIBL servers but it's possible.  If your SA server is directly on 
the Internet as an edge mail gateway then this won't be a problem.


--
David Jones

Re: Invoice phish

[David Jones]

On 05/09/2018 12:39 PM, Alex wrote:

Hi,


header  __RCVD_OFFICE365Received =~
/\.outbound\.protection\.outlook\.com \[/
header  __RCVD_OFFICE365_PROXY  X-ClientProxiedBy =~
/\.outlook\.com
\(/

header  __OFFICE365_TRUST_ORG   X-OriginatorOrg =~
/^(ena\.com|example\.com)/



You've set this to be your local system, but what if the mail relay
does not process outbound email? What are legitimate values for this
header?



I don't have "ena.com" in my own rule.  Rather I have dozens of others
listed.  Sorry if this is confusing to imply this is for outbound mail.


In other words, is this helpful if your mail relay doesn't process
your outbound mail?



Yes.  It's not meant for outbound but inbound.  I shouldn't have put
"ena.com" in there for me but you could put it in there for your local rules
if you think our email is trustworthy.  :)


I think I'm still a little confused. I did a quick search on existing
email received with that header, and it's a hugely varied list with an
equal amount trustworthy as untrustworthy. I don't think it's feasible
to maintain a list of trustworthy domains for this header, unless I'm
missing something?

Also, the youngliving.com domain that was listed in my spample still
isn't blacklisted on any legitimate list. This looks to be a
legitimate domain with a compromised account. You had mentioned your
rules don't really work on compromised accounts from legitimate
domains, but also that it cuts down on invoice spam.

Is this one of those cases where your rules don't really help?



The problem is anything coming out of O365 is not going to show up on 
traditional IP block lists.  Domain block lists (DBLs) are going to be 
very slow to react and list a domain.  SPFBL would help and IVM might 
quickly list them if there were a URL that could be blocked.


My approach is to bump up the score a little on any invoice-related 
emails and then a little more on "freemail/commonly abused" platforms 
where it's a common source of phishing lately.


Then you only have to maintain the X-OriginatorOrg list for a small 
subset of legit invoice sending domains.


For example, let's say your blocking threshold is the SA default of 5.0. 
 Add a point for all freemail/commonly abused platforms so now their 
threshold is effectively 4.0.  Now add another point for invoice-related 
emails bringing the threshold down to 3.0.


Now legit invoice senders will either score low on their own from good 
reputation, get whitelist_auth/whitelist_from_rcvd entries, or get added 
to the X-OriginatorOrg if they come from O365.


Does that make sense?  This has been working well for me the past couple 
of months after seeing many phishing attacks from O365 compromised 
accounts.  I would rather risk slightly over blocking invoices and 
release/whitelist them later than have a customer get phished and cause 
a lot of financial damage.


I have customers that will blindly pay invoices without matching POs or 
any confirmation if the company name sounds familiar.  I am seeing a lot 
of construction-related phishing emails.  Since there is always 
construction going on, they just assume these are legit.


--
David Jones

Re: Invoice phish

[David Jones]

On 05/09/2018 10:59 AM, Alex wrote:

Hi,


https://pastebin.com/raw/TfvhUu0X


...

What I have had to do is basically increase the score on all invoice emails
to try to block the bad ones and then whitelist the good ones.

That email was BCC'd which is another suspicious trait which is why I bump
up the score for MISSING HEADERS.  I have other ways to penalize these
emails at SMTP time based on the number of BCC'd recipients if this were
received by my servers but I can't tell after the fact like this.


Yes, we've similarly created rules for missing headers.


There is so much junk coming out of Office 365 right now from compromised
accounts and otherwise that it's really hard to accurately filtering O365
email.  I have created a rule based on the X-OriginatorOrg: header to start
subtracting points for known OK senders and then bumping up other rule hits
like invoice-related ones that come from O365.  I know this doesn't help
with compromised accounts in known OK Orgs but it definitely cuts down the
majority of the fake invoice emails.

header  __RCVD_OFFICE365Received =~
/\.outbound\.protection\.outlook\.com \[/
header  __RCVD_OFFICE365_PROXY  X-ClientProxiedBy =~ /\.outlook\.com
\(/

header  __OFFICE365_TRUST_ORG   X-OriginatorOrg =~
/^(ena\.com|example\.com)/


You've set this to be your local system, but what if the mail relay
does not process outbound email? What are legitimate values for this
header?



I don't have "ena.com" in my own rule.  Rather I have dozens of others 
listed.  Sorry if this is confusing to imply this is for outbound mail.



In other words, is this helpful if your mail relay doesn't process
your outbound mail?



Yes.  It's not meant for outbound but inbound.  I shouldn't have put 
"ena.com" in there for me but you could put it in there for your local 
rules if you think our email is trustworthy.  :)


--
David Jones

Re: Invoice phish

[David Jones]

On 05/09/2018 10:02 AM, Alex wrote:

Hi,


One more thing.  I have expanded my definition of FREEMAIL to any Google and
Office 365 senders like this:

header  __RCVD_YAHOOReceived =~ /\.yahoo\.com \[/
header  __RCVD_HOTMAIL  Received =~ /\.hotmail\.com \[/
header  __RCVD_GOOGLE   Received =~ /\.google\.com \[/
header  __RCVD_OFFICE365Received =~
/\.outbound\.protection\.outlook\.com \[/
header  __RCVD_COX_NET  Received =~ /\.cox\.net \[/
header  __RCVD_RR_COM   Received =~ /\.rr\.com \[/
header  __RCVD_CHARTER_NET  Received =~ /\.charter\.net \[/
header  __RCVD_COMCAST_NET  Received =~ /\.comcast\.net \[/
header  __RCVD_CENTURYLINK_NET  Received =~ /\.onyx\.syn-alias\.com
\[/
header  __RCVD_HUGHES_NET   Received =~ /\(mx\.hughes\.net \[/

meta__RCVD_FREEMAIL (__RCVD_YAHOO || __RCVD_HOTMAIL ||
__RCVD_GOOGLE || __RCVD_OFFICE365 || __RCVD_COX_NET || __RCVD_CHARTER_NET ||
__RCVD_COMCAST_NET || __RCVD_RR_COM || __RCVD_CENTURYLINK_NET ||
__RCVD_HUGHES_NET)

metaENA_FREEMAIL(FREEMAIL_FROM || FREEMAIL_REPLYTO
|| FREEMAIL_FORGED_REPLYTO || __RCVD_FREEMAIL)
score   ENA_FREEMAIL0.2

Then I use the ENA_FREEMAIL rule in meta rules to bump up the sensitivity of
mail passing through these sources just like other non-trusted FREEMAIL.


What's the difference between creating new header rules for these or
just adding the domains to the freemail_domains list?



The freemail list are domains from any source.  My rules above are the 
opposite with any domains from/through specific sources that are 
commonly abused.


Microsoft allows customers to sign up an trial O365 potentially abusing 
their platform sending out junk/spam on their own domain.  It would be 
impossible to add every potential domain on the Microsoft platform to 
the freemail domains list.




Wouldn't we want to just add them to SA proper so everyone has them?



If enough people think this is a good idea, we could open a feature 
request to do this properly where any MTA header could be parsed by SA, 
not just Postfix-style Received headers.  Maybe there is already 
something in SA that is very close that can be easily extended.


--
David Jones

Re: Invoice phish

[David Jones]

On 05/09/2018 03:03 AM, Rupert Gallagher wrote:

Is O365 freemail now? Free from Microsoft is an oxymoron.


If you look at the comments in the rule files (20_freemail_domains.cf) 
you will find that FREEMAIL is actually any mail provider that is 
commonly abused and often sends spam.  O365 does fall into this category.


--
David Jones

Re: Invoice phish

[David Jones]

On 05/08/2018 03:47 PM, David Jones wrote:

On 05/08/2018 03:02 PM, Alex wrote:

Hi,
Does anyone have any special techniques for catching these invoice 
phish emails?


https://pastebin.com/raw/TfvhUu0X

I've added a few body rules, and even despite training previous
similar messages as spam, they continue. These emails very closely
resemble legitimate email regarding invoices that purchasing people
fall for them all the time.

Senderscore greater than 90, and routed through O365.

The domain is no longer defined in DNS, but even the x-originating-ip
is not currently listed on any RBL.



I ran it through my SA platform manually:

Content analysis details:   (5.9 points, 5.0 required)

  pts rule name  description
 -- 
--

  1.2 ENA_SUBJ_INVOICE   Subject contains suspicious invoice wording
  2.2 MISSING_HEADERS    Missing To: header
-3.2 BAYES_00   BODY: Bayes spam probability is 0 to 1%
     [score: 0.]
  0.0 HTML_MESSAGE   BODY: HTML included in message
-0.1 DKIM_VALID_AU  Message has a valid DKIM or DK signature 
from author's

     domain
  3.2 DCC_CHECK  Detected as bulk mail by DCC (dcc-servers.net)
-0.1 DKIM_VALID Message has at least one valid DKIM or DK 
signature
  0.1 DKIM_SIGNED    Message has a DKIM or DK signature, not 
necessarily valid
  0.2 ENA_NO_TO_CC   No To: or Cc: so it must have been 
completely Bcc'd

  0.2 ENA_FREEMAIL   No description available.
  2.2 ENA_DIGEST_FREEMAIL    Freemail account hitting message digest 
spam seen

  by the Internet (DCC, Pyzor, or Razor).


What I have had to do is basically increase the score on all invoice 
emails to try to block the bad ones and then whitelist the good ones.


That email was BCC'd which is another suspicious trait which is why I 
bump up the score for MISSING HEADERS.  I have other ways to penalize 
these emails at SMTP time based on the number of BCC'd recipients if 
this were received by my servers but I can't tell after the fact like this.


There is so much junk coming out of Office 365 right now from 
compromised accounts and otherwise that it's really hard to accurately 
filtering O365 email.  I have created a rule based on the 
X-OriginatorOrg: header to start subtracting points for known OK senders 
and then bumping up other rule hits like invoice-related ones that come 
from O365.  I know this doesn't help with compromised accounts in known 
OK Orgs but it definitely cuts down the majority of the fake invoice 
emails.


header  __RCVD_OFFICE365    Received =~ 
/\.outbound\.protection\.outlook\.com \[/
header  __RCVD_OFFICE365_PROXY  X-ClientProxiedBy =~ 
/\.outlook\.com \(/


header  __OFFICE365_TRUST_ORG   X-OriginatorOrg =~ 
/^(ena\.com|example\.com)/


meta    TRUSTED_O365_ORG    __RCVD_OFFICE365 && __OFFICE365_TRUST_ORG && 
(SFP_PASS || DKIM_VALID_AU)

score    TRUSTED_O365_ORG    -1.2

meta    TRUSTED_O365_ORG_INVOICE    TRUSTED_O365_ORG && SUBJ_INVOICE
score    TRUSTED_O365_ORG_INVOICE    -1.2

header  SUBJ_INVOICE    Subject =~ 
/([Ii]nvoice|ACH|Payment.Advice)/
describe    SUBJ_INVOICE    Subject contains suspicious invoice 
wording

score   SUBJ_INVOICE    1.2


I am sure that now I have posted my SUBJ_INVOICE rule on this list that 
the spammers will deviate slightly and I will have to add more regex to 
my rule. I know they want to mimic other commercial invoices so they 
will not change it too much.





One more thing.  I have expanded my definition of FREEMAIL to any Google 
and Office 365 senders like this:


header  __RCVD_YAHOOReceived =~ /\.yahoo\.com \[/
header  __RCVD_HOTMAIL  Received =~ /\.hotmail\.com \[/
header  __RCVD_GOOGLE   Received =~ /\.google\.com \[/
header  __RCVD_OFFICE365Received =~ 
/\.outbound\.protection\.outlook\.com \[/

header  __RCVD_COX_NET  Received =~ /\.cox\.net \[/
header  __RCVD_RR_COM   Received =~ /\.rr\.com \[/
header  __RCVD_CHARTER_NET  Received =~ /\.charter\.net \[/
header  __RCVD_COMCAST_NET  Received =~ /\.comcast\.net \[/
header  __RCVD_CENTURYLINK_NET  Received =~ 
/\.onyx\.syn-alias\.com \[/

header  __RCVD_HUGHES_NET   Received =~ /\(mx\.hughes\.net \[/

meta__RCVD_FREEMAIL (__RCVD_YAHOO || __RCVD_HOTMAIL 
|| __RCVD_GOOGLE || __RCVD_OFFICE365 || __RCVD_COX_NET || 
__RCVD_CHARTER_NET || __RCVD_COMCAST_NET || __RCVD_RR_COM || 
__RCVD_CENTURYLINK_NET || __RCVD_HUGHES_NET)


metaENA_FREEMAIL(FREEMAIL_FROM || 
FREEMAIL_REPLYTO || FREEMAIL_FORGED_REPLYTO || __RCVD_FREEMAIL)

score   ENA_FREEMAIL0.2

Then I use the ENA_FREEMAIL rule in meta rules to 

Re: Invoice phish

[David Jones]

On 05/08/2018 03:02 PM, Alex wrote:

Hi,
Does anyone have any special techniques for catching these invoice phish emails?

https://pastebin.com/raw/TfvhUu0X

I've added a few body rules, and even despite training previous
similar messages as spam, they continue. These emails very closely
resemble legitimate email regarding invoices that purchasing people
fall for them all the time.

Senderscore greater than 90, and routed through O365.

The domain is no longer defined in DNS, but even the x-originating-ip
is not currently listed on any RBL.



I ran it through my SA platform manually:

Content analysis details:   (5.9 points, 5.0 required)

 pts rule name  description
 -- 
--

 1.2 ENA_SUBJ_INVOICE   Subject contains suspicious invoice wording
 2.2 MISSING_HEADERSMissing To: header
-3.2 BAYES_00   BODY: Bayes spam probability is 0 to 1%
[score: 0.]
 0.0 HTML_MESSAGE   BODY: HTML included in message
-0.1 DKIM_VALID_AU  Message has a valid DKIM or DK signature 
from author's

domain
 3.2 DCC_CHECK  Detected as bulk mail by DCC (dcc-servers.net)
-0.1 DKIM_VALID Message has at least one valid DKIM or DK 
signature
 0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not 
necessarily valid
 0.2 ENA_NO_TO_CC   No To: or Cc: so it must have been 
completely Bcc'd

 0.2 ENA_FREEMAIL   No description available.
 2.2 ENA_DIGEST_FREEMAILFreemail account hitting message digest 
spam seen

 by the Internet (DCC, Pyzor, or Razor).


What I have had to do is basically increase the score on all invoice 
emails to try to block the bad ones and then whitelist the good ones.


That email was BCC'd which is another suspicious trait which is why I 
bump up the score for MISSING HEADERS.  I have other ways to penalize 
these emails at SMTP time based on the number of BCC'd recipients if 
this were received by my servers but I can't tell after the fact like this.


There is so much junk coming out of Office 365 right now from 
compromised accounts and otherwise that it's really hard to accurately 
filtering O365 email.  I have created a rule based on the 
X-OriginatorOrg: header to start subtracting points for known OK senders 
and then bumping up other rule hits like invoice-related ones that come 
from O365.  I know this doesn't help with compromised accounts in known 
OK Orgs but it definitely cuts down the majority of the fake invoice emails.


header  __RCVD_OFFICE365Received =~ 
/\.outbound\.protection\.outlook\.com \[/
header  __RCVD_OFFICE365_PROXY  X-ClientProxiedBy =~ 
/\.outlook\.com \(/


header  __OFFICE365_TRUST_ORG   X-OriginatorOrg =~ 
/^(ena\.com|example\.com)/


meta	TRUSTED_O365_ORG	__RCVD_OFFICE365 && __OFFICE365_TRUST_ORG && 
(SFP_PASS || DKIM_VALID_AU)

score   TRUSTED_O365_ORG-1.2

metaTRUSTED_O365_ORG_INVOICETRUSTED_O365_ORG && SUBJ_INVOICE
score   TRUSTED_O365_ORG_INVOICE-1.2

header  SUBJ_INVOICESubject =~ 
/([Ii]nvoice|ACH|Payment.Advice)/
describeSUBJ_INVOICESubject contains suspicious invoice 
wording

score   SUBJ_INVOICE1.2


I am sure that now I have posted my SUBJ_INVOICE rule on this list that 
the spammers will deviate slightly and I will have to add more regex to 
my rule. I know they want to mimic other commercial invoices so they 
will not change it too much.



--
David Jones

Re: Invoice phish

[David Jones]

On 05/08/2018 03:02 PM, Alex wrote:

Hi,
Does anyone have any special techniques for catching these invoice phish emails?

https://pastebin.com/raw/TfvhUu0X

I've added a few body rules, and even despite training previous
similar messages as spam, they continue. These emails very closely
resemble legitimate email regarding invoices that purchasing people
fall for them all the time.

Senderscore greater than 90, and routed through O365.

The domain is no longer defined in DNS, but even the x-originating-ip
is not currently listed on any RBL.



I would also report that to SpamCop since that IP is Google's.

It's now listed on dnsbl.spfbl.net probably because of your original 
posting.  :)


--
David Jones

Re: Weird Long-Term Whitelist Issue

[David Jones]

On 05/05/2018 07:37 PM, Charlie Wilkinson wrote:

Hi,

I've been running spamassassin forever and this has been bugging me for 
probably a decade, across various versions of SA and RedHat/FedoraCore 
Linux. Googling has produced nothing helpful and the problem has finally 
annoyed me enough to join this list and ask the real experts...


Basically, I have a whitelist.cf file with about 2000 whitelist_from and 
whitelist_from_rcvd entries in it. It seems that at some point into the 
file, SA quits reading it, or is otherwise ignoring entries, such that 
emails from "whitelisted" senders several hundred lines into the 
whitelist.cf file are not being detected as whitelisted.


My quickie fix has been to move ignored entries to the top of the 
whitelist file when I notice them. That reliably fixes the non-working 
entries, but gets me no closer to a real solution.


Running --lint and --lint -D shows no issues.

Any ideas of what might be causing this, or further troubleshooting 
steps to try?


My setup is currently a fairly plain-vanilla 
spamassassin-3.4.0-13.fc21.i686 on Fedora Core 21. (Yeah, I know... 
Migration to new server/CentOS is underway.) I'm running sendmail with 
MimeDefang and a heavily customized mimedefang-filter script to feed 
emails to SA.


Thanks for any help,

             Charlie



I too have thousands of whitelist_* entries but haven't seen any 
problems like this.  I put each type of entry in it's own file like:


# wc -l 99_white*
  6503 99_whitelist_auth.cf
   216 99_whitelist_from.cf
 2 99_whitelist_from_dkim.cf
  1363 99_whitelist_from_rcvd.cf
15 99_whitelist_to.cf
  8099 total

Currently running SA 3.4.1 on CentOS 6 upgrading to latest MailScanner 
on C7 and hopefully SA 3.4.2 this summer.


I would highly recommend switching from sendmail to postfix with 
postscreen and postwhite when you migrate to CentOS.  You will see a 
major improvement in your mail filtering just from that change.


--
David Jones

Re: Malforrmed List-id

[David Jones]

On 05/03/2018 12:05 PM, Alex wrote:

Hi,


header__KP_LIST_ID_DOMAIN_IN_BRACKETS List-id =~
/<([\w-]+)?(\.[\w-]+)+>/

describe KP_LIST_ID_DOMAIN_IN_BRACKETS List-id has domain in angle brackets
meta   KP_LIST_ID_DOMAIN_IN_BRACKETS __KP_LIST_ID_DOMAIN_IN_BRACKETS
score  KP_LIST_ID_DOMAIN_IN_BRACKETS -15.0


I think this rule is a bad idea. If the email is not spam, then
there's nothing that otherwise would restrict it from being delivered.
Including a valid List-ID while the rest of the email is spam is an
almost guarantee it will be delivered. No reason to deduct points for
a valid email.




I agree.  Whitelisting or subtracting points should be tied to domain 
authentication or IP reputation.  Spammers are reading this email thread 
and are already crafting emails to match this rule.


--
David Jones

Re: Just to lighten your day?

[David Jones]

On 05/02/2018 01:21 PM, Joe Acquisto-j4 wrote:

One slipped through, with this subtle sig line (thought it might brighten 
someones day . . . )

"Note: Failure to Verify will lead to final termination of your email account.

Technical Team
Email Administrator
All Right Reversed 2018.(c)"



Please post the full email, with all headers, minimally redacted to 
pastebin.com and send us a link.


--
David Jones

Re: Anti Phish Rules

[David Jones]

MailScanner became very mature and didn't need any major updates for years then 
Jules turned it over to Jerry Benton who had a commercial product based on it.  
It's still being updated and runs fine now on systemd-based OSes and newer 
versions of Perl.  One of our customers, Shawn Iversion, is helping Jerry 
maintain MailScanner now as part of his EFA project.  https://efa-project.org/


From: Kevin Miller 
Sent: Thursday, April 26, 2018 4:16 PM
To: users@spamassassin.apache.org
Subject: RE: Anti Phish Rules


It’s not abandonware – Jules handed it off to some other folks that are 
actively putting out new versions.  As a matter of fact one came out not too 
long ago.  MailWatch for MailScanner is also being actively developed still.

Latest/greatest is available at 
www.mailscanner.info for anyone wanting to check 
it out.



...Kevin

--

Kevin Miller

Network/email Administrator, CBJ MIS Dept.

155 South Seward Street

Juneau, Alaska 99801

Phone: (907) 586-0242, Fax: (907) 586-4588 Registered Linux User No: 307357



From: Noel Butler [mailto:noel.but...@ausics.net]
Sent: Thursday, April 26, 2018 12:51 PM
To: users@spamassassin.apache.org
Subject: Re: Anti Phish Rules



On 26/04/2018 18:12, Matus UHLAR - fantomas wrote:

On 26.04.18 18:00, Nick Edwards wrote:

We've been using a separate product to do this, but it struck me, maybe
spamassassin can do this easier (or without having to call yet another
binary to run as can over mails)

Rules that look at URLs in a html message  href and src tags, check the "A"
tag to see if there is a URL there, and if they do not match,  consider it
a phis so apply said phis score to the message.

Has anyone done this? module even?

the main problem: may non-spam senders do that, see:

https://wiki.apache.org/spamassassin/AntiPhishFakeUrlRule

and further the discussion in linked bug:

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=4255





I suspect Nick is still using and referring to mailscanner (which is/was 
written in perl), it has/had this ability, I (like a good few of the names 
around here) used it back in the day as well, until it became clear it was 
abandonware, and did not like certain newer versions of perl causing exits 
after each scan, mind you, I did dump it for amavisd back around 2008/9/10, 
that said I liked that function, and rarely noticed any FP's, my memorys hazy, 
but IIRC, it disarmed the links, rather than take any scoring action... I might 
be wrong though, like I said, its been along time.



--

Kind Regards,

Noel Butler

This Email, including any attachments, may contain legally privileged 
information, therefore remains confidential and subject to copyright protected 
under international law. You may not disseminate, discuss, or reveal, any part, 
to anyone, without the authors express written authority to do so. If you are 
not the intended recipient, please notify the sender then delete all copies of 
this message including attachments, immediately. Confidentiality, copyright, 
and legal privilege are not waived or lost by reason of the mistaken delivery 
of this message. Only PDF and 
ODF documents accepted, please do 
not send proprietary formatted documents

Re: Anti Phish Rules

[David Jones]

That was for a specific spoofing campaign so remove it if you want.  That was 
only an example to show what can be done to pair up with 
whitelist_auth/whitelist_dkim entries.  I would not put that particular one in 
the core SA ruleset if there was enough interest to add this rule.


I also have a rule just like this with full names of spoofing targets like CEOs 
and finance people to block fake requests to transfer money or click a link.


From: sha...@shanew.net <sha...@shanew.net>
Sent: Thursday, April 26, 2018 10:01 AM
To: users@spamassassin.apache.org
Subject: Re: Anti Phish Rules

On Thu, 26 Apr 2018, David Jones wrote:

> header  __BAD_FROM_NAME From:name =~
> /(^chase$|chase\.com|Internal Revenue Service|banking|Bank of
> America|American Express|Wells Fargo|NavyFederal|Geico|E-fax|Share.oint|UPS
> Delivery|FedEx|PayPal|Apple Support|USAA|.ropbox|Dro.box)/i
> metaBAD_FROM_NAME   __BAD_FROM_NAME && !ALL_TRUSTED
> describe  BAD_FROM_NAME   Displayed From contains bad information to
> trick the recipients
> score   BAD_FROM_NAME   4.0

People named Chase may not care for that first item in the grouping

--
Public key #7BBC68D9 at| Shane Williams
http://pgp.mit.edu/|  System Admin - UT CompSci
=--+---
All syllogisms contain three lines |  sha...@shanew.net
Therefore this is not a syllogism  | 
www.ischool.utexas.edu/~shanew<http://www.ischool.utexas.edu/~shanew>

Re: Anti Phish Rules

[David Jones]

I have a local rule that adds a few points for commonly spoofed companies like 
Paypal, Bank of America, Chase, Fedex, etc. since all of these will have good 
SPF/DKIM and now have def_whitelist_auth entries in the 60_whitelist_auth.cf.


Maybe we need to consider putting these in the SA core ruleset to help 
everyone.  The problem is that once we publish the exact rules for everyone 
running sa-update, then the spammers just start spoofing new companies.  At 
least it does help stop many of the common spoofing emails.


header  __BAD_FROM_NAME From:name =~ /(^chase$|chase\.com|Internal 
Revenue Service|banking|Bank of America|American Express|Wells 
Fargo|NavyFederal|Geico|E-fax|Share.oint|UPS Delivery|FedEx|PayPal|Apple 
Support|USAA|.ropbox|Dro.box)/i
metaBAD_FROM_NAME   __BAD_FROM_NAME && !ALL_TRUSTED
describe  BAD_FROM_NAME   Displayed From contains bad information to 
trick the recipients
score   BAD_FROM_NAME   4.0

You need to make sure any company name above has a whitelist_auth or 
whitelist_dkim entry for their real emails.

Dave



From: Emanuel Gonzalez 
Sent: Thursday, April 26, 2018 7:08 AM
To: Matus UHLAR - fantomas; users@spamassassin.apache.org
Subject: Re: Anti Phish Rules


Here is an example of an phishing email:

Authentication-Results: spf=none (sender IP is 200.58.117.126)
 smtp.mailfrom=ppl3.com; hotmail.com; dkim=fail (body hash did not verify)
 header.d=c0800455.domain.com;hotmail.com; dmarc=none action=none
 header.from=ppl3.com;
Received-SPF: None (protection.outlook.com: ppl3.com does not designate
 permitted sender hosts)
Received: from smht-x-x.domain.com (200.58.117.126) by
 DB5EUR03FT006.mail.protection.outlook.com (10.152.20.106) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id
 15.20.696.11 via Frontend Transport; Thu, 26 Apr 2018 10:22:41 +
X-IncomingTopHeaderMarker: 
OriginalChecksum:BC2CEE5C26E95CD829053392BF062A8A8EF5B80B38721334E4D422793F5D4711;UpperCasedChecksum:DBAACD04967E0EBE075BAE00C7F9A355386276A19553DE2D32FBB1B903C63A0B;SizeAsReceived:3262;Count:21
Received: from c00.domain.com (c00 [172.x.x.x])
by smarthost.domain.com (Postfix) with ESMTPS id 4FC2A2A24
for ; Thu, 26 Apr 2018 07:22:39 -0300 (-03)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
d=c0800455.domain; s=mail; h=Content-Transfer-Encoding:Content-Type:

MIME-Version:Date:Subject:To:From:Message-ID:Sender:Reply-To:Cc:Content-ID:

Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc

:Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:
List-Subscribe:List-Post:List-Owner:List-Archive;
bh=LUcrH5hRyj2Ujx36ZGDIENRVn7MtrTTfammZnXLJGrg=; 
b=RXl8e5v1c/TQQo/kLRo+tyg4VA

54BiXbsaC0z2TFM3dMDf4uNZpILl2RXYzhwcKptr9UVm+LQHUXW9UJmdqXKywlisZXyyJtk4U5KSP
LcaKmcWO+d9HwQWLY3MeDjBT4iw4xEiEeVN4Myra1K8Mf8Pfs3U42IqPHJWF4lLVPSeo=;
Received: from [105.155.80.137] (helo=Abdo-PC)
by c000.domain.com with esmtpsa (TLSv1:EDH-RSA-DES-CBC3-SHA:168)
(Exim 4.87_1)
(envelope-from )
id 1fBe2q-0006i7-4d
for mkch...@hotmail.com; Thu, 26 Apr 2018 07:22:39 -0300
Message-ID: <0364314f-43216-021f47358625@abdo-pc>
From: PayPal Inc 


Could you apply some verification for the signature dkim? I'm working in it




De: Matus UHLAR - fantomas 
Enviado: jueves, 26 de abril de 2018 5:12:05
Para: users@spamassassin.apache.org
Asunto: Re: Anti Phish Rules

On 26.04.18 18:00, Nick Edwards wrote:
>We've been using a separate product to do this, but it struck me, maybe
>spamassassin can do this easier (or without having to call yet another
>binary to run as can over mails)
>
>Rules that look at URLs in a html message  href and src tags, check the "A"
>tag to see if there is a URL there, and if they do not match,  consider it
>a phis so apply said phis score to the message.
>
>Has anyone done this? module even?

the main problem: may non-spam senders do that, see:

https://wiki.apache.org/spamassassin/AntiPhishFakeUrlRule

and further the discussion in linked bug:

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=4255

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...

Re: plugin: eval failed: __alarm__ignore__(xxx) how to troubleshoot

[David Jones]

On 04/18/2018 09:00 AM, Chris Conn wrote:

Hello,

We recently started having problems on a few servers with spamd 
processes starting to hog RAM (3G++ per child) and we are noting some 
timeouts in the logs.  as well,  we are seeing this msg


Apr 18 08:48:24 spamd[2451]: plugin: eval failed: __alarm__ignore__(156)
Apr 18 08:58:25 spamd[11283]: plugin: eval failed: __alarm__ignore__(156)
Apr 18 09:05:28 spamd[17774]: plugin: eval failed: __alarm__ignore__(156)
Apr 18 09:09:06 spamd[21486]: plugin: eval failed: __alarm__ignore__(156)
Apr 18 09:11:56 spamd[23412]: plugin: eval failed: __alarm__ignore__(156)
Apr 18 09:13:50 spamd[24872]: plugin: eval failed: __alarm__ignore__(156)
Apr 18 09:14:11 spamd[25147]: plugin: eval failed: __alarm__ignore__(246)
Apr 18 09:19:37 spamd[29797]: plugin: eval failed: __alarm__ignore__(246)
Apr 18 09:28:54 spamd[5298]: plugin: eval failed: __alarm__ignore__(246)
Apr 18 09:40:00 spamd[13590]: plugin: eval failed: __alarm__ignore__(246)
Apr 18 09:40:28 spamd[13847]: plugin: eval failed: __alarm__ignore__(246)
Apr 18 09:41:21 spamd[14389]: plugin: eval failed: __alarm__ignore__(246)
Apr 18 09:50:45 spamd[24479]: plugin: eval failed: __alarm__ignore__(331)

This alarm__ignore seems to be a function in Timeout.pm

this is a relatively old install, SA 3.3.1 on Centos6 (stock RPMs)

Any tips on how to troubleshoot the essence or root of this issue?

Thanks in advance,

Chris


I too have been seeing a very high number of SA timeouts via MailScanner 
the past week and would like to know how to troubleshoot these timeouts. 
 I have never been able to catch problem messages in the act to figure 
out what is causing them.


--
David Jones

Re: SpamAssassin 3.4.2.

[David Jones]

On 04/17/2018 05:19 PM, Bill Cole wrote:

On 17 Apr 2018, at 16:54, John Hardin wrote:


On Tue, 17 Apr 2018, David Jones wrote:


On 04/17/2018 03:29 PM, Kevin A. McGrail wrote:

Dave, why would it go into EPEL?  SpamAssassin is a core RPM.


I will be updating my main SA platform servers to CentOS 7 this 
summer so this should be good timing to get SA 3.4.2 from the core 
repo update.  :)


RHEL 7 / CentOS 7 core is still on SA 3.4.0 - I had to manually roll 
my own SA 3.4.1 RPMs from Fedora SRPMs.


Anybody here from RH that can commit to packaging SA 3.4.2 for a RHEL 
7 core update or explain why it's behind?


It's a Red Hat long-standing stability policy. They backport security 
and some bugfix patches (which is why they have a version '3.4.0-2' RPM) 
but they do not generally import any upstream version updates that have 
any potential backward compatibility risk at all except at major EL 
version releases. So EL7 systems will never get anything but a patched 
3.4.0.




They won't go to 3.5 or 4.0 for sure and they do backport security 
patches in the dash versions but I would think they would do minor 
(third dot point) updates from 3.4.0 to 3.4.1.  Well if they don't then 
I guess we can find someone to help with the RPM building and put it 
somewhere that we can refer to on the wiki or on this list when people ask.


If you want to track current releases of software on something like 
RHEL, use Fedora.


Fedora is way too unstable to use for a server that you want to keep 
around for years.  There is so much tweaking and customization to 
setting up a mail filtering server, that would be tough to maintain on 
Fedora unless you do a vanilla install of everything and stop in which 
case you aren't going to get very good results in the mail filtering 
accuracy.


--
David Jones

Re: SpamAssassin 3.4.2.

[David Jones]

On 04/17/2018 04:39 PM, Bill Cole wrote:

On 17 Apr 2018, at 16:38, David Jones wrote:


On 04/17/2018 03:29 PM, Kevin A. McGrail wrote:

Dave, why would it go into EPEL?  SpamAssassin is a core RPM.



Oh yeh.  I guess because it's been so long since we had an update and 
my main boxes are running CentOS/SL 6.9 that I forgot it was a core 
package.  The CentOS 5 and 6 boxes out there aren't going to get the 
new version unless it gets put in some other repo like EPEL or another 
third party since they are not getting any updates.


My understanding of EPEL policy is that its packages never replace the 
EL base packages.


It is often possible to install RPMs from the Fedora updates repos that 
are analogous to your EL/CentOS version.





You are correct.  When 3.4.2 is released soon, we (the SA team) should 
come up with an "official" way to install it on the wiki or something 
for all major OSes to promote upgrading.  Many currently supported and 
updated OSes would simply be handled by the core/primary packaging 
methods but there will be some older OSes that need to be handled 
differently.


This page is a bit outdated:
https://wiki.apache.org/spamassassin/UpgradingVersion

Even the latest versions of CentOS and RHEL 7 only have 3.4.0.  It 
should have 3.4.1 years ago and 3.4.2 in a couple of months.  Why hasn't 
the packaging in RHEL/CentOS been updated to 3.4.1?


--
David Jones

Re: SpamAssassin 3.4.2.

[David Jones]

On 04/17/2018 03:29 PM, Kevin A. McGrail wrote:

Dave, why would it go into EPEL?  SpamAssassin is a core RPM.



Oh yeh.  I guess because it's been so long since we had an update and my 
main boxes are running CentOS/SL 6.9 that I forgot it was a core 
package.  The CentOS 5 and 6 boxes out there aren't going to get the new 
version unless it gets put in some other repo like EPEL or another third 
party since they are not getting any updates.


I will be updating my main SA platform servers to CentOS 7 this summer 
so this should be good timing to get SA 3.4.2 from the core repo update.  :)



--
Kevin A. McGrail
Asst. Treasurer & VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

On Tue, Apr 17, 2018 at 3:19 PM, Amir Caspi <ceph...@3phase.com 
<mailto:ceph...@3phase.com>> wrote:


On Apr 17, 2018, at 1:12 PM, David Jones <djo...@ena.com
<mailto:djo...@ena.com>> wrote:
> 
> Once 3.4.2 comes out soon, we need to get an official version in EPEL or something.  Hopefully someone knows someone at EPEL to make this happen.  I think everyone had to build 3.4.1 themselves from the Fedora RPM spec file.  That hinders adoption of new versions in the RPM-based OSes then we on this list have to address issues that have been solved already by the new version.  :)


One of the Fedora maintainers, Kevin Fenzi, had previously set up a
repo for SpamAssassin.  If he's still on this list, he can post the
info.  I'm not sure if that personal repo is still maintained or not
... but that's how I got 3.3.2 and subsequent releases up to 3.4.1
on my CentOS system.

Speaking of which, even though CentOS 5 is EOL, I hope that he (or
someone else) will release a 3.4.2 RPM for RHEL 5.  Unfortunately
there are still a number of production systems out there that can't
be upgraded yet...

Cheers.

--- Amir





--
David Jones

Re: SpamAssassin 3.4.2.

[David Jones]

On 04/17/2018 02:00 PM, Reio Remma wrote:

Mkay. Half an evening of figuring out how RPM building works and voila.

Let the testing commence. :)

Running transaction
   Updating   : spamassassin-3.4.2-0.el7.centos.x86_64
   Cleanup    : spamassassin-3.4.1-17.el7.centos.x86_64
   Verifying  : spamassassin-3.4.2-0.el7.centos.x86_64
   Verifying  : spamassassin-3.4.1-17.el7.centos.x86_64



Once 3.4.2 comes out soon, we need to get an official version in EPEL or 
something.  Hopefully someone knows someone at EPEL to make this happen. 
 I think everyone had to build 3.4.1 themselves from the Fedora RPM 
spec file.  That hinders adoption of new versions in the RPM-based OSes 
then we on this list have to address issues that have been solved 
already by the new version.  :)




On 17.04.2018 18:49, Kevin A. McGrail wrote:
Svn for 3.4 is very stable and suitable for most production level 
machines IMO.


--
Kevin A. McGrail
Asst. Treasurer & VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

On Tue, Apr 17, 2018 at 10:45 AM, Alex <mysqlstud...@gmail.com 
<mailto:mysqlstud...@gmail.com>> wrote:


Hi,

> Can't wait for all the little fixes. For now I've carried some
over by hand
> to cut down on log errors (TxRep etc).

Are you pulling from svn? I've been doing that for some time and it's
stable and I'm assuming has all the current fixes.

It requires you to build it yourself from scratch, but it compiles and
builds easily.
https://wiki.apache.org/spamassassin/DownloadFromSvn
<https://wiki.apache.org/spamassassin/DownloadFromSvn>



>
    > Reio
>







--
David Jones

Please add these blocks

[David Jones]

Calling all RBL/URIBL operators on this list, please block these:

https://pastebin.com/gGkK2gMq

https://pastebin.com/L7gygRn7

https://pastebin.com/ukaQ1pps

https://pastebin.com/DBiUT6k3

https://pastebin.com/Hcm6mLzx

I receive a ton of these daily and they aren't listed on anyone's RBL/URIBL.

P.S.  I would love to help with any RBL/URIBLs with honeypot/spamtrap 
accounts if anyone would like to contact me off list.


--
David Jones