Using -t to test rule changes

[Alex]

Hi, I'm using the latest version of SA from trunk (although I don't think
that matters) and trying to make adjustments to rules on a particular
false-positive email that was quarantined by amavis so I can adjust the
rules to prevent it from being quarantined.

The problem is that amavis manipulates the headers to prevent me from being
able to process them with spamassassin -t again.

I've tried using -d to remove the previous reports first, adding the
envelope-from and return-path but SPF fails, of course, and it also prints
twice the triggered rules, one set after the other.

What can be done to be able to process a quarantined email again so I can
make adjustments to prevent it from being quarantined?

Tips for improving bounce message deliverability?

[Alex]

Hi,
I'm using SA 4.0.1 and amavisd with postfix. I've identified a few bounce
messages in the quarantine because they weren't identified properly. Here's
one:
https://pastebin.com/RMNkcyhF

For example, it matches on
*  3.1 URI_IMG_CWINDOWSNET Non-MSFT image hosted by Microsoft Azure
infra, possible phishing
 *  2.6 HOSTED_IMG_DIRECT_MX Image hosted at large ecomm, CDN or hosting
 *  site, message direct-to-mx

It also matches on ANY_BOUNCE_MESSAGE and BOUNCE_MESSAGE. Should metas be
created to avoid adding the above scores?

What more can be done to improve deliverability of these messages? Perhaps
this is something postfix can identify and bypass scanning?

Re: dcc on empty email

[Alex]

Hi,

> I'm noticing DCC is triggering on emails with an empty body. I'd like to
> create a hash that matches messages with an empty body and other simple
> messages.
>
> What am I doing wrong? I've tried it with a zero-length file as well as
> one with just a few characters. It looks like I don't understand what the
> format of the file should be.
>
> [root@beast dcc]# /usr/bin/dccproc -QCw whiteclnt < dcc-empty
> missing message body; fatal error
>

I have a better understanding of how this works now. It apparently still
requires a valid email to be used, just with an actual empty body if I
wanted to whitelist that portion of it.

/usr/bin/dccproc -QCw whiteclnt < email-with-empty-body
X-DCC-www.nova53.net-Metrics: beast.example.com 1204; Body=0 Fuz1=0 Fuz2=0
reported: 0   checksum  server
 env_From: d41d8cd9 8f00b204 e9800998 ecf8427e
 From: 55a5141a 442cf35f 22622946 72511b73
   Message-ID: cebc1e5c 40f54129 90709930 ee918829
 Received: 9afca068 dc711459 c84cadb6 627877e9
 Body: d1b04397 6af3d941 68459a63 a155b202   0
 Fuz1: 3d69b970 c60c2b73 95050fee 7971cae8   0
 Fuz2: d35d922e 637a421c 0da33c04 8498ab36   0

although I'm not sure now where these values should be used/stored for dcc?

I also happened across this link that contains a list of checksums for
nearly empty messages, although I don't know how current or useful it is.
https://www.iecc.com/dcc-testmsg-whitelist.txt

dcc on empty email

[Alex]

Hi,
I'm noticing DCC is triggering on emails with an empty body. I'd like to
create a hash that matches messages with an empty body and other simple
messages.

What am I doing wrong? I've tried it with a zero-length file as well as one
with just a few characters. It looks like I don't understand what the
format of the file should be.

[root@beast dcc]# /usr/bin/dccproc -QCw whiteclnt < dcc-empty
missing message body; fatal error

Re: QR code phish?

[Alex]

Hi,

On Thu, Feb 1, 2024 at 5:01 PM Kevin A. McGrail  wrote:

> Hi Alex, we are definitely seeing them.  There is code in trunk for this
> with one of the plugins and rules in the KAM ruleset using the new
> code.  LMK if you need more info.
>

It looks like it's tied to the Raptor service and the ExtractText plugin.
Do you have more details on doing that?

Thanks,
Alex

QR code phish?

[Alex]

Hi,

I'm just wondering if there is any mechanism for detecting and blocking QR
code emails? Would that require using image detection? Perhaps instead it's
a database of known malicious QR codes?

Has anyone even really seen any?

wellsfargo/google drive

[Alex]

Hi,
Google Drive is being used to send links with malicious content. I know,
shocking. But should Google Drive be in the DKIM WL?

What more can be done to stop these? I have a few body filters, but these
are just links sent using Google to PDFs with malicious links.

https://pastebin.com/Qpj1drSa

Spreadsheet::Excel ?

[Alex]

Hi,

Barracuda recently announced they've identified a vulnerability in the
Spreadsheet::Excel library used by amavis in their appliances. I didn't
realize they were still using amavis and open source (and presumably
spamassassin?).
https://www.barracuda.com/company/legal/esg-vulnerability

I don't have this library on my system - is there a plugin that enables
parsing of Excel spreadsheets for malicious code? I realize there is the
ExtractText plugin, and although it doesn't actually work to identify any
potentially malicious code within an Excel file, it does look to be much
more comprehensive and capable.

https://www.techtarget.com/searchsecurity/news/366564654/Another-Barracuda-ESG-zero-day-flaw-exploited-in-the-wild

Re: Too many dots?

[Alex]

Hi,

>>Does it sound reasonable to add 3 points plus another 1.5 simply for
> >>having been sent by sendgrid? How do we offset those points? Do we
> >>just rely on bayes/txrep?
> >>
> >>I think my bayes db is pretty well-trained, but there's also a lot
> >>of account activation fraud emails.
>
> On 16.11.23 10:29, Kris Deugau wrote:
> >Third party rule sets always need evaluation for your local mail flow.
>
> Just FYI:
> AC_FROM_MANY_DOTS stock SA rule and has score 3 as OP complained:
>
> score  AC_FROM_MANY_DOTS  2.999 2.999 2.999 2.999
>

Yes, of course, I realize I can control scores on my own system - I was
just requesting an analysis because it seems quite high, and thought it
deserved to be evaluated for everyone.

Also, the KAM rules are designed to be used in conjunction with the stock
rules, so it also seemed somewhat punitive to award so many points and to
be expected to offset them for a completely benign email.

Thanks,
Alex

Too many dots?

[Alex]

Hi,
I recently had an account activation email blocked due to AC_FROM_MANY_DOTS
in the From address:

From: VitalSource 

It also hit KAM_SENDGRID and BAYES_50 and KAM_MARKETINGBL_PCCC, pushing it
over to spam.
 *  1.5 KAM_SENDGRID Sendgrid being exploited by scammers
 *  0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
 *  0.2 KAM_MARKETINGBL_PCCC Message contains URI associated with

in addition to a few smaller rules, like KAM_DMARC_NONE.

Does it sound reasonable to add 3 points plus another 1.5 simply for having
been sent by sendgrid? How do we offset those points? Do we just rely on
bayes/txrep?

I think my bayes db is pretty well-trained, but there's also a lot of
account activation fraud emails.

Re: sorbs :/

[Alex]

> https://www.irccloud.com/pastebin/XPl5OZ0y/sorbs.pl
>
> lets just test more dns fails, please fix qname, reduce zones that ends
> in same nameserver ip
>

Yes, seeing that here, too, for months and months.

Spamhaus also sucks real bad.
06-Oct-2023 13:57:12.880 resolver: loop detected resolving '
musashi.spamhaus.net/A'
06-Oct-2023 13:57:12.880 resolver: loop detected resolving '
hideyoshi.spamhaus.net/A'

These are also ultimately qname security problems they simply won't even
acknowledge, let alone fix.
26-Sep-2023 21:04:49.571 lame-servers: FORMERR resolving '
mykey.hbl.dq.spamhaus.net/NS/IN': 82.117.252.122#53
26-Sep-2023 21:04:49.598 resolver: DNS format error from 209.239.115.2#53
resolving mykey.hbl.dq.spamhaus.net/NS for : reply has no answer

Also seeing this?
07-Oct-2023 11:49:03.789 resolver: loop detected resolving 'ns3.pccc.com/A'

And barracuda using an IP registered with them:
05-Oct-2023 11:10:11.868 query-errors: client @0x7f3234752b68
127.0.0.1#55912 (20.0.135.147.bb.barracudacentral.org): query failed (timed
out) for 20.0.135.147.bb.barracudacentral.org/IN/A at
../../../lib/ns/query.c:7824

DMARC and SA4

[Alex]

Hi,
All the way back in 2016, RW posted these rules on pastebin for DMARC,
before it was part of SA proper:
https://pastebin.com/gr41CvCc

Is this effectively what's been implemented in functions in the latest SA?
The scores from the above are a lot more aggressive than what's currently
in SA 50_rules - if DMARC fails and it instructs to quarantine, isn't that
what it should do, and not just add on a few points?

score DMARC_REJECT 0.001 1.797 0.001 1.797 # n=0 n=2
score DMARC_QUAR 0.001 1.198 0.001 1.198 # n=0 n=2
score DMARC_NONE 0.001 0.898 0.001 0.898 # n=0 n=2

This became an issue for me when I received an email from ny.frb.org.
Because the email hit BAYES_00, the DMARC rule only added 0.1 points. It
also appeared that the email passed SPF, so I'm really not sure how it even
failed DMARC.

X-Envelope-From: >
...
X-Spam-Status: Yes, score=8.613 tag=-200 tag2=5 kill=5 tests=[BAYES_00=-1.9,
 DMARC_FAIL_REJECT=5.5, DMARC_REJECT=0.1, DMARC_REJ_NO_DKIM=1,
 FORGED_SPF_HELO=1, KAM_DMARC_REJECT=1, KAM_DMARC_STATUS=0.01,
 KAM_LAZY_DOMAIN_SECURITY=1, RELAYCOUNTRY_US=0.01, SPF_HELO_PASS=-0.001,
 TXREP=0.874, T_DMARC_POLICY_REJECT=0.01, T_DMARC_TESTS_FAIL=0.01]
 autolearn=disabled
X-Spam-Report:
 * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record
 *  0.0 T_DMARC_POLICY_REJECT No description available.
 *  1.0 DMARC_REJ_NO_DKIM MARC policy is reject without any DKIM signatures
 *  0.0 KAM_DMARC_STATUS Test Rule for DKIM or SPF Failure with Strict
 *  Alignment
 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
 *  [score: 0.]
 *  0.0 RELAYCOUNTRY_US Relayed through United States
 *  1.0 KAM_LAZY_DOMAIN_SECURITY Sending domain does not have any
 *  anti-forgery methods
 *  1.0 FORGED_SPF_HELO No description available.
 *  5.5 DMARC_FAIL_REJECT DMARC validation failed and policy is to reject
 *  0.0 T_DMARC_TESTS_FAIL No description available.
 *  1.0 KAM_DMARC_REJECT DKIM has Failed or SPF has failed on the message
 *  and the domain has a DMARC reject policy
 *  0.1 DMARC_REJECT DMARC reject policy
 *  0.9 TXREP TXREP: Score normalizing based on sender's reputation
...
X-Spam-RelaysUntrusted: [ ip=199.30.234.79 rdns=spfdal-b.zixsmbhosted.com

The 199.30.234.79 IP is in the SPF record:
$ dig txt ny.frb.org|grep v=spf1
ny.frb.org. 3593IN  TXT "v=spf1 ip4:199.169.200.4
ip4:199.169.204.4 ip4:199.169.240.69 ip4:199.169.208.69 ip4:199.169.174.2
ip4:170.209.35.2 ip4:199.30.234.56/29 ip4:74.203.184.208/30 ip4:
199.30.234.64/26 ip4:199.30.234.192/27 ip4:74.203.184.32/27 ip4:
68.142.184.144/28 ip4:68.142" ".185.0/25 ip4:209.190.248.144/28
ip4:199.169.200.5 ip4:152.70.150.118 ip4:129.213.11.79 exists:%{i}.
spf.frb.iphmx.com include:_spf.qualtrics.com include:service.govdelivery.com
include:amazonses.com ~all"

There seems to be a lot wrong here.  I'd appreciate some pointers on what's
going on. Of course I realize it's my choice to add the other DMARC rules
and scores on top of the default, but the default scores don't make sense
to me.

Re: uninitialized value $result in string eq at AuthRes.pm line 302

[Alex]

Hi,

>
> Aug 19 23:02:27 xavier amavis[3615]: (03615-10) _WARN: Use of
> uninitialized value $result in string eq at
> /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/AuthRes.pm line 302.
>
>292  sub check_authres_result {
>293my ($self, $pms, $method, $wanted_result) = @_;
>294
>295my $result = $pms->{authres_result}->{$method};
>296$wanted_result = lc($wanted_result);
>297
>298if ($wanted_result eq 'missing') {
>299  return !defined($result) ? 1 : 0;
>300}
>301
>302return ($wanted_result eq $result);
>303  }
>
> Perhaps there's an interim fix?
>
>
> Yes. Fixed in trunk by r1907983:
>
>
> http://svn.apache.org/viewvc/spamassassin/trunk/lib/Mail/SpamAssassin/Plugin/AuthRes.pm?r1=1907938=1907937=1907938
>
Thank you - patch applied. Can you also help me with this one now?

Aug 20 18:46:57 xavier amavis[928452]: _WARN: deprecated method; size() is
an alias of "UDPsize()" at
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/DnsResolver.pm line 602.

   594if ($packet) {
   595  # RD flag needs to be set explicitly since Net::DNS 1.01, Bug
7223
   596  $packet->header->rd(1);
   597
   598  # my $udp_payload_size = $self->{res}->udppacketsize;
   599  my $udp_payload_size = $self->{conf}->{dns_options}->{edns};
   600  if ($udp_payload_size && $udp_payload_size > 512) {
   601# dbg("dns: adding EDNS ext, UDP payload size %d",
$udp_payload_size);
   602$packet->edns->size($udp_payload_size);
   603  }
   604}

I plan to rebuild locally with the latest, but need to fix a bunch of
dependencies here first.

uninitialized value $result in string eq at AuthRes.pm line 302

[Alex]

Hi,

Just upgraded to fedora38, using the spamassassin included with it and have
the following warning:

Aug 19 23:02:27 xavier amavis[3615]: (03615-10) _WARN: Use of uninitialized
value $result in string eq at
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/AuthRes.pm line 302.

   292  sub check_authres_result {
   293my ($self, $pms, $method, $wanted_result) = @_;
   294
   295my $result = $pms->{authres_result}->{$method};
   296$wanted_result = lc($wanted_result);
   297
   298if ($wanted_result eq 'missing') {
   299  return !defined($result) ? 1 : 0;
   300}
   301
   302return ($wanted_result eq $result);
   303  }

Perhaps there's an interim fix?

unsubscore down?

[Alex]

Hi all, anyone else having problems with unsubscore?

Aug  9 15:57:41 polaris postfix-126/dnsblog[3671494]: warning:
dnsblog_query: lookup error for  DNS query 154.51.76.80.ubl.unsubscore.com:
Host or domain name not found. Name service error for name=
154.51.76.80.ubl.unsubscore.com type=A: Host not found, try again

09-Aug-2023 15:57:40.354 query-errors: client @0x7f743d494768
127.0.0.1#59605 (154.51.76.80.ubl.unsubscore.com): query failed (operation
canceled) for 154.51.76.80.ubl.unsubscore.com/IN/A at
../../../lib/ns/query.c:7824

Aug  9 16:04:14 xavier amavis[1666493]: (1666493-15) SA info: async:
aborting after 2.005 s, deadline shrunk: DNSBL, A/
41.146.71.216.ubl.unsubscore.com, rules: __RCVD_IN_LASHBACK,
RCVD_IN_LASHBACK_LASTEXT

URL Time-of-Click Protection

[Alex]

Hi all,

I'm curious what people think of URL rewriting or otherwise having some
kind of idea of whether a URL could or should be scanned at some later time
to determine if it's potentially malicious at the current time where it may
not have been initially?

Is anyone implementing that in open source?

What are the disadvantages of doing this? I'm not talking about actually
checking the URL in advance, but I suppose some kind of wrapper that scans
it at the time the user visits.

Re: AuthRes plugin test rules

[Alex]

Hi,

I'm trying to use it with amavis but there's a warning/error:

Mar 18 09:30:12 iceman amavis[2970427]: (2970427-10) _WARN: Use of
uninitialized value $result in string eq at
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/AuthRes.pm line
302.

Mar 18 09:31:50.577 [2987252] dbg: plugin: loading
Mail::SpamAssassin::Plugin::AuthRes from @INC

This is from SA 4.0.0:

  298if ($wanted_result eq 'missing') {
   299  return !defined($result) ? 1 : 0;
   300}
   301
   302return ($wanted_result eq $result);
   303  }
   304
   305  sub parsed_metadata {
   306my ($self, $opts) = @_;
   307

Any idea how to troubleshoot this?

Thanks,
Alex

On Sun, Mar 12, 2023 at 11:41 AM Matus UHLAR - fantomas 
wrote:

> >>>Matus UHLAR - fantomas skrev den 2023-03-12 10:15:
> >>>>I have also commited patch to bug 6918 to handle "arc.chain="
> >>>>results.
> >>>>Let's see how these will go.
>
> >>On 12.03.23 14:20, Benny Pedersen wrote:
> >>>miss ARC rules imho
>
> >Matus UHLAR - fantomas skrev den 2023-03-12 14:38:
> >>Or, so you mean something else than my patch?
>
> On 12.03.23 15:34, Benny Pedersen wrote:
> >your posted rules have arc testing, but it miss testing for untrusted
> >/ trusted authserv-id's
>
> in such case it would be great to remove what you are NOT commenting about
> and keep what your comments are related to, not vice versa.
>
> rules I posted use only what AuthRes plugin found.
>
> The plugin has options which headers to handle (internal/trusted/all, the
> default is "internal"), and trusted authentication servers (default: none)
> - you must configure at least one server.
>
> So the trust is processes out of rules (correct approach imho).
>
> I set SA only to trust authentication server on my machine and I'm
> watching
> the results.
>
> --
> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> He who laughs last thinks slowest.
>

SHORT_WORD_LINES & KAM_LINEPADDING

[Alex]

Hi,

I'm curious about the SHORT_WORD_LINES, KAM_LINEPADDING and HK_RANDOM
rules. I received a legitimate email from a gmail sender that was pushed
beyond 5.0 because of these rules. It hit both SCC_5_SHORT_WORD_LINES and
SCC_10_SHORT_WORD_LINES, and because a score isn't explicitly set, the two
rules added 2.0 points to the score.

describe SCC_5_SHORT_WORD_LINES 5 lines with many short words
meta SCC_5_SHORT_WORD_LINES __SCC_SHORT_WORDS >= 5
describe SCC_10_SHORT_WORD_LINES 10 lines with many short words
meta SCC_10_SHORT_WORD_LINES__SCC_SHORT_WORDS >= 10
describe SCC_20_SHORT_WORD_LINES 20 lines with many short words
meta SCC_20_SHORT_WORD_LINES__SCC_SHORT_WORDS >= 20
describe SCC_35_SHORT_WORD_LINES 35 lines with many short words
meta SCC_35_SHORT_WORD_LINES__SCC_SHORT_WORDS >= 35

KAM_LINEPADDING was hit because it was a longer email chain that involved
many ">" line characters.

rawbody  __KAM_LINEPADDING /(\n[^\n]){8}/
meta KAM_LINEPADDING (__KAM_LINEPADDING >= 1)
scoreKAM_LINEPADDING 1.2
describe KAM_LINEPADDING Spam that tries to get past blank line filters

 1.0 HK_RANDOM_FROM From username looks random
 1.0 HK_RANDOM_ENVFROM  Envelope sender username looks random

The envelope-from and From address were both the same (
killercopywriting...@gmail.com), so because they "look random" another 2.0
points were added.

Add to that the IP Gmail used to send it had a relatively poor sender score:
 0.7 RCVD_IN_SENDERSCORE_70_79 RBL: Senderscore.org score of 70 to 79
[209.85.208.54 listed in score.senderscore.com]

It also hit BAYES_50, which pushed it beyond 5.0.

Of course I could welcomelist the sender, train bayes or manually reduce
the scores of these rules, but they stood out to me as something that's
worth consideration. Should they be reevaluated?

Re: ExtractText tuning

[Alex]

Hi,

I have successfully set up ExtractText plugin with proposed settings (those
> in pod/manual page) and here's a tip:
>
> - put extracttext.pm into /etc/spamassassin or similar directory
>(extracttest settings aren't loaded from user_prefs)
>
> - tesseract takes too much time to process (at least on my server),
>so I recommend to set:
>
> extracttext_timeout 20  60
>

Have you noticed an increase in false positives due to legitimate "invoice"
PDFs or other attachments being processed by body filters and getting
tagged incorrectly?

Re: BAYES_00 BODY. Negative score?

[Alex]

Hi,

>
> However, many of tokens in even Forbes and WP newsletters may occure in
> different spamy newsletters, so be careful when traning even these.
>

This is exactly what I was thinking. When going through the quarantine,
it's also very difficult to always not only identify which newsletters may
have been miscategorized or trained incorrectly, but also ever being able
to correct an improperly trained newsletter (or email in general).


> If you get the score down enough not to be classified as spam, you've won
> and should not contine (unless you are willing to check all BAYES_0 mail
> for
> suspicious newsletters and train those as spam, seeing how much it affects
> mentioned Forbes and WP newsletters.
>

Too bad it wasn't possible to build a shared list of trusted
newsletters/senders to compensate for these mistakes.

On a related note, how about emails with only an image attachment? People
use email to send pictures, screenshots and other emails with nothing in
the body and sometimes even no subject, but aren't spam. The ones I see in
the quarantine are almost always ham, and despite training them as ham
(even with --max-size 0), they continue to be tagged as spam.

I've always also had difficulty with marking them so DCC ignores them.

Re: BAYES_00 BODY. Negative score?

[Alex]

Hi,

>*-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
> >*  [score: 0.]
>
> This indicates a mistrained database, which means you have trained too
> many
> spams or spam-like messages (commercial messages) as ham.
>
> Proper training of spams should help. Just keep your spam (and optionally
> ham) corpora for retraining in case you would drop the database.
>
> I also recommend to abstain from training commercial mail (notices from
> e-shops, companies you done business with etc) as ham, unless they
> generate
> BAYES_999 score and you want it lower.  I often train them as spam so
> those
> give uncertain BAYES_50 result.
>

Is there any ability to distinguish a legitimate newsletter from a spam
newsletter?

In other words, if I train emails from Forbes or Washington Post as ham,
then train similar newsletter emails from other other providers that are
more suspect, will bayes still be able to distinguish Forbes and WP as ham?

The problem is that if I avoid training newsletters or bulk email
altogether, then I'm also left with spam newsletters still only hitting
bayes50.

I'm actually in a situation now where Forbes and WP newsletters are being
marked as spam, so considering retraining, but wondering what approach/best
practices I should be following.

 # sa-learn --dump magic
0.000  0  3  0  non-token data: bayes db version
0.000  0  97002  0  non-token data: nspam
0.000  0  90173  0  non-token data: nham
0.000  0   11581565  0  non-token data: ntokens
0.000  0 1054224948  0  non-token data: oldest atime
0.000  0 1676433889  0  non-token data: newest atime
0.000  0  0  0  non-token data: last journal sync
atime
0.000  0 1648164856  0  non-token data: last expiry atime
0.000  0  0  0  non-token data: last expire atime
delta
0.000  0  0  0  non-token data: last expire
reduction count

FROM_GOV_SPOOF and Zix SPF softfail?

[Alex]

Hi,

I received an email from ncua.gov sent through Zix that apparently was an
SPF softfail. It also hit FROM_GOV_SPOOF. I wanted to see if the two were
related, or what the reason was for this email hitting so many spam rules.

meta FROM_GOV_SPOOF  !__NOT_SPOOFED && __FROM_ADDRLIST_GOV && (!
NO_RELAYS && ! ALL_TRUSTED)
tflags   FROM_GOV_SPOOF  net publish
describe FROM_GOV_SPOOF  From Government domain but matches SPOOFED

Why is there a SPF softfail with Zix? Certainly it's possible there just so
happened to be a DNS problem at that time, but just wanted to be sure
something else wasn't happening - I don't want to wait until an email is
rejected from this sender before doing something about it.

X-Spam-Status: No, score=3.449 tagged_above=-200 required=5
 tests=[BAYES_05=-0.5, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, DMARC_NONE=0.1,
 FORGED_SPF_HELO=1, FROM_GOV_SPOOF=1, HTML_FONT_LOW_CONTRAST=0.001,
 HTML_MESSAGE=0.001, KAM_DMARC_NONE=0.25, KAM_DMARC_STATUS=0.01,
 KAM_EVIL_NUMBERS4=1, KAM_LOTSOFHASH=0.25, LOC_CDIS_INLINE=0.1,
 RCVD_IN_DNSWL_LOW=-0.7, RELAYCOUNTRY_LOW=0.1, RELAYCOUNTRY_US=0.01,
 SPF_HELO_PASS=-0.001, SPF_SOFTFAIL=0.665, TXREP=-0.177,
 T_KAM_HTML_FONT_INVALID=0.01] autolearn=disabled

https://pastebin.com/8sSqYh9u

Re: sharepoint phish routed through sharepointonline/outlook

[Alex]

Hi,

> RBL checks for FQDN not just domains would be a good idea...
>
...

>
> I assume you are not running SA4. That does this. (And the sharepoint
> domain you have in your mail is listed on SURBL )
>

Yes, I am running SA4 and have been for probably more than a year. What am
I doing wrong that RBL checks wouldn't be checking the FQDN?

uniabujaedung-my[.]sharepoint[.]com[.]multi[.]surbl[.]org
> has address 127.0.0.64
>
> Meaning its lised in ABUSE.
>

I suspect then that I received it prior to it being listed there. Any way
to correlate those dates (if it's even worth it)?

Thanks! Raymond
>

Thank you :-)

sharepoint phish routed through sharepointonline/outlook

[Alex]

Hi,

X-Spam-Status: No, score=1.102 tagged_above=-200 required=5
 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
 DKIM_VALID_EF=-0.1, DMARC_PASS=-0.1, FMBLA_HELO_OUTMX=-0.01,
 FMBLA_RDNS_OUTMX=-0.01, HTML_MESSAGE=0.001, LOC_CDIS_INLINE=0.1,
 LOC_FILE_SHARE_PHISH1=0.75, LOC_FROMADDR=0.01, LOC_FROMNAME=0.01,
 LOC_IMGSPAM=0.1, LOC_XORIGORG=0.01, MIME_HTML_ONLY=0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001,
 RCVD_IN_SENDERSCORE_80_89=-0.4, RELAYCOUNTRY_LOW=0.1, RELAYCOUNTRY_US=0.01,
 SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, TXREP=-0.166] autolearn=disabled

I'm reporting it to spamcop and training bayes, but does anyone have any
other ideas?

Is this just someone using their sharepoint account to send a phish?
Perhaps account takeover?

https://pastebin.com/2CJ3SLf2

Re: welcomelist_auth and SPF

[Alex]

Hi,

On Fri, Dec 16, 2022 at 5:35 PM Marc  wrote:

> > The sender's SPF record includes the sending IP (40.107.96.128) in the
> > secureserver.net   entry, and SPF_PASS is hit.
> >
>
> Without even checking anything I can already remember that this
> secureserver.net is shit. I have blocked whole ranges of them, they send
> spam, try passwords etc. I have the impression that there is nothing secure
> about secureserver and everything seems to be hacked there.
>
> You will always have false positives, and probably even more in the
> future, there is going to be more and more networks trying to mix spam with
> legitimate email.
> For this you have to create some way to unmark / whitelist email addresses.
>

Yes, GoDaddy is shit, but should that mean there's no expectation of being
able to add it to a trusted senders list for individual senders?

I'm now more curious why it says SPF_PASSed, yet my welcomelist entry
didn't work to keep it from being marked as spam.

Whether or not it's listed on the valli blocklists should also be
irrelevant - that GoDaddy is shit is the exact reason why I'm trying to add
this (unsuccessfully) to the welcomelist.

welcomelist_auth and SPF

[Alex]

Hi,

This GoDaddy/M365 quarantined email passes SPF, but despite now adding it
to my welcomelist, it is still marked as spam.

https://pastebin.com/VpPmgGN4

Only when I create a welcomelist_from_rcvd does it get delivered.

The sender's SPF record includes the sending IP (40.107.96.128) in the
secureserver.net entry, and SPF_PASS is hit.

-0.0 SPF_HELO_PASS  SPF: HELO matches SPF record
-0.0 SPF_PASS   SPF: sender matches SPF record

There's also a FP on KAM_ZWNJ, or at the least is not a malicious email
intended to elude anything.

Can someone help me understand what's happening here?

RBL timeouts

[Alex]

Hi,

Is anyone (everyone?) also experiencing DNS timeouts with barracuda?

02-Dec-2022 07:03:02.229 query-errors: client @0x7fd19d26c968
127.0.0.1#37098 (168.22.111.13.bb.barracudacentral.org): query failed
(timed out) for 168.22.111.13.bb.barracudacentral.org/IN/A at
../../../lib/ns/query.c:7729
02-Dec-2022 07:03:21.458 lame-servers: SERVFAIL unexpected RCODE resolving '
216.209.245.104.bb.barracudacentral.org/A/IN': 3.13.7.254#53

I'm also seeing a few timeouts from mcafee:

24-Nov-2022 16:12:37.151 query-errors: client @0x7fd19f7a4f68
127.0.0.1#47466 (17.31.10.37.cidr.bl.mcafee.com): query failed (timed out)
for 17.31.10.37.cidr.bl.mcafee.com/IN/A at ../../../lib/ns/query.c:7729

I don't necessarily think there's something wrong with my nameservers - I'm
more just surprised that such high-profile companies are having problems
and wanted to confirm.

Any bind experts know of a way to record which nameserver is timing out so
I can perhaps exclude them? Any idea why it wouldn't just rotate to the
next one, or even how to confirm whether it's doing that?

Re: Mial hits MISSING rules despite presence of headers

[Alex]

On Mon, Nov 28, 2022 at 10:42 AM Kevin A. McGrail 
wrote:

> What's the score on that short circuit Validity rule?
>

-2.0 RCVD_IN_VALIDITY_SAFE  RBL: Sender in Validity Safe - Contact
certificat...@validity.com
[Return Path SenderScore Safe List (formerly]
[Habeas Safelist) - ]
-0.0 SHORTCIRCUIT   Not all rules were run, due to a shortcircuited
rule

So despite saying it's shortcircuiting rules, it still erroneously adds
enough points for it to be marked as spam when without the rule it wouldn't
have been marked as such.

I think the expectation is that it's a -100 type rule but I could be
> wrong.  Did you confirm with -D that the behavior is as you describe and
> more rules kept running after the short circuit?  I don't use the short
> circuit.
>
> Also, would be helpful to know if this is different than 3.4.6's behavior.
>

Oh yes, I meant to mention that it is different behavior for 3.4.6. Same
score for the rule, but it appears to actually shortcircuits the processing
of additional rules. At the least, it doesn't add those MISSING_* rules.

Re: Mial hits MISSING rules despite presence of headers

[Alex]

Hi,

> Well, a short circuit rule kind of breaks things in the middle so I do not
> think you should really spend too much time on rules that hit/didn't hit.
>
> I like validity but I don't think it justifies a short circuit, FYI.
>

Okay, it's been removed, but somehow the presence of that didn't have the
effect of bypassing any further checks, but actually causing it to be
classified as spam and was quarantined.

Re: Mial hits MISSING rules despite presence of headers

[Alex]

Hi,

> I have emails from wayfair and Dell that hit many of the MISSING_*
>> > rules
>> > but these headers are clearly displayed.
>> >
>> >  *  0.5 MISSING_MID Missing Message-Id: header
>> >  *  1.0 MISSING_FROM Missing From: header
>> >  *  1.8 MISSING_SUBJECT Missing Subject: header
>> >  *  1.4 MISSING_DATE Missing Date: header
>> >  *  2.3 EMPTY_MESSAGE Message appears to have no textual parts and no
>> >  *  Subject: text
>> >
>> > This also consequently causes DMARC/DKIM to fail.
>> >
>> > https://pastebin.com/yFCRx76x
>> >
>> > $ spamassassin --version
>> > SpamAssassin version 4.0.0-r1904221
>> >   running on Perl version 5.36.0
>>
>> Cannot reproduce. Pasting a copy of that from the 'raw' view and feeding
>> it to 'spamassassin  -t' doesn't result in hits on any of those rules.
>>
>> How are you calling SA?
>>
>> I have a theory about what might be happening, but it would require
>> using report_safe=1 and a flow that passes twice through SA...
>>
>
> I'm calling SA through amavis, but it happens even when running SA from
> the command-line:
>
> $ spamassassin -t < email.eml
>
> I do actually notice it does print the rules that are triggered twice, but
> I don't think the scores are duplicated.
>
> report_safe=1 is set in 10_defaults.pref in the updates.spamassassin.org
> ruleset.
>

It has something to do with this shortcircuit rule I added to my local.cf
some time ago:

shortcircuit RCVD_IN_VALIDITY_SAFE on

Commenting this out results in normal operation. Any idea how that could
possibly happen?!

Re: Mial hits MISSING rules despite presence of headers

[Alex]

Hi,

> I have emails from wayfair and Dell that hit many of the MISSING_*
> > rules
> > but these headers are clearly displayed.
> >
> >  *  0.5 MISSING_MID Missing Message-Id: header
> >  *  1.0 MISSING_FROM Missing From: header
> >  *  1.8 MISSING_SUBJECT Missing Subject: header
> >  *  1.4 MISSING_DATE Missing Date: header
> >  *  2.3 EMPTY_MESSAGE Message appears to have no textual parts and no
> >  *  Subject: text
> >
> > This also consequently causes DMARC/DKIM to fail.
> >
> > https://pastebin.com/yFCRx76x
> >
> > $ spamassassin --version
> > SpamAssassin version 4.0.0-r1904221
> >   running on Perl version 5.36.0
>
> Cannot reproduce. Pasting a copy of that from the 'raw' view and feeding
> it to 'spamassassin  -t' doesn't result in hits on any of those rules.
>
> How are you calling SA?
>
> I have a theory about what might be happening, but it would require
> using report_safe=1 and a flow that passes twice through SA...
>

I'm calling SA through amavis, but it happens even when running SA from the
command-line:

$ spamassassin -t < email.eml

I do actually notice it does print the rules that are triggered twice, but
I don't think the scores are duplicated.

report_safe=1 is set in 10_defaults.pref in the updates.spamassassin.org
ruleset.







>
> --
> Bill Cole
> b...@scconsult.com or billc...@apache.org
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
> Not Currently Available For Hire
>

Mial hits MISSING rules despite presence of headers

[Alex]

Hi,
I have emails from wayfair and Dell that hit many of the MISSING_* rules
but these headers are clearly displayed.

 *  0.5 MISSING_MID Missing Message-Id: header
 *  1.0 MISSING_FROM Missing From: header
 *  1.8 MISSING_SUBJECT Missing Subject: header
 *  1.4 MISSING_DATE Missing Date: header
 *  2.3 EMPTY_MESSAGE Message appears to have no textual parts and no
 *  Subject: text

This also consequently causes DMARC/DKIM to fail.

https://pastebin.com/yFCRx76x

$ spamassassin --version
SpamAssassin version 4.0.0-r1904221
  running on Perl version 5.36.0

Re: pyzor and failure to parse response

[Alex]

On Sun, Nov 20, 2022 at 12:54 PM Henrik K  wrote:

> On Sun, Nov 20, 2022 at 11:58:31AM -0500, Alex wrote:
> > Hi,
> > I'm using the latest SA from trunk and trying to get pyzor working. It
> runs
> > correctly to check a message from the command-line, but SA apparently
> fails to
> > properly parse the output?
> >
> > Nov 20 11:55:21.970 [2531521] dbg: pyzor: opening pipe: /usr/bin/pyzor
> > --homedir /var/spool/amavisd/.pyzor --debug --log-file /var/spool/amavisd
> > /.pyzor/pyzor.log check 
> Your --debug parameter is messing up the output, remove it from
> pyzor_options.
>

Ah yes, of course, thanks.

pyzor and failure to parse response

[Alex]

Hi,
I'm using the latest SA from trunk and trying to get pyzor working. It runs
correctly to check a message from the command-line, but SA apparently fails
to properly parse the output?

Nov 20 11:55:13.213 [2531397] dbg: pyzor: network tests on, attempting Pyzor
Nov 20 11:55:15.756 [2531397] dbg: pyzor: adjusting rule PYZOR_CHECK
priority to -100
Nov 20 11:55:21.963 [2531397] dbg: pyzor: pyzor is available: /usr/bin/pyzor
Nov 20 11:55:21.968 [2531521] dbg: pyzor: child process 2531521 forked
Nov 20 11:55:21.970 [2531521] dbg: pyzor: opening pipe: /usr/bin/pyzor
--homedir /var/spool/amavisd/.pyzor --debug --log-file
/var/spool/amavisd/.pyzor/pyzor.log check

Re: FMBLA_NDBLOCKED and DKIMWL_BLOCKED

[Alex]

Hi,


> Boring Stuff
> We have some restrictions on the usage of our data. You can read all
> about it here.
>

Yeah, turns out not so much. I'm working with Paul directly, thanks,

FMBLA_NDBLOCKED and DKIMWL_BLOCKED

[Alex]

Hi,

I just noticed I've apparently hit the regular limits of use for fmbla and
dkimwl for my few domains and honeypots. I believe this is a service
provided by Paul Stead - does anyone know if there's a "pro" version or how
I might be able increase the permissible capacity allowed?

Given it's integrated into SA now, it would be nice to be able to benefit
from it. There's nothing on the fmb.la website to indicate how I might be
able to do that.

I'm using a personal resolver, not a public DNS server.

Re: PBL and rejects

[Alex]

Hi,

>
> >These aren't new netblocks for us from them, but it seems awfully weird
> >that we would be operating on these IPs for 2+ years then all of the
> sudden
> >have them listed like they're dialup IPs.
>
> generic/dialup DNS names can help here. If they aren't dynamically
> allocated, their DNS records should contain at least string "static".
>
> It's generally advised to use personalized DNS names (reverse and direct)
> for mailservers or any hosts supposed to send e-mail.
>
> Note that if you send mail using authentication, you usually don't need
> this.
>

These are bare metal servers dedicated to us in their datacenter. They
control reverse DNS, but we have given them the hostnames we have
explicitly defined for this, and to match our forward DNS.


> >I don't know if that's just a boilerplate message or it actually refers to
> >the precise reason why my IPs were added to the PBL.
>
> yes, they are explicitly telling you to use mailserver outside of this
> range.
> If you have own mailserver, you should dedicate IP address for it, one
> that
> won't be added to PBL by your ISP.
>

Our DNS entries and the way we operate our mail server (primarily relaying
scanned mail to M365 systems) hasn't changed in the two years we've been
with this provider.

So you believe these IPs would have been added to the PBL by our ISP? Do
you know how that might have been done?

I'm trying to understand why or how we would have been added to the PBL
when nothing was changed. It was the entirety of our netblocks, not even
just the IPs we're currently using.

Thanks so much for your help.

Re: PBL and rejects

[Alex]

Hi,

>
> > I'm hoping I can ask this question here. Somehow the PBL considered the
> IP
> > addresses given to us by our ISP (I can share this if needed) as
> ineligible
> > to send email, resulting in any recipient domain that checks the PBL to
> > reject our email,
>
> AIUI, PBL is supposed to be for dynamic-type IP addresses for
> residential service, so if you have business service something seems
> off.
>
> What did your ISP say when you asked them about this?   I would expect
> them to be concerned because giving customers addresses in RBL is
> obviously going to get them sorted into giving not-really-ok service and
> negative recommendations, if that's what is really going on.
>

They denied any knowledge of three /29s being listed or having any
involvement in it happening.

They said they have a spamhaus license, which I'm assuming is for their own
servers, and that they would leverage that to ask a support question, but
they're disclaiming any responsibility.

These aren't new netblocks for us from them, but it seems awfully weird
that we would be operating on these IPs for 2+ years then all of the sudden
have them listed like they're dialup IPs.

The message I received during the delisting process with spamhaus/PBL for
"MyProvider" was:

Outbound Email policy of MyProvider LLC for this IP range
It is the policy of MyProvider LLC that unauthenticated email sent from
this IP address should be sent out only via the designated outbound mail
server allocated to MyProvider LLC customers. To find the hostname of the
correct mail server to use, customers should consult the original signup
documentation or contact MyProvider LLC Technical Support.

I don't know if that's just a boilerplate message or it actually refers to
the precise reason why my IPs were added to the PBL.

PBL and rejects

[Alex]

Hi,

I'm hoping I can ask this question here. Somehow the PBL considered the IP
addresses given to us by our ISP (I can share this if needed) as ineligible
to send email, resulting in any recipient domain that checks the PBL to
reject our email, including every email sent to a Microsoft 365 domain.
This is also despite having a rule to bypass spam filtering on the M365
side with our own M365 domain - apparently that is not bypassed?

Does anyone know how this might happen? Would my ISP have listed them
intentionally? I've now delisted all of our IPs successfully, and mail is
again flowing, but it obviously resulted in a pretty significant impact on
our delivery.

I'm also trying to confirm I've configured my system properly to best
utilize RBLs.

Any ideas greatly appreciated.

Re: Gmail confidential mode

[Alex]

>
>
> > What do you know about "Gmail confidential mode" emails? I'm starting to
> > see a few of these come in to users now, and not sure how to treat them.
> > They are sent through gmail, but require a one-time passcode sent to the
> > recipient,
>
> Did you actually look at them?  What do they look like?  What does the
> recipient have to do to actually get the mail?  Does this only work
> gmail to gmail?
>

Some of those questions I was hoping others could help me to answer. This
is a legitimate email service provided by gmail. It was routed through
google's servers only. It passed DKIM and SPF, but not DMARC. I don't think
it's only gmail-to-gmail, as the recipient is not a gmail account.

You can experiment with this by composing a new message in Gmail, then
clicking the "toggle confidential mode" lock/timer icon in the same tray as
where fonts and attachments are controlled.

The email includes a link to "view the email" where the user is then
directed to https://confidential-mail.google.com/ with a prompt to get a
one-time passcode to the same email address that apparently authorizes the
recipient to reveal the contents of the "secure" email. I didn't "send
passcode" on that URL because it would then send it to the real recipient
as well. It requires the passcode only if it's necessary to authenticate as
the recipient - if you're not already logged in as that recipient, for
example.

It's definitely suspect, as the subject is just "Fwd: Information" and
there are no details in the body as to its contents. The email is base64
encoded.

> so any potential threat is not transferred through the same
> > email (or any email at all).
>
> huh?  I don't follow this at all.
>

Once you've authenticated yourself, the email is displayed there, at the
confidential-mail.google.com URL directly, not through some follow-up email.

> otherwise have no other spam indicators.
>
> When you looked at the raw bytes in the mailspool, what was in it?  What
> does the SA debug output look like?  It doesn't make sense that wouldn't
> have done these things before posting, but you didn't explain.
>

Yes, the initial email is relatively benign - it is a legitimate gmail
email sent through their servers and signed by them.

The spample I'm looking at now was quarantined only because their domain (
pcfixpos.com) is apparently blocklisted.  It also hit BAYES_99.

 *  1.0 DKIMWL_BULKMAILER_LOW ASKDNS: DKIMwl.org - Low scoring bulkmailer
 *  [pcfixpos-com.20210112.gappssmtp.com.lookup.dkimwl.org A:127.0.2.1]
 *  1.5 DKIMWL_BL ASKDNS: DKIMwl.org - Low trust sender
 *  [pcfixpos-com.20210112.gappssmtp.com.lookup.dkimwl.org A:127.0.2.1]

Given that, I suspect this one is spam, but this is an interesting way to
distribute malicious links.

Gmail confidential mode

[Alex]

Hi,

What do you know about "Gmail confidential mode" emails? I'm starting to
see a few of these come in to users now, and not sure how to treat them.
They are sent through gmail, but require a one-time passcode sent to the
recipient, so any potential threat is not transferred through the same
email (or any email at all).

The messages I've received were all tagged as spam due to bayes, but they
otherwise have no other spam indicators.

This doesn't appear to be anything new, but it's the first time I'm seeing
it. Just thought I'd share and see if anyone had any input on how they're
managing them.

Re: Mail with image marked as spam

[Alex]

Hi,

>  *  1.8 MIME_IMAGE_JPG contains wrong MIME type image\\/jpg
>
> That rule is nowhere in the current standard rules or the KAM rules.
>
> If you don't like your custom local rules, only you can change them.
>

Ah, thanks. Usually my local rules are indicated as such, so I didn't even
realize it. I've disabled it for now, thanks.


>

Re: Mail with image marked as spam

[Alex]

On Sun, Sep 25, 2022 at 1:56 PM Matus UHLAR - fantomas 
wrote:

> On 25.09.22 13:35, Alex wrote:
> >I've asked variations of this question in the past, but I'm still not sure
> >what to do about it. Should an email with just an image attachment, with
> no
> >subject and no body be treated as spam? This is the circumstance where
> >users are using email as a file transfer device.
> >
> >There seems to be one irregularity with this email that causes it to be
> >marked as spam:
> >
> > *  1.8 MIME_IMAGE_JPG contains wrong MIME type image\\/jpg
>
> correct mime type is image/jpeg
>

All indications are that this message was crafted and sent by Gmail. I
don't see that an email client connecting to gmail was used.


>
> >but should that be enough? Here are the other spam indicators for this
> >message where only a 9MB attachment was included:
> >
> > *  0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
>
> you can train these, if it makes sense
>

Yes, I've been doing that, but there are apparently too many slight
variations.


> > *  0.2 KAM_BLANKSUBJECT Message has a blank Subject
> > *  0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in
> > *  0.8 MPART_ALT_DIFF BODY: HTML and text parts are different
>
> so, does the message contain something or doesn't it? looks like either
> HTML
> or text part does contain something.
>

Content-Type: text/html; charset="UTF-8"


sending empty message with empty subject really looks like spam
>

Do we have more info on what percentage of similar messages are actually
spam? It sure seems to me like people are just using email to share
pictures (licenses, legal docs, as well as pictures of the kids.)


> >It otherwise hit no local rules, passed SPF and DKIM as it went through
> >gmail, and even had TXREP deduct a point.
> >
> >Perhaps we create a meta rule that deducts points for instances where all
> >of these rules are hit, indicating it was just an image attachment?
> >
> >What are others doing here? This is with the latest SA v4 from svn.
>
> If you can advise the sender not to send blank subject/body, AND possibly
> to
> fix the mime type, your problem is over
>

There are too many variations and one-timers for this to be practical.

>
>

Mail with image marked as spam

[Alex]

Hi,

I've asked variations of this question in the past, but I'm still not sure
what to do about it. Should an email with just an image attachment, with no
subject and no body be treated as spam? This is the circumstance where
users are using email as a file transfer device.

There seems to be one irregularity with this email that causes it to be
marked as spam:

 *  1.8 MIME_IMAGE_JPG contains wrong MIME type image\\/jpg

but should that be enough? Here are the other spam indicators for this
message where only a 9MB attachment was included:

 *  0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
 *  0.2 KAM_BLANKSUBJECT Message has a blank Subject
 *  0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in
 *  0.8 MPART_ALT_DIFF BODY: HTML and text parts are different
 *  1.8 MIME_IMAGE_JPG contains wrong MIME type image\\/jpg
 *  1.2 HTML_IMAGE_ONLY_04 BODY: HTML: images with 0-400 bytes of words
 *  2.3 EMPTY_MESSAGE Message appears to have no textual parts and no
 *  Subject: text

It otherwise hit no local rules, passed SPF and DKIM as it went through
gmail, and even had TXREP deduct a point.

Perhaps we create a meta rule that deducts points for instances where all
of these rules are hit, indicating it was just an image attachment?

What are others doing here? This is with the latest SA v4 from svn.

Re: Matching on missing To field?

[Alex]

Hi,

>I have a number of rules that match on the To field, but what to do if the
> >To field is missing?
>
> 10_hasbase.cf:header__HAS_TOexists:To
>
> define:
>
> metaMISSING_TO  !__HAS_TO
>
> >Received: from test.com (wsip-72-214-24-18.sd.sd.cox.net [72.214.24.18])
> >by mail01.example.com (Postfix) with SMTP id 12425B9B
> >for ; Fri, 15 Jul 2022 18:50:34 -0400 (EDT)
> >
> >I realize I can match on the Received header here, but that would require
> >creating an additional rule for each corresponding To rule. Perhaps
> there's
> >a way to combine them, or a tag that can be used for both?
>
> the main question is what do you want to match.


The problem I'm having is that my To header rules aren't matching because
there is no To header, and I'm otherwise unsure what to match on. The only
occurrence of the recipient in the entire email is in that Received header.

My reference to the __HDRS_MISSP was to possibly use that as an example for
a new rule that would allow me to match on the j...@gooddom.com using the
same "ALL:raw" idea in some way.

It does match on "ALL", but I think I need to be more specific than that,
to avoid matching on "From:" or Return-Path or EnvelopeFrom./

Thanks,
Alex

Matching on missing To field?

[Alex]

Hi,

I have a number of rules that match on the To field, but what to do if the
To field is missing?

Received: from test.com (wsip-72-214-24-18.sd.sd.cox.net [72.214.24.18])
by mail01.example.com (Postfix) with SMTP id 12425B9B
for ; Fri, 15 Jul 2022 18:50:34 -0400 (EDT)

I realize I can match on the Received header here, but that would require
creating an additional rule for each corresponding To rule. Perhaps there's
a way to combine them, or a tag that can be used for both?

I'm also aware of using ALL, but I think that may be too broad and may
catch instances that shouldn't be. Can someone explain how this rule works
and if something similar would apply to my situation?

header __HDRS_MISSP  ALL:raw =~
/^(?:Subject|From|To|Reply-To):\S/ism

Thanks,
Alex

Re: Attachment policy

[Alex]

> Those sound like perfectly legitimate emails so working to classify them
> as decent emails would be our goal. Was there anything malicious snuck in
> there?
>

No, they were all just very basic PDF documents, mostly from gmail
accounts, but also from another dozen or different providers. Some looked
to be pasted from Google Translate from their native language to English,
and a few were in languages other than English.

Some were tagged because of the DKIM/DMARC bug, lol.

Okay, I'll train them as ham and hope it can later discern these from
malicious emails with PDF attachments.

Please keep us updated on the progress of the ExtractText plugin.

Thanks,
Alex

Attachment policy

[Alex]

Hi,
I'm looking for input from people on how they handle attachments, and
people using email as a file transfer service. One of our users must have
posted to a job site recently, soliciting resumes from people
internationally. This resulted in 100+ emails from random people who had
never emailed this user before, many of which had no subject and no body,
just a PDF attachment. Some had the "Sent by my iPhone" signatures, but
that's about it. Virtually all of them were tagged as spam due to bayes.

Any recommendations? There wasn't otherwise anything wrong with the
attachments - they were all legitimate resumes from legitimate sources.
Should they be blocked? Should I retrain bayes to not consider these spam?
I'm now training bayes with them as ham, but it will take a lot to
offset these. Same with emails that only contain images. Should an email
with only an image attachment with no subject and no body but sent from a
legitimate source and otherwise not dangerous be considered spam?

Many also hit DCC, presumably because of the empty body. Is it possible to
train DCC with one of these to be ignored that would then apply to all
similar messages? I've generated a signature of an empty email before, but
unsure how much variation is allowed before it's no longer considered the
same signature.

Somewhat related, is the ExtractText plugin useful anymore? I had to
disable it altogether because of the money rules and people emailing their
credit card statements, and even though they talk about money, it's not
malicious.

Re: DKIM fails on v4

[Alex]

Hi,

>> At some point after that, and even until yesterday's version, DKIM
> stopped
> >> working. DMARC still passes with SPF, but there are no longer any
> occurrences
> >> of DKIM.
> >
> > I think Giovannis changes don't work when amavisd is passing
> $suppl_attrib:
> >
> > https://svn.apache.org/viewvc?view=revision=1901719
> >
> > Sub _check_signature() isn't called at all in that case and things like
> tags
> > are not set.  I'll leave it for Giovanni to fix..
> >
> thanks for the hint, I've just committed a fix.
>

That looks to have fixed it, thanks. Whew. That was very tricky. Great work.

Re: DKIM fails on v4

[Alex]

>
> Amavisd-new works fine here. Maybe $enable_dkim_verification or something
> is different.
>

It's good to know you're using amavisd. It's very dependent upon the SA
version you're using, though.

It appears both DKIM and DMARC worked until the May 29th version from svn
(1901385).

At some point after that, and even until yesterday's version, DKIM stopped
working. DMARC still passes with SPF, but there are no longer any
occurrences of DKIM.

Nothing changed with amavisd.

$ grep dkim amavisd.conf
$sa_debug = 'info,dkim,DMARC,dmarc';
$enable_dkim_verification = 1;  # enable DKIM signatures verification
$enable_dkim_signing = 1;# load DKIM signing code, keys defined by
dkim_key

With the broken versions, DKIM still seems to be evaluated, but no DKIM
rules are triggered.
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: signatures
provided by the caller, 2 signatures
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: adsp:
performing lookup on _adsp._domainkey.agoda.com
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: adsp
result: U/unknown (dns: unknown), author domain 'agoda.com'
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: VALID
signature by agoda.com, author no-re...@agoda.com, no valid matches
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: VALID
signature by agoda.com, author no-re...@agoda.com, no valid matches
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: author
no-re...@agoda.com, not in any dkim welcomelist
Jun 26 12:40:09 xavier amavis[752588]: (752588-04) SA dbg: DMARC: result:
pass, disposition: none, dkim: fail, spf: pass (spf: pass, spf_helo: fail)

Here's an email from the same sender once the May 29th version was
installed. This passed both DKIM_VALID_AU and DMARC_PASS
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: signatures
provided by the caller, 2 signatures
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: VALID
DKIM, i=no-re...@agoda.com, d=agoda.com, s=keyx, a=rsa-sha1,
c=relaxed/relaxed, key_bits=2048, pass,matches author domain
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: VALID DK,
i=no-re...@agoda.com, d=agoda.com, s=keyx, a=rsa-sha1, c=nofws,
key_bits=2048, pass, matches author domain
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: signature
verification result: PASS
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: adsp not
retrieved, author domain signature is valid
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: adsp
result: - (valid a. d. signature), author domain 'agoda.com'
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: VALID
signature by agoda.com, author no-re...@agoda.com, no valid matches
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: VALID
signature by agoda.com, author no-re...@agoda.com, no valid matches
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: author
no-re...@agoda.com, not in any dkim welcomelist
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: DMARC: result:
pass, disposition: none, dkim: fail, spf: pass (spf: pass, spf_helo: fail)

I see the version of DMARC.pm is completely different from May 29th to
today. Should I try using the DMARC.pm from this month with the SA from
last month?

Re: DKIM fails on v4

[Alex]

Hi,

It's definitely a problem with the current spamassassin from github v4. I
went back to an old version I built on May 29th and it immediately starts
reporting DKIM normally again.

I just built the latest version and it's still exhibiting the same problem.
Based on my logs, it started happening on or around June 14th.

DMARC is not working with my version from May 29th. I wonder if I could
drop in the DMARC.pm that was updated at the end of June into the May 29th
version and see if now they both work?

btw, I previously mentioned github, but meant svn.apache.org.
svn checkout http://svn.apache.org/repos/asf/spamassassin/trunk
Mail-SpamAssassin-4.0.0







On Sat, Jun 25, 2022 at 3:07 PM Alex  wrote:

> Hi,
> I've been having problems with DMARC failing over the past few weeks using
> the latest SA, even on sites I know have passed. It appears to have
> coincided with an update to DMARC.pm related to timing. I just now happened
> to notice that maybe the problem is with DKIM, or there's a separate DKIM
> problem or something I simply don't understand. Installing v3.4.6 over the
> latest v4 fixes the problem instantly.
>
> It appears DKIM is loading in amavis:
> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Signer
>  1.20200907
> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Verifier
> 1.20200907
> Jun 25 00:13:09 mail03 amavis[4119158]: DKIM codeloaded
> Jun 25 00:13:18 mail03 amavis[4119158]: SpamAssassin loaded plugins: ASN,
> AskDNS, AutoLearnThreshold, BTCBL, Bayes, BodyEval, Check, ClamAV, DCC,
> DKIM, DMARC, DNSEval, DecodeShortURLs, FreeMail, FromNameSpoof, HTMLEval,
> HTTPSMismatch, HashBL, HeaderEval, ImageInfo, Levenshtein, MIMEEval,
> MIMEHeader, OLEVBMacro, PDFInfo, PhishTag, Phishing, Pyzor, Razor2,
> RecipientMsgID, RelayCountry, RelayEval, ReplaceTags, SPF, SendGrid,
> Shortcircuit, SpamCop, TextCat, TxRep, URIDNSBL, URIDetail, URIEval,
> URILocalBL, VBounce, WLBLEval, WelcomeListSubject, iXhash2
>
> Yet it never fires. The only references to DKIM in emails are
> from DKIM_ADSP_ALL. What could I be missing, or is this possibly a bug?
>
> You might also recall from my previous reports that DKIM succeeds on an
> email where it otherwise failed when running it through SA directly.
>
> $ spamassassin --version
> SpamAssassin version 4.0.0-r1901426
>   running on Perl version 5.34.1
>
> This is on fedora35. Installing the stock 3.4.6 immediately starts
> triggering DKIM hits.
>
> Is there a backport of RaciallyCharged, Esp and ExtractText (although I
> don't really use that anymore) that's available for v3.4.6, so my
> welcomelist entries work in the meantime?
>
>
>

DKIM fails on v4

[Alex]

Hi,
I've been having problems with DMARC failing over the past few weeks using
the latest SA, even on sites I know have passed. It appears to have
coincided with an update to DMARC.pm related to timing. I just now happened
to notice that maybe the problem is with DKIM, or there's a separate DKIM
problem or something I simply don't understand. Installing v3.4.6 over the
latest v4 fixes the problem instantly.

It appears DKIM is loading in amavis:
Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Signer
 1.20200907
Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Verifier
1.20200907
Jun 25 00:13:09 mail03 amavis[4119158]: DKIM codeloaded
Jun 25 00:13:18 mail03 amavis[4119158]: SpamAssassin loaded plugins: ASN,
AskDNS, AutoLearnThreshold, BTCBL, Bayes, BodyEval, Check, ClamAV, DCC,
DKIM, DMARC, DNSEval, DecodeShortURLs, FreeMail, FromNameSpoof, HTMLEval,
HTTPSMismatch, HashBL, HeaderEval, ImageInfo, Levenshtein, MIMEEval,
MIMEHeader, OLEVBMacro, PDFInfo, PhishTag, Phishing, Pyzor, Razor2,
RecipientMsgID, RelayCountry, RelayEval, ReplaceTags, SPF, SendGrid,
Shortcircuit, SpamCop, TextCat, TxRep, URIDNSBL, URIDetail, URIEval,
URILocalBL, VBounce, WLBLEval, WelcomeListSubject, iXhash2

Yet it never fires. The only references to DKIM in emails are
from DKIM_ADSP_ALL. What could I be missing, or is this possibly a bug?

You might also recall from my previous reports that DKIM succeeds on an
email where it otherwise failed when running it through SA directly.

$ spamassassin --version
SpamAssassin version 4.0.0-r1901426
  running on Perl version 5.34.1

This is on fedora35. Installing the stock 3.4.6 immediately starts
triggering DKIM hits.

Is there a backport of RaciallyCharged, Esp and ExtractText (although I
don't really use that anymore) that's available for v3.4.6, so my
welcomelist entries work in the meantime?

Re: block emails with fake FROM

[Alex]

Hi,

seems it did not catch this one:
>
> From: " Dr Perfect "@mail.gepesdaru.hu
>
> but still it's a leap forward
>

Is it designed to also identify From addresses that have no name component?

 From: l...@beroe-inc.com

This is an invoice phish that isn't tagged. Ideas on how to block these
would be appreciated.

https://pastebin.com/FXX8cx5f

This is with v4 SA from a week ago with FromNameSpoof enabled.

$ spamassassin --version
SpamAssassin version 4.0.0-r1901426
  running on Perl version 5.34.1

Jun 24 08:11:42.828 [3222587] dbg: plugin: loading
Mail::SpamAssassin::Plugin::FromNameSpoof from @INC
Jun 24 08:11:46.669 [3222587] dbg: FromNameSpoof: no From-name addr found

Re: DMARC fails for valid record?

[Alex]

Hi,


> >> doesn't amavisd by any chance use old SA installation/libraries?
>
> On 30.05.22 15:12, Alex wrote:
> >I don't think so - the current paths it uses are:
> >
> >/usr/share/spamassassin
> >/var/lib/spamassassin/4.00/updates_spamassassin_org
> >/var/lib/spamassassin/4.00/kam_sa-channels_mcgrail_com
> >/etc/mail/spamassassin/
>
> these are rules, not libraries.
>

Yes, I was responding to the "installation" part of your question.

there is a possibility that you have multiple versions of SA installed and
> amavis uses the old one.
>
> try running:
>
> % locate SpamAssassin.pm DMARC.pm
>

# locate SpamAssassin.pm DMARC.pm
/usr/share/perl5/vendor_perl/Mail/DMARC.pm
/usr/share/perl5/vendor_perl/Mail/SpamAssassin.pm
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/DMARC.pm

# ls -l /usr/share/perl5/vendor_perl/Mail/DMARC.pm
/usr/share/perl5/vendor_perl/Mail/SpamAssassin.pm
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/
DMARC.pm
-rw-r--r-- 1 root root 18600 Dec  8 23:01
/usr/share/perl5/vendor_perl/Mail/DMARC.pm
-r--r--r-- 1 root root  9752 May 29 11:14
/usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/DMARC.pm
-r--r--r-- 1 root root 77572 May 29 11:14
/usr/share/perl5/vendor_perl/Mail/SpamAssassin.pm

# rpm -qf /usr/share/perl5/vendor_perl/Mail/DMARC.pm
perl-Mail-Dmarc-PurePerl-1.20211209-3.fc35.noarch

# rpm -qf /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/DMARC.pm
spamassassin-4.0.0-85.fc35.x86_64

Those are both packages I've created and built for fedora and are based on
existing fedora packages.

>If I understand Kevin's comments correctly, we know there are still DMARC
> >problems. I think maybe this is related?
> >
> >$ spamassassin -t -D DMARC < dmarc-reject1 2>&1|grep -i dmarc
> >May 30 14:59:14.894 [1250699] dbg: DMARC: using Mail::DMARC::PurePerl for
> >DMARC checks
> >May 30 14:59:15.034 [1250699] dbg: DMARC: result: pass, disposition: none,
> >dkim: pass, spf: fail (spf: pass, spf_helo: fail)
> >DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DMARC_PASS,
>
> it hit DMARC_PASS, which is the opposite of DMARC_REJECT or
> KAM_DMARC_REJECT.
>

I was referring to the "spf: fail" component of that, which appears to
conflict with the "spf: pass" within the parentheses. Perhaps the first is
result of the combination of the two checks (HELO and envelope)?

Re: DMARC fails for valid record?

[Alex]

>
>
>
> >> did you reload/restart amavis after installing new SA?
> >> This header is added by amavis which uses SA libraries internally.
>
> On 30.05.22 09:50, Alex wrote:
> >Yes, thanks. This has been ongoing for weeks.
>
> doesn't amavisd by any chance use old SA installation/libraries?
>

I don't think so - the current paths it uses are:

/usr/share/spamassassin
/var/lib/spamassassin/4.00/updates_spamassassin_org
/var/lib/spamassassin/4.00/kam_sa-channels_mcgrail_com
/etc/mail/spamassassin/

May 30 15:05:16.089 [1254396] dbg: generic: Perl 5.034001, PREFIX=/usr,
DEF_RULES_DIR=/usr/share/spamassassin,
LOCAL_RULES_DIR=/etc/mail/spamassassin, LOCAL_STATE_DIR=/va
r/lib/spamassassin

The only rules in the /var/lib/spamassassin/ directory are those listed
above.

I used to have a local DMARC.cf file in /etc/mail/spamassassin before DMARC
was included in v4, but that's been removed.

If I understand Kevin's comments correctly, we know there are still DMARC
problems. I think maybe this is related?

$ spamassassin -t -D DMARC < dmarc-reject1 2>&1|grep -i dmarc
May 30 14:59:14.894 [1250699] dbg: DMARC: using Mail::DMARC::PurePerl for
DMARC checks
May 30 14:59:15.034 [1250699] dbg: DMARC: result: pass, disposition: none,
dkim: pass, spf: fail (spf: pass, spf_helo: fail)
DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DMARC_PASS,

Did SPF fail or pass above? It did hit SPF_PASS but it also
hit SPF_HELO_NONE.

It is curious that SA succeeds on its own but it's under amavisd that it
appears to fail.

I also see the following debug messages:

May 30 15:06:54.097 [1255659] dbg: check: tagrun - tag AUTHORDOMAIN is now
ready, value: indeedemail.com
May 30 15:06:54.325 [1255659] dbg: askdns: rule __KAM_DMARC_POLICY_REJECT
depends on tags: AUTHORDOMAIN
May 30 15:06:54.325 [1255659] dbg: check: tagrun - tag AUTHORDOMAIN was
ready, runnable immediately: CODE(0x563c09e23d70)
May 30 15:06:54.325 [1255659] dbg: askdns: launching query
(__KAM_DMARC_POLICY_REJECT): _dmarc.indeedemail.com
May 30 15:06:54.325 [1255659] dbg: async: query 50034/IN/TXT/_
dmarc.indeedemail.com already underway, adding no.4, rules:
__KAM_DMARC_POLICY_REJECT
May 30 15:06:54.518 [1255659] dbg: async: calling callback on key TXT/_
dmarc.indeedemail.com, rules: __KAM_DMARC_POLICY_REJECT
May 30 15:06:54.518 [1255659] dbg: askdns: answer received
(__KAM_DMARC_POLICY_REJECT), rcode NOERROR, query IN/TXT/_
dmarc.indeedemail.com, answer has 1 records
May 30 15:06:54.518 [1255659] dbg: askdns: domain "_dmarc.indeedemail.com"
listed (__KAM_DMARC_POLICY_REJECT): v=DMARC1; p=reject; sp=reject;
rua=mailto:f48jz-9...@rua.dm
arc.emailanalyst.com,mailto:dm...@indeed.com; ruf=mailto:
f48jz-9...@ruf.dmarc.emailanalyst.com; adkim=r; aspf=r; pct=100

So it did hit __KAM_DMARC_POLICY_REJECT but just not whatever else was
necessary to fulfill the requirements for the KAM_DMARC_REJECT when run
with SA manually.

Re: DMARC fails for valid record?

[Alex]

>
> >X-Spam-Status: No, score=-2.383 tagged_above=-200 required=5
> >tests=[BAYES_00=-1.9, DCC_REPUT_00_12=-0.4, DKIM_SIGNED=0.1,
> >DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DMARC_REJECT=0.1,
> >FROM_EXCESS_BASE64=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25,
> >HTML_IMAGE_RATIO_08=0.001, HTML_MESSAGE=0.001, KAM_DMARC_REJECT=1,
> >KAM_REALLYHUGEIMGSRC=0.5, LOC_MKTING=0.25, MIME_HTML_ONLY=0.1,
> >POISEN_SPAM_PILL=0.1, POISEN_SPAM_PILL_1=0.1,
> >RCVD_IN_HOSTKARMA_W=-2.5, RCVD_IN_SENDERSCORE_90_100=-0.6,
> >RELAYCOUNTRY_US=0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001,
> >TXREP=0.714, T_SCC_BODY_TEXT_LINE=-0.01] autolearn=disabled
>
> did you reload/restart amavis after installing new SA?
> This header is added by amavis which uses SA libraries internally.
>

Yes, thanks. This has been ongoing for weeks.

Re: DMARC fails for valid record?

[Alex]

Hi,

On Sun, May 29, 2022 at 8:10 PM Kevin A. McGrail 
wrote:

> There is also a rule update for priority levels.  Did you install the
> latest rules too?
>

Yes, sa-update runs every day. Last run was 00:29 this morning.

Re: DMARC fails for valid record?

[Alex]

Hi,

We have been DMARC issues so no, it is not you  Are you running the latest
> trunk right now?  There have been a flurry of patches and some of them are
> for this issue.
>

Yes, just downloaded, compiled, and installed the latest as of this moment
and still seeing the same problems initially. This is from realtor.com,
sent through cons.6...@envfrm.rsys2.com.

X-Spam-Status: No, score=-2.383 tagged_above=-200 required=5
tests=[BAYES_00=-1.9, DCC_REPUT_00_12=-0.4, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DMARC_REJECT=0.1,
FROM_EXCESS_BASE64=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25,
HTML_IMAGE_RATIO_08=0.001, HTML_MESSAGE=0.001, KAM_DMARC_REJECT=1,
KAM_REALLYHUGEIMGSRC=0.5, LOC_MKTING=0.25, MIME_HTML_ONLY=0.1,
POISEN_SPAM_PILL=0.1, POISEN_SPAM_PILL_1=0.1,
RCVD_IN_HOSTKARMA_W=-2.5, RCVD_IN_SENDERSCORE_90_100=-0.6,
RELAYCOUNTRY_US=0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001,
TXREP=0.714, T_SCC_BODY_TEXT_LINE=-0.01] autolearn=disabled

However, when I run it through SA after it's received, it doesn't hit
KAM_DMARC_REJECT or DMARC_REJECT. In fact, it hits DMARC_PASS. It
also continues to hit DKIM_VALID_AU. I don't know how to explain that.

I've changed the rule scores a bit, but have otherwise made no changes.
Perhaps when I ran it manually the timing of the checks were different?

I think we are having inconsistencies as well right now where the
> authentication header or lack thereof results in failing SPF in my
> environment soin my environment we are using other parts of the glue for a
> solution.
>
> When you look at the FPs for DMARC, are you seeing SPF failures or
> anything that you can track?
>

These also typically pass SPF, which is why I suppose my welcomelist_auth
rules continue to work.

Re: DMARC fails for valid record?

[Alex]

Hi, just wondering if anyone else has any ideas on how to solve this?

Is everyone with any v4 having problems with DMARC now or is it something
specific to my environment?

On Thu, May 26, 2022 at 2:36 PM Alex  wrote:

> Hi,
>
>
> On Thu, May 26, 2022 at 1:15 PM Bill Cole <
> sausers-20150...@billmail.scconsult.com> wrote:
>
>> On 2022-05-26 at 10:59:29 UTC-0400 (Thu, 26 May 2022 10:59:29 -0400)
>> Alex 
>> is rumored to have said:
>>
>> [...]
>> > Ugh, and again we already have DKIM_AU and SPF_PASS and DMARC_REJECT
>> > all
>> > hitting.
>>
>> Can you get these to match by re-running the same message with the
>> 'spamassassin' script?  If so, try it with "-D DMARC" to get all the
>> messages from the plugin. They may be illuminating.
>>
>
> This is from the example provided earlier today.  It says SPF failed(?)
> but it hit SPF_PASS
>
> May 26 14:25:12.080 [370198] dbg: DMARC: using Mail::DMARC::PurePerl for
> DMARC checks
> May 26 14:25:12.146 [370198] dbg: DMARC: result: pass, disposition: none,
> dkim: pass, spf: fail (spf: pass, spf_helo: fail)
>
> My suspicion *from a very quick 1st look at the code* is that the logic
>> for DMARC_REJECT is wrong, in that it seems to mean 'DMARC validation is
>> good' && 'p=reject,' which seems less than useful.
>>
>
> Any idea when this bug may have been introduced? It seems like a pretty
> serious problem to just be overlooked?
>
> And my confusion was actually only with the comments in the new DMARC.pm
> not reflecting 25_dmarc.cf with the new priority settings. It does appear
> I'm using the latest.
>
>
>

Re: DMARC fails for valid record?

[Alex]

Hi,


On Thu, May 26, 2022 at 1:15 PM Bill Cole <
sausers-20150...@billmail.scconsult.com> wrote:

> On 2022-05-26 at 10:59:29 UTC-0400 (Thu, 26 May 2022 10:59:29 -0400)
> Alex 
> is rumored to have said:
>
> [...]
> > Ugh, and again we already have DKIM_AU and SPF_PASS and DMARC_REJECT
> > all
> > hitting.
>
> Can you get these to match by re-running the same message with the
> 'spamassassin' script?  If so, try it with "-D DMARC" to get all the
> messages from the plugin. They may be illuminating.
>

This is from the example provided earlier today.  It says SPF failed(?) but
it hit SPF_PASS

May 26 14:25:12.080 [370198] dbg: DMARC: using Mail::DMARC::PurePerl for
DMARC checks
May 26 14:25:12.146 [370198] dbg: DMARC: result: pass, disposition: none,
dkim: pass, spf: fail (spf: pass, spf_helo: fail)

My suspicion *from a very quick 1st look at the code* is that the logic
> for DMARC_REJECT is wrong, in that it seems to mean 'DMARC validation is
> good' && 'p=reject,' which seems less than useful.
>

Any idea when this bug may have been introduced? It seems like a pretty
serious problem to just be overlooked?

And my confusion was actually only with the comments in the new DMARC.pm
not reflecting 25_dmarc.cf with the new priority settings. It does appear
I'm using the latest.

Re: DMARC fails for valid record?

[Alex]

Hi,

>> no matter if you have Mail::SpamAssassin::Plugin::DMARC loaded or not.
> >>
> >> Latest trunk has fix for DMARC waiting for SPF and DKIM results.  Might
> be
> >> relevant to this thread.
>
> according to:
>
> https://github.com/apache/spamassassin/commit/63fa58d814837f5d12b5d587ab4b72fa3c7501c3
>
> it should fix the problem.
>

Okay, wait, it doesn't appear that I have those changes.

$ spamassassin --version
SpamAssassin version 4.0.0-r1900857
  running on Perl version 5.34.1

I built SA using the following:

$ svn checkout http://svn.apache.org/repos/asf/spamassassin/trunk
Mail-SpamAssassin-4.0.0

This gave me revision 1901294.

Is that not the proper trunk?

Re: DMARC fails for valid record?

[Alex]

On Thu, May 26, 2022 at 10:40 AM Alex  wrote:

> Hi,
>
> > > Any further thoughts on this? It appears removing the DMARC perl
>> library
>> > > has disabled any DMARC support altogether.
>> >
>> > disabling Mail::SpamAssassin::Plugin::DMARC should
>> > make KAM.cf revert to it's simpler DMARC
>> > functioality
>> >
>> > note that it requires:
>> > Mail::SpamAssassin::Plugin::AskDNS
>> > Mail::SpamAssassin::Plugin::DKIM
>> > Mail::SpamAssassin::Plugin::SPF
>>
>
> Yes, these plugins are already enabled.
>
> > no matter if you have Mail::SpamAssassin::Plugin::DMARC loaded or not.
>>
>> Latest trunk has fix for DMARC waiting for SPF and DKIM results.  Might be
>> relevant to this thread.
>>
>
> Okay, new version in place, but without that perl DMARC plugin, still the
> same results with only KAM_DMARC_STATUS hitting.
>
> Going back to installing the PurePerl DMARC lib now as well.
>

Ugh, and again we already have DKIM_AU and SPF_PASS and DMARC_REJECT all
hitting.

 *  0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
 * -0.0 SPF_PASS SPF: sender matches SPF record
 * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
 *  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
 *  valid
 * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
 *   domain
 * -0.7 DKIMWL_WL_HIGH DKIMwl.org - High trust sender
 * -1.5 DKIMWL_WL ASKDNS: DKIMwl.org - Whitelisted sender
 *  [wish.com.lookup.dkimwl.org A:127.0.13.5]
  *  0.1 DMARC_REJECT DMARC reject policy
 *  1.0 KAM_DMARC_REJECT DKIM has Failed or SPF has failed on the message
 *  and the domain has a DMARC reject policy

It was quarantined because it also hit BAYES_99 and a local rule, despite
lowering KAM_DMARC_REJECT to just 1 point.





>
>
>

Re: DMARC fails for valid record?

[Alex]

Hi,

> > Any further thoughts on this? It appears removing the DMARC perl library
> > > has disabled any DMARC support altogether.
> >
> > disabling Mail::SpamAssassin::Plugin::DMARC should
> > make KAM.cf revert to it's simpler DMARC
> > functioality
> >
> > note that it requires:
> > Mail::SpamAssassin::Plugin::AskDNS
> > Mail::SpamAssassin::Plugin::DKIM
> > Mail::SpamAssassin::Plugin::SPF
>

Yes, these plugins are already enabled.

> no matter if you have Mail::SpamAssassin::Plugin::DMARC loaded or not.
>
> Latest trunk has fix for DMARC waiting for SPF and DKIM results.  Might be
> relevant to this thread.
>

Okay, new version in place, but without that perl DMARC plugin, still the
same results with only KAM_DMARC_STATUS hitting.

Going back to installing the PurePerl DMARC lib now as well.

Re: DMARC fails for valid record?

[Alex]

Hi,

>
> >I also haven't any references to DMARC whatsoever from any SA rules since
> >it was uninstalled.
>
> >I otherwise have no way of telling if there should have been any hits, but
> >I'd imagine there should have been at least one in 24-hours.
> >
> >It appears to have disabled DMARC functionality entirely.
>
> KAM.cf has some DMARC rules even without Mail::SpamAssassin::Plugin::DMARC
> available, but I'm not sure if loading that plugin doesn't disable them.
>
> I have disabled loading it so let's see.
>

Any further thoughts on this? It appears removing the DMARC perl library
has disabled any DMARC support altogether.

Re: DMARC fails for valid record?

[Alex]

>
>
>
> >On Tue, May 24, 2022 at 1:09 PM Matus UHLAR - fantomas  >
> >wrote:
> >> have there been rejects often before?
>
> On 24.05.22 13:58, Alex wrote:
> >I have hundreds of these over the last few days (week?), but they could go
> >back even further than that. It appears to primarily hit mailing lists or
> >statements from providers like AmEx or notices from Delta, for example.
>
>
> >> can you re-run spamassassin over those messages to see if uninstalling
> >> that package fixed the error with the same e-mails?
>
> >Yes, without that library, there's no reference to DMARC in the SA results
> >at all, even when T_DMARC_POLICY_NONE or T_DMARC_SIMPLE_DKIM would
> trigger.
>
> but you still get KAM_DMARC_REJECT for some mail?  because
> KAM_DMARC_REJECT
> has a workaround where it works w/o Mail::Dmarc::PurePerl
>

No, I haven't seen any hits since uninstalling the perl library.

I also haven't any references to DMARC whatsoever from any SA rules since
it was uninstalled.

I otherwise have no way of telling if there should have been any hits, but
I'd imagine there should have been at least one in 24-hours.

It appears to have disabled DMARC functionality entirely.

Re: DMARC fails for valid record?

[Alex]

On Tue, May 24, 2022 at 1:09 PM Matus UHLAR - fantomas 
wrote:

> >>> >I have perl-Mail-Dmarc-PurePerl-1.20211209-2.fc35.noarch installed.
> >>>
> >>> ... and this is the perl library.
> >>>
> >>> I see you have both  KAM_DMARC_REJECT and DMARC_REJECT
> >>> - KAM_DMARC_REJECT has workarounds if Mail::SpamAssassin::Plugin::DMARC
> >>> isn't available, but uses the library if it does.
> >>>
> >>> could you (temporarily) uninstall the
> >>> perl-Mail-Dmarc-PurePerl-1.20211209-2.fc35.noarch
> >>> if it fixes the problem?
>
> >On Mon, May 23, 2022 at 8:16 PM Alex  wrote:
> >> Since uninstalling it this morning, there have been no other occurrences
> >> of KAM_DMARC_REJECT all day for any emails.
>
> have there been rejects often before?
>

I have hundreds of these over the last few days (week?), but they could go
back even further than that. It appears to primarily hit mailing lists or
statements from providers like AmEx or notices from Delta, for example.



> can you re-run spamassassin over those messages to see if uninstalling
> that
> package fixed the error with the same e-mails?
>

Yes, without that library, there's no reference to DMARC in the SA results
at all, even when T_DMARC_POLICY_NONE or T_DMARC_SIMPLE_DKIM would trigger.

Re: DMARC fails for valid record?

[Alex]

On Mon, May 23, 2022 at 8:16 PM Alex  wrote:

>
>>
>> >I have perl-Mail-Dmarc-PurePerl-1.20211209-2.fc35.noarch installed.
>>
>> ... and this is the perl library.
>>
>> I see you have both  KAM_DMARC_REJECT and DMARC_REJECT
>> - KAM_DMARC_REJECT has workarounds if Mail::SpamAssassin::Plugin::DMARC
>> isn't available, but uses the library if it does.
>>
>> could you (temporarily) uninstall the
>> perl-Mail-Dmarc-PurePerl-1.20211209-2.fc35.noarch
>> if it fixes the problem?
>>
>
> Since uninstalling it this morning, there have been no other occurrences
> of KAM_DMARC_REJECT all day for any emails.
>
> The last DMARC_REJECT was also this morning prior to uninstalling
> perl-Mail-Dmarc-PurePerl.
>
> The only other references to DMARC today have been from KAM_DMARC_STATUS
>

What are the proper libraries that should be used to support DMARC with SA?

Re: DMARC fails for valid record?

[Alex]

>
>
>
> >I have perl-Mail-Dmarc-PurePerl-1.20211209-2.fc35.noarch installed.
>
> ... and this is the perl library.
>
> I see you have both  KAM_DMARC_REJECT and DMARC_REJECT
> - KAM_DMARC_REJECT has workarounds if Mail::SpamAssassin::Plugin::DMARC
> isn't available, but uses the library if it does.
>
> could you (temporarily) uninstall the
> perl-Mail-Dmarc-PurePerl-1.20211209-2.fc35.noarch
> if it fixes the problem?
>

Since uninstalling it this morning, there have been no other occurrences of
KAM_DMARC_REJECT all day for any emails.

The last DMARC_REJECT was also this morning prior to uninstalling
perl-Mail-Dmarc-PurePerl.

The only other references to DMARC today have been from KAM_DMARC_STATUS

Re: DMARC fails for valid record?

[Alex]

On Sun, May 22, 2022 at 1:51 PM Matus UHLAR - fantomas 
wrote:

> On 22.05.22 12:25, Kevin A. McGrail wrote:
> >#1 you can use the welcomelist entries but NOT the welcomelist_auth
> entries
> >if DMARC is failing.
>
> isn't welcomelist_auth okay with DKIM_VALID_AU ?
>

It looks like welcomelist_auth works with SPF even when this DMARC_REJECT
occurs, I believe.


> >#2 There are definitely some issues with SA 4.0 Trunk and DMARC issues
> that
> >we are working through, sorry to say it's been rougher than I wanted too.
> >But we have it in production and we are working on edge cases from my end.
>
> Alex (OP), do you have Mail::DMARC installed?
>

May 22 15:12:59.482 [865542] dbg: plugin: loading
Mail::SpamAssassin::Plugin::DMARC from @INC

I have perl-Mail-Dmarc-PurePerl-1.20211209-2.fc35.noarch installed.

Re: DMARC fails for valid record?

[Alex]

Hi, I think this is another - this one also includes KAM_DMARC_REJECT

https://pastebin.com/9g9VrgVK

 *  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
 *  valid
 * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
 *   domain
 * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
 *  6.0 KAM_DMARC_REJECT DKIM has Failed or SPF has failed on the message
 *  and the domain has a DMARC reject policy
 *  1.8 DMARC_REJECT DMARC reject policy

Can this info even be added to the welcomelist or will that also now fail?



On Sun, May 22, 2022 at 11:10 AM Alex  wrote:

> Hi, is it possible the DMARC_REJECT problem still exists?
>
> https://pastebin.com/DCu9cq4t
>
>  * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
>  *  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
>  *  valid
>  * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
> author's
>  *   domain
>  *  1.8 DMARC_REJECT DMARC reject policy
>
> Authentication-Results: xavier.example.com (amavisd-new);
> dkim=pass (1024-bit key) header.d=hotwire.com
> header.b="NEdhsCdV";
> dkim=pass (1024-bit key) header.d=amazonses.com
> header.b="UglVB1nr"
>
> $ spamassassin --version
> SpamAssassin version 4.0.0-r1900583
>   running on Perl version 5.34.1
>
>
> On Wed, May 11, 2022 at 9:01 AM Alex  wrote:
>
>> Hi,
>>
>> On Tue, May 10, 2022 at 7:00 PM Kevin A. McGrail 
>> wrote:
>>
>>> I believe this is a bug and fixed in trunk.
>>>
>>> On 5/10/2022 1:55 PM, Bill Cole wrote:
>>> > Looks like a bug. It should not be possible to hit DKIM_VALID_AU and
>>> also DMARC_REJECT and/or KAM_DMARC_REJECT
>>>
>>
>>
>> This was from svn version 1900493. I've now checked out 1900794, but that
>> somehow appears different from the version SA reports?
>>
>> $ spamassassin --version
>> SpamAssassin version 4.0.0-r1900583
>>   running on Perl version 5.34.1
>>
>> My firstdata email does appear to now pass DKIM properly,
>> without DMARC_REJECT or KAM_DMARC_REJECT.
>>
>> Any idea under what circumstances the DKIM check fails so I can watch for
>> it? Or can we consider it solved?
>>
>>
>>

Re: DMARC fails for valid record?

[Alex]

Hi, is it possible the DMARC_REJECT problem still exists?

https://pastebin.com/DCu9cq4t

 * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
 *  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
 *  valid
 * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
 *   domain
 *  1.8 DMARC_REJECT DMARC reject policy

Authentication-Results: xavier.example.com (amavisd-new);
dkim=pass (1024-bit key) header.d=hotwire.com
header.b="NEdhsCdV";
dkim=pass (1024-bit key) header.d=amazonses.com
header.b="UglVB1nr"

$ spamassassin --version
SpamAssassin version 4.0.0-r1900583
  running on Perl version 5.34.1


On Wed, May 11, 2022 at 9:01 AM Alex  wrote:

> Hi,
>
> On Tue, May 10, 2022 at 7:00 PM Kevin A. McGrail 
> wrote:
>
>> I believe this is a bug and fixed in trunk.
>>
>> On 5/10/2022 1:55 PM, Bill Cole wrote:
>> > Looks like a bug. It should not be possible to hit DKIM_VALID_AU and
>> also DMARC_REJECT and/or KAM_DMARC_REJECT
>>
>
>
> This was from svn version 1900493. I've now checked out 1900794, but that
> somehow appears different from the version SA reports?
>
> $ spamassassin --version
> SpamAssassin version 4.0.0-r1900583
>   running on Perl version 5.34.1
>
> My firstdata email does appear to now pass DKIM properly,
> without DMARC_REJECT or KAM_DMARC_REJECT.
>
> Any idea under what circumstances the DKIM check fails so I can watch for
> it? Or can we consider it solved?
>
>
>

Re: DMARC fails for valid record?

[Alex]

Hi,

On Tue, May 10, 2022 at 7:00 PM Kevin A. McGrail 
wrote:

> I believe this is a bug and fixed in trunk.
>
> On 5/10/2022 1:55 PM, Bill Cole wrote:
> > Looks like a bug. It should not be possible to hit DKIM_VALID_AU and
> also DMARC_REJECT and/or KAM_DMARC_REJECT
>


This was from svn version 1900493. I've now checked out 1900794, but that
somehow appears different from the version SA reports?

$ spamassassin --version
SpamAssassin version 4.0.0-r1900583
  running on Perl version 5.34.1

My firstdata email does appear to now pass DKIM properly,
without DMARC_REJECT or KAM_DMARC_REJECT.

Any idea under what circumstances the DKIM check fails so I can watch for
it? Or can we consider it solved?

DMARC fails for valid record?

[Alex]

Hi,

I'm trying to understand why this email from a bank fails DMARC when
mxlookup says the DMARC record is just fine.

https://pastebin.com/0T4Gjn3v

 *  1.8 DMARC_REJECT DMARC reject policy
 *  6.0 KAM_DMARC_REJECT DKIM has Failed or SPF has failed on the message
 *  and the domain has a DMARC reject policy

It also passes SPF and DKIM

 *  0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
 * -0.0 SPF_PASS SPF: sender matches SPF record
 * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
 *   domain
 * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
 *  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
 *  valid

I'm using a local DNS resolver, not a public server.

Re: SPF skipped for whitelisted relay domain

[Alex]

Hi,


> this is question for policyd-spf and its configuration.
>
> >The problem here is that something appears to be preventing my
> >welcomelist_auth entries from working properly, but I don't really
> >understand how.
>
> I guess it's the whitelist in policyd-spf.


Is it possible that it's somehow being passed through the port it uses to
communicate with postfix, and it's somehow using some postfix whitelist?

It most certainly isn't coming from a whitelist in policyd-spf, because it
happens even when it's completely removed.

I've also asked on the policyd-spf github page with no response.

Re: SPF skipped for whitelisted relay domain

[Alex]

Hi,


> >https://pastebin.com/TvTx6KzY
>
> X-Comment: SPF skipped for whitelisted relay domain -
> client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com;
> envelope-from=re...@support.meridianlink.com; receiver=
> X-Greylist: whitelisted by SQLgrey-1.8.0
>
> isn't it possible that it's sqlgrey that whitelisted your domain?
>

Yes, I suppose that's possible - meridianlink.com is listed in
my clients_fqdn_whitelist.local file, but how would policyd-spf interpret
that it should whitelist SPF? How would that communication even occur? That
"SPF skipped for whitelisted relay domain" content is coming from
policyd-spf.

The problem here is that something appears to be preventing my
welcomelist_auth entries from working properly, but I don't really
understand how.

Thanks so much for your help.

Re: SPF skipped for whitelisted relay domain

[Alex]

> >I'm trying to understand why some domains are not whitelisted even
> >though they pass SPF and are in my local welcomelist_auth entries. I'm
> >using policyd-spf with postfix, and it appears to be adding the
> >following header:
> >
> >X-Comment: SPF skipped for whitelisted relay domain -
> >client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com;
> >envelope-from=re...@support.meridianlink.com; receiver=
>
> you seem to have domain listed in whitelist policyd-spf whitelist.
> salesforce.com probably?

I figured out where it's whitelisted, but still don't understand how it works.

It's somehow referencing the postscreen access list I'm using:

postscreen_access_list =
permit_mynetworks, cidr:$config_directory/postscreen_access.cidr

In that file are cidr entries like:
13.110.208.0/21 permit
13.110.216.0/22 permit
13.110.224.0/20 permit

This file is auto-generated from my postwhite script that gathers IPs
for the "too big to fail" providers like salesforce and google and
microsoft.

which match the client IP for salesforce:
client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com

I was aware of this access list, but I wasn't aware that the policy
daemon was also using it as well as postscreen.

The problem now is that I don't know _how_ it's using it, and how to
prevent it from affecting my welcomelist_auth entries. I don't see any
reference in the code that would indicate it's somehow getting this
info from postscreen/postfix and using it when making these decisions.

The unmodified original messages also no longer pass SPF - shouldn't
they? It does still pass DKIM from the command-line, and therefore my
welcomelist_auth entry, but not when it's first received.

There was a reason I added this email to the welcomelist in the first
place. Perhaps a temporary solution would be to just remove the
postscreen access lists for now? Other ideas? Someone would like to
help me troubleshoot this? I'm thinking the fact that the IP is
whitelisted in postscreen is somehow being passed through the socket
to policyd-spf in a structure somewhere.

> >My welcomelist entry in SA for this specific email is as:
> >welcomelist_auth re...@support.meridianlink.com
>
> is this in spamassassin's local.cf ?

Yes

> >salesforce is also listed in their SPF record:
> >$ dig +short txt support.meridianlink.com
> >"v=spf1 include:spf.protection.outlook.com include:_spf.salesforce.com -all"
>
> SPF_PASS idicates that the SPF hit.
>
> however, posting full headers could help us a bit.

https://pastebin.com/TvTx6KzY

$ spamassassin --version
SpamAssassin version 4.0.0-r1889518
  running on Perl version 5.32.1

SPF skipped for whitelisted relay domain

[Alex]

Hi,

I'm trying to understand why some domains are not whitelisted even
though they pass SPF and are in my local welcomelist_auth entries. I'm
using policyd-spf with postfix, and it appears to be adding the
following header:

X-Comment: SPF skipped for whitelisted relay domain -
client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com;
envelope-from=re...@support.meridianlink.com; receiver=

I realize this may not necessarily be directly related to SA, but it's
apparently affecting my ability to process SPF headers with
amavisd/SA, and I hoped someone could help.

What's happening where the mail passes SPF but still bypasses my
welcomelist entries? My skip_addresses list doesn't include this
particular IP:
skip_addresses =
139.138.56.0/24,127.0.0.0/8,:::127.0.0.0/104,::1,52.128.98.0/24,74.203.184.0/24,74.200.60.0/24,209.222.82.0/24,12.15.90.10


My welcomelist entry in SA for this specific email is as:
welcomelist_auth re...@support.meridianlink.com

The amavisd headers show it passed SPF:

Return-Path: 
X-Spam-Status: No, score=-2.491 tagged_above=-200 required=5
tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, EXTRACTTEXT=0.001,
FMBLA_HELO_OUTMX=-0.01, FMBLA_RDNS_OUTMX=-0.01,
HTML_IMAGE_RATIO_08=0.001, HTML_MESSAGE=0.001, LOC_CDIS_INLINE=0.1,
LOC_IMGSPAM=0.1, RCVD_IN_DNSWL_NONE=-0.0001,
RCVD_IN_SENDERSCORE_90_100=-0.6, RELAYCOUNTRY_US=0.01,
SPF_HELO_NONE=0.001, SPF_PASS=-0.001, TXREP=0.016] autolearn=disabled

This one didn't need to be added to the welcomelist, but others do.
The last header received before reaching our server is as:

Received: from smtp14-ph2-sp4.mta.salesforce.com
(smtp14-ph2-sp4.mta.salesforce.com [13.110.6.221])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by mail01.example.com (Postfix) with ESMTPS id 5FC7010024E93
for ; Thu,  5 May 2022 12:01:59 -0400 (EDT)

salesforce is also listed in their SPF record:
$ dig +short txt support.meridianlink.com
"v=spf1 include:spf.protection.outlook.com include:_spf.salesforce.com -all"

Thanks,
Alex

Re: Untrustworthy TLDs and KAM

[Alex]

On Sun, May 1, 2022 at 9:47 PM Kevin A. McGrail  wrote:
>
> Did it cause a fp with a score of 5.0 or higher?

Yes.

https://pastebin.com/AqezMHjQ

Thanks!

Untrustworthy TLDs and KAM

[Alex]

Hi,

Four points for a .online TLD with KAM rules

 *  2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs
 *  [URI: www.lci-mtc.online (online)]

 *  2.0 KAM_SOMETLD_ARE_BAD_TLD .bar, .buzz, .cam, .casa, .cfd, .club,
 *  .date, .guru, .live, .online, .press, .pw, .quest, .rest, .sbs,
 *  .shop, .stream, .top, .trade, .work, .xyz TLD abuse

$ spamassassin --version
SpamAssassin version 4.0.0-r1898781
  running on Perl version 5.34.1

Re: How to deal with bounce messages

[Alex]

Hi,

> >> >https://pastebin.com/s032ndrA
> >> >
> >> >It's not only hitting DMARC_REJ_NO_DKIM and DMARC_FAIL_REJECT, but
> >>
> >> where did you get these from?
>
> On 22.04.22 10:02, Alex wrote:
> >I just realized these are from my local rules, put together from a
> >conversation many years ago, apparently from before SA had built-in
> >DMARC support.
> >
> >https://www.mail-archive.com/users@spamassassin.apache.org/msg95643.html
>
> now I really wonder why these aren't part of stock SA rules.

Does this mean you are investigating further?

Are these rules from the link above useful?

Re: How to deal with bounce messages

[Alex]

> >https://pastebin.com/s032ndrA
> >
> >It's not only hitting DMARC_REJ_NO_DKIM and DMARC_FAIL_REJECT, but
>
> where did you get these from?

I just realized these are from my local rules, put together from a
conversation many years ago, apparently from before SA had built-in
DMARC support.

https://www.mail-archive.com/users@spamassassin.apache.org/msg95643.html

> DMARC is built on header From:, this needs either correct DKIM, or correct SPF
> with envelope from domain equal to header From: domain.
>
> this message has no DKIM signature. policy is reject:
>
> _dmarc.deltra.de.   86400   IN  TXT 
> "v=DMARC1;p=reject;rua=mailto:postmas...@deltra.de;

Ah yes, thanks.

> and spf is unapplicable since the envelope from is null.

Isn't that the case with all bounce messages?

> >It's also somehow hitting BAYES_99 - do you train your bounce messages?
>
> yes.

Great, thanks.

How to deal with bounce messages

[Alex]

Hi,

I'm seeing bounce messages being tagged incorrectly and would like
some help in how to fix it. This is mail sent from our servers.

https://pastebin.com/s032ndrA

It's not only hitting DMARC_REJ_NO_DKIM and DMARC_FAIL_REJECT, but
also KAM_DMARC_REJECT, giving it no opportunity to ever get delivered.
I'm assuming it's failing DMARC because the envelope sender is the
same as the original message but the message body and From address are
that of the rejecting server.

How do others deal with this? I'm just now seeing
welcomelist_bounce_relays - I don't think I've seen that before or
have defined it. The description says the BOUNCE_MESSAGE won't fire if
this isn't defined, yet this rule was triggered.

It's also somehow hitting BAYES_99 - do you train your bounce messages?

Thanks,
Alex

Microsoft to block Office VBA macros by default

[Alex]

Hi,

I'm just curious if this announcement has changed anyone's thinking
about how we should be handling docx/xlsx/etc attachments in email?
This obviously doesn't prevent someone from emailing a document with a
malicious macro, but is this going to provide sufficient protection
once a potentially malicious document is received to relax email
protections a bit?

https://www.theverge.com/2022/2/7/22922032/microsoft-block-office-vba-macros-default-change

Are you outright blocking these attachments? Perhaps you're only
blocking those with macros?

Is the ExtractText plugin good enough to extract potentially malicious
links to be checked?

DCC/pyzor questions

[Alex]

Hi,

I'm seeing a lot of DCC/pyzor mail being marked as spam that shouldn't
be, and want to see what can be done to prevent that.

For example, many emails with just an image attachment and an empty
body are hitting DCC. I thought I recalled a way to create a checksum
of these empty messages and add them to an allow list, but it seems it
is specific to the sender, based on /var/lib/dcc/testmsg-whitelist:

# empty Exchange
ok  hex fuz1 e038b933 6003e07e 8e990536 110cfa90

How do I generate that signature? I've been unable to find any
instructions on how to do it. Same with pyzor?

Another example is an email I received from Pizza Hut. Their marketing
emails hit DCC and pyzor and sendgrid, making it very difficult for
that email to be delivered unless it also hits some negative bayes or
is allowlisted. Do people add them to the welcomelist? Do you train
marketing emails for bayes?

 *  1.5 KAM_SENDGRID Sendgrid being exploited by scammers
 *  0.3 DIGEST_MULTIPLE Message hits more than one network digest check
 *  1.0 DCC_REPUT_95_98 DCC reputation between 95 and 98 %  (mostly spam)
 *  0.5 KAM_REALLYHUGEIMGSRC RAW: Spam with image tags with ridiculously
 *  huge http urls
 *  1.4 PYZOR_CHECK Listed in Pyzor
 *  3.0 BAYES_95 BODY: Bayes spam probability is 95 to 99%
 *  [score: 0.9668]
 *  0.1 POISEN_SPAM_PILL_3 BODY: random spam to be learned in bayes

Is sendgrid still as big of a problem as it was a year ago?

There are a few negative rules, like TXREP and DKIMWL_WL and
RCVD_IN_SENDERSCORE_90_100, but someone really doesn't want Pizza Hut
email to be delivered.

Separately, is ExtractText broken? I have legitimate invoices that are
hitting multiple money rules. Is this the expected behavior? Any
advice on how to deal with it?

Re: fuglu 1.0.1

[Alex]

Hi,

> We use fuglu in production at work and it works very nicely. But it was
> on a centos machine. I have it too on a debian raspberry pi and just
> updated from gitlab. I had a domainmagic dependency missing too but
> mentionned it to a dev, who's working on a fix. I have no experience
> with gentoo though.
>
> At work, we switched to the dockerized version of fuglu, maybe you could
> consider it.

I'm also interested in this, as I'm having problems with amavisd (and
development has effectively stopped).

Does anyone know how it compares? Is it possible to more specifically
define the policies it applies to domains it processes? For example,
the problem I'm currently having with amavisd is that things like
virus/spam destiny, or banned filenames are limited to affecting all
domains the amavisd instance processes. It would be good to have more
flexibility there - one domain may wish to allow html files while
another would like to block them.

I've posted this on the amavisd list before, and it's gone unanswered.
I'm hoping fuglu is a more modern replacement without too much
difficulty in the conversion/replacement.

I also like fuglu for it being coded in python - it's much easier to
find python developers than perl developers these days.

> But I doubt this mailing list is the best place to talk about fuglu.

Yes, not strictly related, but I'm hoping it's closely related enough
for someone to give me some pointers, given we're all using SA.

Thanks,
Alex


>
> Best regards,
> Laurent
>
> On 24.09.21 05:12, Benny Pedersen wrote:
> >
> > anyone using it ? :)
> >
> > i added it to ::fidonet gentoo overlay, it missed dev-python/pygeoip,
> > dev-python/domainmagic in gentoo portage to work, with i found after i
> > had created the first gentoo ebuild for fuglu 1.0.1, late night works,
> > hehe :)
> >
> > if others is really using fuglu please share
> >
>

Re: freshworks and DKIM and KAM

[Alex]

Hi,

> > I can't figure out why attempts at adding emails from the
> > freshworks.com domain to the welcome list aren't successful. This is
> > from a quarantined message on my amavis/SA/fedora system.
> >
> > I'm not sure why the entirety of freshworks.com would be blocked in
> > the first place?
> >   *  9.0 KAM_FROM_URIBL_PCCC RBL: From address listed in PCCC URIBL
> >   *  (https://raptor.pccc.com/RBL)
> >   *  [listed in freshworks.com.wild.pccc.com]
> >   *  9.0 KAM_BODY_URIBL_PCCC Body contains URI listed in PCCC URIBL
> >   *  (https://raptor.pccc.com/RBL)
> >   *  [URI: freshworks.com]
>
> Looking at the channel rules file that's surrounded by an ifplugin:
>
>ifplugin Mail::SpamAssassin::Plugin::KAMOnly
>
> and so unless you are one of Peregrine Computer Consultants Corporation
> paying customers you are likely (correctly, justifiably) getting funky
> DNSBL lookup results.

Yes, it seems I have misinterpreted the meaning of the publically
available KAMOnly.cf file.

Fixed, thanks

freshworks and DKIM and KAM

[Alex]

Hi,

I can't figure out why attempts at adding emails from the
freshworks.com domain to the welcome list aren't successful. This is
from a quarantined message on my amavis/SA/fedora system.

I'm not sure why the entirety of freshworks.com would be blocked in
the first place?
 *  9.0 KAM_FROM_URIBL_PCCC RBL: From address listed in PCCC URIBL
 *  (https://raptor.pccc.com/RBL)
 *  [listed in freshworks.com.wild.pccc.com]
 *  9.0 KAM_BODY_URIBL_PCCC Body contains URI listed in PCCC URIBL
 *  (https://raptor.pccc.com/RBL)
 *  [URI: freshworks.com]

X-Envelope-From:

Authentication-Results: xavier.example.com (amavisd-new);
dkim=pass (2048-bit key) header.d=freshworks.com

All of these rules were hit for this email, yet it is still being quarantined.
"DKIM_SIGNED","DKIM_VALID","DKIM_VALID_AU", "USER_IN_DKIM_WELCOMELIST"

USER_IN_DKIM_WELCOMELIST is only scored with 0.01. I saw a reference
to feature_blocklist_welcomelist in 60_whitelist_dkim.cf. Perhaps
that's related?

$ spamassassin --version
SpamAssassin version 4.0.0-r1889518
  running on Perl version 5.32.1

Here are my welcomelist/whitelist entries:
welcomelist_from_dkim *@frwfiddkim.freshworks.com
welcomelist_from_dkim *@freshworks.com
welcomelist_from *@frwfiddkim.freshworks.com
whitelist_from *@frwfiddkim.freshworks.com
welcomelist_from *@freshworks.com
welcomelist_from *@freshsales.com
welcomelist_from *@*.freshsales.com
welcomelist_from *@*.freshworks.com
welcomelist_from *@freshmarketer.com
welcomelist_from *@freshsales.io
welcomelist_auth *@freshworks.com
welcomelist_auth *@*.freshworks.com
welcomelist_spf *@frwfiddkim.freshworks.com
whitelist_spf *@frwfiddkim.freshworks.com

Many of these are just temporary to test different options (especially
the *_from entries).

Is this a bug?

https://pastebin.com/6u4uNnLQ

Ideas greatly appreciated.

Re: Office phish

[Alex]

Hi,

> >> I realize blocking all javascript is prone to error,
> > What legitimate email uses javascript?
> And more important: which email clients do actually process Javascript
> that comes within an email? Thunderbird doesn't since 10 or 20 years
> ago. I don't know of any other as well. This phish is probably targeted
> to inferior web-based email readers who don't filter Javascript well.
> Are there any?

It's not a matter of processing/rendering javascript by default in an
email, but someone clicking the ".htm" file, even in Thunderbird,
which then renders the HTML/javascript in the browser.

In this case, the ".htm" file is a rogue O365 login page.

Re: Office phish

[Alex Woick]

I realize blocking all javascript is prone to error,

What legitimate email uses javascript?
And more important: which email clients do actually process Javascript 
that comes within an email? Thunderbird doesn't since 10 or 20 years 
ago. I don't know of any other as well. This phish is probably targeted 
to inferior web-based email readers who don't filter Javascript well. 
Are there any?

Re: Office phish

[Alex]

Hi,

> > I modified the ExtractText plugin to also process HTML files
> >
> > extracttext_externalhtmlcat /usr/bin/cat {}
> > extracttext_use htmlcat   .htm .html
> >
>
> Quite horrible hack, as the result should be _rendered_ text.  Inserting raw
> HTML for all body rules is probably breaking more things than fixing.
>
> But yeah a "mimebody" ruletype would probably be useful..

Would you explain a bit further? Until such a ruletype exists, how do
you propose we solve this javascript issue? How do we search through
MIME attachments without using ExtractText?

Block the resulting URI in the javascript body? I was hoping for
something more generic.

I realize blocking all javascript is prone to error, but what about
blocking all "location.href" attempts? Or "document.write"?

Am I really the only one seeing these attacks?

Re: Office phish

[Alex]

Hi,

> SpamAssassin has plugins for PhishTank and OpenPhish. I would suggest
> you submit the link to them.
> You can also reach out to the domain provider, hosting provider(s) and
> other companies involved.

> > https://pastebin.com/JMSrY6KU

We've got to do better than that. These O365 phishing attacks are
significant and severe and constant.

I modified the ExtractText plugin to also process HTML files

extracttext_externalhtmlcat /usr/bin/cat {}
extracttext_use htmlcat   .htm .html

then created the following rule to look for 

Office phish

[Alex]

Hi,
Would anyone like to help me block this office phish? It includes an
HTML file that presents an O365 login page:

https://pastebin.com/JMSrY6KU

More javascript in an HTML file.

adobe cloud malicious link

[Alex]

Hi,

I received what appears to be a legitimate email from what looks like
a compromised adobe account that itself contains no malicious links,
but redirects to a malicious link once on the adobe site.

https://pastebin.com/thp1Atah

I don't suppose there's any protection against this, considering the
malicious link isn't contained within the email itself?

Once on the site, it displays a PDF designed to look like a docusign
document with a malicious link.

Re: KAM_SENDGRID and SPF_HELO_NONE

[Alex]

Hi,

> > I have an email that matched KAM_SENDGRID because it also matched
> > SPF_HELO_NONE, despite it apparently being a legitimate sendgrid
> > email. This is from SA trunk.

I only meant it as a reference for the version of SA (and SPF.pm)
that's being used, in case it was necessary.

> > X-Envelope-From:
> > 
> >
> > I'm noticing what I think are a lot of false positives for this rule.
>
> In what way is this a false positive? Looks like a correct positive to
> me.

Because it was a legitimate email with an invoice from a pest control
company to their customer.

> If you disagree with the scoring or purpose of that rule, you are free
> to reduce the score locally or discuss it with KAM. He's a very

Nope, just trying to understand.

> KAM's
> QA is a 100% black box but he makes changes fast when needed.

Yes, and just wanted to be sure that wasn't necessary here.

> > Perhaps it's because Return-Path is null?
> > Return-Path: <>
>
> That's a different problem, apparently with your MTA->SA glue. The fact
> that something added a non-null "X-Envelope-From:" header and something
> (else?) added a null "Return-Path:" header indicates fundamental
> breakage. Whether SA is seeing that or if it is a delivery artifact is
> unclear.

Perhaps this is a problem with my amavis configuration? It appears all
quarantined messages have a null Return-Path header.

KAM_SENDGRID and SPF_HELO_NONE

[Alex]

Hi,

I have an email that matched KAM_SENDGRID because it also matched
SPF_HELO_NONE, despite it apparently being a legitimate sendgrid
email. This is from SA trunk.

 0.0 SPF_HELO_NONE  SPF: HELO does not publish an SPF Record
-0.0 SPF_PASS   SPF: sender matches SPF record
-0.1 DKIM_VALID_AU  Message has a valid DKIM or DK signature
from author's
 0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not
necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
 1.5 KAM_SENDGRID   Sendgrid being exploited by scammers

Received-SPF: Pass (mailfrom) identity=mailfrom;
client-ip=167.89.39.250; helo=o1678939x250.outbound-mail.sendgrid.net;
envelope-from=bounces+3940809-b10a-43194=hotel.example@em8909.cookspest.com;
receiver=

X-Envelope-From:


I'm noticing what I think are a lot of false positives for this rule.
Is there something more we should be doing to reduce the false
positives here, or is it really warranted?

The mail server does appear to have an SPF record:

# dig +short txt em8909.cookspest.com
u3940809.wl060.sendgrid.net.
"v=spf1 ip4:167.89.39.18 ip4:167.89.39.188 ip4:167.89.39.217
ip4:167.89.39.227 ip4:167.89.39.248 ip4:167.89.39.250 ip4:167.89
.39.45 ip4:167.89.39.75 ip4:167.89.39.79 ip4:208.117.61.64 -all"

Or perhaps it's because it's announcing itself as
o1678939x250.outbound-mail.sendgrid.net, which does not have an SPF
record?

Is it even possible for a sendgrid client to control their SPF record,
let alone SPF HELO?

Perhaps it's because Return-Path is null?
Return-Path: <>

FROMNAME and PDS_FROM_2_EMAILS

[Alex]

Hi,
I'm trying to understand the FROMNAME rules and a potential conflict
with PDS_FROM_2_EMAILS.

I understand FROMNAME_SPOOF is designed to catch differences like:

From: "no-re...@amazon.com" 

but what other spoofs is the FromName.pm plugin designed to catch?

And I would assume it would be DKIM for differences between the
address in the From and the SPF record for the envelope sender,
correct?

I've also noticed that the PDS_FROM_2_EMAILS meta and
PDS_FROMNAME_SPOOFED_EMAIL hits on many similar components that I
wondered if there's overlap or if I'm misunderstanding how it works.

ExtractText and docx

[Alex]

Hi,

I'm trying to use the latest ExtractText plugin, but the docx2txt
program the plugin references is no longer available from
http://docx2txt.sourceforge.net

I've located a working replacement at
https://github.com/ankushshah89/python-docx2txt/ (although it's
written in python and I don't have a distro package for that), it
doesn't appear to output to stdout.

extracttext_external  docx2txt   /usr/local/bin/docx2txt {} -
extracttext_use   docx2txt   .docx application/docx

Do you have any recommendations for an alternative or how to modify
this python script to pipe its text to stdout?

# /usr/local/bin/docx2txt -h
usage: docx2txt [-h] [-i IMG_DIR] docx

A pure python-based utility to extract text and images from docx files.

positional arguments:
  docx  path of the docx file

optional arguments:
  -h, --helpshow this help message and exit
  -i IMG_DIR, --img_dir IMG_DIR
path of directory to extract images

Also, has anyone written any meta rules for use with ExtractText that
they'd like to share? I'd like to block all PDF file that contain any
type of javascript - malicious or otherwise. I'd also like to block
all PDFs that's a single page and contain a single URL - that appears
to be the vast majority of all malicious PDFs.

Re: More fake order spam

[Alex]

Hi,

> >-1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list
> >manager
>
> I have disabled his rule some time ago.
> Many spammers use mailing list or their signatures.

Where is the score coming from for this rule? There isn't an explicit
"score" value associated with the rule.

describe MAILING_LIST_MULTI Multiple indicators imply a
widely-seen list manager
meta   MAILING_LIST_MULTI   __HAS_X_LOOP + __HAS_X_MAILING_LIST +
__HAS_X_MAILMAN_VERSION + __HAS_LIST_ID + __HAS_X_BEEN_THERE
+__DOS_HAS_LIST_UNSUB + __ML1 + __ML3 + __ML4 + __ML5 > 2
tflags MAILING_LIST_MULTI   nice

If everyone (figuratively speaking, I suppose) is disabling it,
wouldn't it be helpful to define it explicitly or see how it's doing
in masschecks?

It seems like it would be helpful to look at ways mailing lists are
manipulated by spammers more closely and perhaps find some anomalies
there.

Re: OT: Re: Unsubscribe link at the bottom.

[Alex Woick]

John Hardin schrieb am 06.04.2021 um 16:34:

On Mon, 5 Apr 2021, Grant Taylor wrote:


On 4/5/21 8:41 PM, Peter West wrote:
I’d agree it’s address verification, as with the Unsubscribe link at 
the bottom.


I'm of the opinion that if I have any inclining of knowledge of the 
company sending the email, and SPF/DKIM/DMARC pass, I'll probably use 
the unsubscribe link.


Recently I ran into a 404 from the unsubscribe link from a company 
that my wife did business with.  *facepalm*


What ticks me off is an unsubscribe link that goes to a 
javascript-heavy page and that *won't work* without javascript.


And an unsubscribe link with a huge identifying key on it, yet the 
unsubscribe page still asks you to enter your email address...


As far as I see it, unsubscribe links from spammers are placebo, because 
the "campaign" or "mailing list" or "newsletter" is created for just one 
mass mailing session, then never used again. You don't need to 
unsubscribe (nor the spammer need to provide a real working unsubscribe 
functionality), because your address isn't used for *this* mailing list 
again anyway. However, it's used for creating a whole new mailing list 
tomorrow and a second one the day after tomorrow, but that's a 
completely different newsletter than the one from today (as the spammer 
would say, why he continues sending although you unsubscribed).

Re: Problem with local.cf rules

[Alex Woick]

Peter West schrieb am 14.03.2021 um 14:30:

header CASINO From =~ /\bcasino\b/i
score 100.0

===


It’s hitting the CASINO rule, but no matter what valoue I assign to the casino 
rules - 5, 20 , 100, these messages always come through with a value of 4.1. 
It’s as though some toerh rule is resetting the score to 0 before proceeding.
You need to tell the rule name with the score keyword, otherwise 
spamassassin cannot know to which rule it should set the score.


score CASINO 100