body rule only for txt or html?

[Tobi]

Hello list

ifplugin Mail::SpamAssassin::Plugin::ExtractText
extracttext_external  pdfgrep  /usr/bin/pdfgrep .+ {}
extracttext_use   pdfgrep  .pdf application/pdf
endif

which leads to the fact that body rules then can also hit on pdf
content. Is there a possibility for a rule to search only in the text
and/or html representation of a message?

Cheers

tobi

Re: SPF_FAIL

[Tobi]

> If I only had a ready-made list of those important domains.

If you filter for customer domains then maybe (depending the customer
domain) adding the customer domain to spf checks is worth a look too.


On 11/11/20 6:29 AM, Victor Sudakov wrote:
> John Hardin wrote:
>>
>>> Moreover, after reading other replies in the thread, I am even begining to
>>> doubt the wizdom of rejecting hard SPF fails in the MTA (which I do in
>>> some installations).
>>
>> "it depends".
>>
>> Doing that for certain domains - like, large banks - would probably be a
>> good idea. By default, for all domains, not so much.
>
> If I only had a ready-made list of those important domains.
>
>

Re: Spamssassin seems to append .com TLD to uri link domains found

[Tobi]

On 11/9/20 3:18 PM, RW wrote:
> When you say FP do you mean that ch.com was listed in a domain
> blocklist?

yes, the "ch.com" was listed in one of the domain blocklists. And as the
customer used his own domain "www.ch" as footer it got caught by the
lookup for "ch.com" which seems to be a chinese news page or something
like that.

Its hard to explain to a customer that a URI domain he did not use in
the message lead to a hit on blocklists lookup ;-)

Cheers

tobi

Re: Spamssassin seems to append .com TLD to uri link domains found

[Tobi]

John,

> Because I think that suppressing that behavior for valid TLDs would be
an appropriate modification to avoid potential URIBL FPs

fully agree. SA should not append .com if the domain has a valid tld and
a domain label. We know of at least one FP related to "www.ch" when (the
expanded version) "ch.com" was checked on uribl lists.

Cheers

tobi

On 11/7/20 8:04 PM, John Hardin wrote:
> On Sat, 7 Nov 2020, RW wrote:
>
>> On Sat, 7 Nov 2020 10:05:21 -0800 (PST)
>> John Hardin wrote:
>>
>>> On Sat, 7 Nov 2020, RW wrote:
>>>
>>>> On Fri, 6 Nov 2020 16:10:18 +
>>>> RW wrote:
>>>>
>>>>
>>>>> However, I can't get an up-to-date Firefox to add .com, so the
>>>>> feature may already be obsolete.
>>>>
>>>> I take that back, it does.
>>>
>>> What does it do for the example at hand, http://www.ch ?
>>
>> Firefox only adds .com if the domain doesn't resolve.
>>
>> www.ch resolves and then redirects to https://meteo.ch/
>>
>> If SA is to allow for what Firefox does then I think the behaviour is
>> reasonable. A DNS lookup would be overkill,
>
> Agreed.
>
>> and there's no particular reason to exclude labels that happen to be
>> TLDs.
>
> Do you mean *valid* TLDs? Because I think that suppressing that behavior
> for valid TLDs would be an appropriate modification to avoid potential
> URIBL FPs (which, granted, is probably fairly unlikely) and to avoid the
> overhead of extra lookups.
>
>

Re: Spamssassin seems to append .com TLD to uri link domains found

[Tobi]

ah understand, should have better checked what SA really adds to domain
list. So both versions are checked. Just bad luck if the expanded
version of the uri domain (ex ch.com) has a blacklisting at uribl or
spamhaus ;-)
But that's another story

Have a good weekend

tobi

On 11/6/20 5:42 PM, RW wrote:
> On Fri, 6 Nov 2020 17:24:58 +0100
> Tobi wrote:
>
>>> www.ch is a weird special case as a
>>> domain name
>>
>> Did you check whois for www.ch? It's a registered domain and it
>> resolves, so the owner has a whitelisting in SA or better his www.ch
>> links which may host a very sophisticated spearphish page will never
>> be queried by SA against blacklists.
>
>
> The .com version is added as an additional URL, www.ch is still
> queried.
>

Re: Spamssassin seems to append .com TLD to uri link domains found

[Tobi]

> www.ch is a weird special case as a
> domain name

Did you check whois for www.ch? It's a registered domain and it
resolves, so the owner has a whitelisting in SA or better his www.ch
links which may host a very sophisticated spearphish page will never be
queried by SA against blacklists.

Sorry but that imho is a bug that should (better must) be fixed :-)

Cheers

tobi

On 11/6/20 5:10 PM, RW wrote:
> On Fri, 6 Nov 2020 15:40:31 +0100
> "Tobi wrote:
>
>> Hi list
>>
>> we currently see the following "issue" where SA does append .com TLD
>> to uri domains found in body.
>>
>> Add a uri like "www.ch" to a txt body part and SA will add "ch.com" to
>> domains to be checked
>>
>> Nov  6 15:23:52.527 [16955] dbg: uri: canonicalizing parsed uri:
>> http://www.ch
>> Nov  6 15:23:52.527 [16955] dbg: uri: cleaned uri: http://www.ch.com
>> Nov  6 15:23:52.527 [16955] dbg: uri: added host: www.ch.com domain:
>> ch.com [...]
>> Nov  6 15:23:52.602 [16955] dbg: async: launching
>> A/ch.com.multi.uribl.com for DNSBL:ch.com:multi.uribl.com
>>
>> Any other hostname than "www" does not trigger that behavior. So
>> ftp.ch get correctly queried as ftp.ch and not ch.com
>
> I think it's from:
>
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6596
>
> It looks to be about Firefox adding .com in some cases where the
> apparent domain is not resolving. www.ch is a weird special case as a
> domain name, so I'm not surprized it isn't handled separately.
>
> However, I can't get an up-to-date Firefox to add .com, so the feature
> may already be obsolete.
>

Spamssassin seems to append .com TLD to uri link domains found

["Tobi]

Hi list

we currently see the following "issue" where SA does append .com TLD to
uri domains found in body.

Add a uri like "www.ch" to a txt body part and SA will add "ch.com" to
domains to be checked

Nov  6 15:23:52.527 [16955] dbg: uri: canonicalizing parsed uri:
http://www.ch
Nov  6 15:23:52.527 [16955] dbg: uri: cleaned uri: http://www.ch.com
Nov  6 15:23:52.527 [16955] dbg: uri: added host: www.ch.com domain: ch.com
[...]
Nov  6 15:23:52.602 [16955] dbg: async: launching
A/ch.com.multi.uribl.com for DNSBL:ch.com:multi.uribl.com

Any other hostname than "www" does not trigger that behavior. So ftp.ch
get correctly queried as ftp.ch and not ch.com

We use SA 3.4.4

SpamAssassin Server version 3.4.4
  running on Perl 5.16.3
  with SSL support (IO::Socket::SSL 1.94)
  with zlib support (Compress::Zlib 2.061)

Is that a bug or intended?

Cheers

tobi

Suggestion to FromNotReplyTo plugin in cwiki.apache.org

[Tobi]

Hi,

Maybe it would make sense to alter the code a bit here:
https://cwiki.apache.org/confluence/display/SPAMASSASSIN/FromNotReplyTo

and ensure that addresses are lower-cased before compared

> my $from = lc $msg->get( 'From:addr' );
> my $replyTo = lc $msg->get( 'Reply-To:addr' );

Cheers
--

tobi

Re: LASTEXTERNALRDNS and LASTEXTERNALHELO not set in PerMsgStatus.pm ?

[Tobi]

Henrik,

thanks a lot, can confirm your fix works in my tests :-)

Cheers

tobi

Am 28.11.19 um 11:09 schrieb Henrik K:
>
> Fixed:
> http://svn.apache.org/viewvc?view=revision=1870552
>
> On Thu, Nov 28, 2019 at 11:29:19AM +0200, Henrik K wrote:
>>
>> Trunk has already many revamps and fixes for these codes, works there.  It
>> seems 3.4 askdns has trouble using those, investigating minimal fix..
>>
>>
>> On Thu, Nov 28, 2019 at 10:20:39AM +0100, Tobi  wrote:
>>> Henrik
>>>
>>> But my current testing clearly shows that without the explicit set_tag
>>> _LASTEXTERNALRDNS_ and _LASTEXTERNALHELO_ won't be set.
>>> I'm testing this using spamassassin in debug mode to scan a testmessage.
>>> As soon as I re-add the set_tag to PerMsgStatus.pm the tags are set and
>>> tests based on that tags are run. Removing the lines from pm and debug
>>> shows that the tests are **not** run anymore.
>>>
>>> So I somewhat doubt that the set_tag "code is redundant and should be
>>> removed" :-)
>>>
>>> Using: SA 3.4.2 on centos7 / perl 5.16.3
>>>
>>>
>>> Cheers
>>>
>>> tobi
>>>
>>> Am 28.11.19 um 08:36 schrieb Henrik K:
>>>>
>>>> There is nothing missing.  All the LASTEXT* tags are already dynamically
>>>> returning functions.  If anything, that if ($lasthop) set_tag code is
>>>> completely redundant and should be removed.
>>>>
>>>>
>>>> BEGIN {
>>>> LASTEXTERNALIP => sub {
>>>>   my $pms = shift;
>>>>   my $lasthop = $pms->{relays_external}->[0];
>>>>   $lasthop ? $lasthop->{ip} : '';
>>>> },
>>>>
>>>> LASTEXTERNALRDNS => sub {
>>>>   my $pms = shift;
>>>>   my $lasthop = $pms->{relays_external}->[0];
>>>>   $lasthop ? $lasthop->{rdns} : '';
>>>> },
>>>>
>>>> LASTEXTERNALHELO => sub {
>>>>   my $pms = shift;
>>>>   my $lasthop = $pms->{relays_external}->[0];
>>>>   $lasthop ? $lasthop->{helo} : '';
>>>> },
>>>>
>>>>
>>>> On Wed, Nov 27, 2019 at 10:18:20AM -0500, Kevin A. McGrail wrote:
>>>>> After a 10 minute or so study of the issue and comparing 3.4 and trunk,
>>>>> it definitely looks like the code is missing.  I am not 100% sure your
>>>>> fix works but it's better than it currently exists :-)
>>>>>
>>>>> Have you been using that change in production?
>>>>>
>>>>> Regards,
>>>>>
>>>>> KAM
>>>>>
>>>>> On 11/27/2019 6:59 AM, Tobi  wrote:
>>>>>> Hi,
>>>>>>
>>>>>> is there any specific reason why the two tags mentioned in subject are
>>>>>> not set in SA? It took me a while to find out why an askdns test was not
>>>>>> running. The test relies on _LASTEXTERNALRDNS_ but after running with
>>>>>> --debug I found that those tags are not set by SA. Although
>>>>>> _LASTEXTERNALIP_ is set. Also the man page states that the two tags
>>>>>> should exist.
>>>>>>
>>>>>> In PerMsgStatus.pm I saw that everything is prepared for the two tags
>>>>>> but they finally not set (around line 1721). So I changed to
>>>>>>
>>>>>>> if ($lasthop) {
>>>>>>>$self->set_tag('LASTEXTERNALIP',  $lasthop->{ip});
>>>>>>>$self->set_tag('LASTEXTERNALRDNS', $lasthop->{rdns});
>>>>>>>$self->set_tag('LASTEXTERNALHELO', $lasthop->{helo});
>>>>>>>  }
>>>>>>
>>>>>> and --debug shows that the tags are now set and the askdns test is run.
>>>>>>
>>>>>> So I wonder if the code has just been forgotten or if there are deeper
>>>>>> reasons to not set the tags?
>>>>>> If no deeper reasons exist it would be nice to have those two tags set
>>>>>> as default in PerMsgStatus.pm
>>>>>>
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>> --
>>>>>>
>>>>>> tobi
>>>>>
>>>>> --
>>>>> Kevin A. McGrail
>>>>> kmcgr...@apache.org
>>>>>
>>>>> Member, Apache Software Foundation
>>>>> Chair Emeritus Apache SpamAssassin Project
>>>>> https://www.linkedin.com/in/kmcgrail - 703.798.0171
>>>>
>

Re: LASTEXTERNALRDNS and LASTEXTERNALHELO not set in PerMsgStatus.pm ?

[Tobi]

Henrik

But my current testing clearly shows that without the explicit set_tag
_LASTEXTERNALRDNS_ and _LASTEXTERNALHELO_ won't be set.
I'm testing this using spamassassin in debug mode to scan a testmessage.
As soon as I re-add the set_tag to PerMsgStatus.pm the tags are set and
tests based on that tags are run. Removing the lines from pm and debug
shows that the tests are **not** run anymore.

So I somewhat doubt that the set_tag "code is redundant and should be
removed" :-)

Using: SA 3.4.2 on centos7 / perl 5.16.3


Cheers

tobi

Am 28.11.19 um 08:36 schrieb Henrik K:
>
> There is nothing missing.  All the LASTEXT* tags are already dynamically
> returning functions.  If anything, that if ($lasthop) set_tag code is
> completely redundant and should be removed.
>
>
> BEGIN {
> LASTEXTERNALIP => sub {
>   my $pms = shift;
>   my $lasthop = $pms->{relays_external}->[0];
>   $lasthop ? $lasthop->{ip} : '';
> },
>
> LASTEXTERNALRDNS => sub {
>   my $pms = shift;
>   my $lasthop = $pms->{relays_external}->[0];
>   $lasthop ? $lasthop->{rdns} : '';
> },
>
> LASTEXTERNALHELO => sub {
>   my $pms = shift;
>   my $lasthop = $pms->{relays_external}->[0];
>   $lasthop ? $lasthop->{helo} : '';
> },
>
>
> On Wed, Nov 27, 2019 at 10:18:20AM -0500, Kevin A. McGrail wrote:
>> After a 10 minute or so study of the issue and comparing 3.4 and trunk,
>> it definitely looks like the code is missing.  I am not 100% sure your
>> fix works but it's better than it currently exists :-)
>>
>> Have you been using that change in production?
>>
>> Regards,
>>
>> KAM
>>
>> On 11/27/2019 6:59 AM, Tobi  wrote:
>>> Hi,
>>>
>>> is there any specific reason why the two tags mentioned in subject are
>>> not set in SA? It took me a while to find out why an askdns test was not
>>> running. The test relies on _LASTEXTERNALRDNS_ but after running with
>>> --debug I found that those tags are not set by SA. Although
>>> _LASTEXTERNALIP_ is set. Also the man page states that the two tags
>>> should exist.
>>>
>>> In PerMsgStatus.pm I saw that everything is prepared for the two tags
>>> but they finally not set (around line 1721). So I changed to
>>>
>>>> if ($lasthop) {
>>>>$self->set_tag('LASTEXTERNALIP',  $lasthop->{ip});
>>>>$self->set_tag('LASTEXTERNALRDNS', $lasthop->{rdns});
>>>>$self->set_tag('LASTEXTERNALHELO', $lasthop->{helo});
>>>>  }
>>>
>>> and --debug shows that the tags are now set and the askdns test is run.
>>>
>>> So I wonder if the code has just been forgotten or if there are deeper
>>> reasons to not set the tags?
>>> If no deeper reasons exist it would be nice to have those two tags set
>>> as default in PerMsgStatus.pm
>>>
>>>
>>> Cheers
>>>
>>> --
>>>
>>> tobi
>>
>> --
>> Kevin A. McGrail
>> kmcgr...@apache.org
>>
>> Member, Apache Software Foundation
>> Chair Emeritus Apache SpamAssassin Project
>> https://www.linkedin.com/in/kmcgrail - 703.798.0171
>

Re: shortcircuit on alread x-spam-flag: yes

[Tobi]

Hi Benny

yeah your links definitely show massive abuse of mta header/body checks :-)
But nonetheless mta header checks are way more performant and efficient
than such checks in a filter software. As long as the header you check
is used for a kill-shot its best place still is the mta header checks
and not in any other filter software ;-)

Cheers

tobi

Am 27.11.19 um 18:30 schrieb Benny Pedersen:
> On 2019-11-27 17:56, Philipp Ewald wrote:
>
>> we only want to trust "X-Spam-Flag: YES" or why should someone
>> (spammer, other mailserver with outgoing spamfilter) set this Flag to
>> Yes?
>
> trustness
>
> https://www.techiepark.com/tutorials/blocking-spam-using-postfix-header_checks-and-spamassassin/
> bad example on what not to do :)
>
> http://www.techiepark.com/resources/postfix-header-checks/ really want
> to make postfix a spam filter ?
>
> bettr is to use fuglu.org as a before queue content filter with then can
> reject spam :=)
>
> i have still not seen mimedefang working
>
>
>

Re: shortcircuit on alread x-spam-flag: yes

[Tobi]

Hi Philipp

> or why should someone (spammer, other mailserver with outgoing
> spamfilter) set this Flag to Yes?

I would not think about the spammers here too much but more about a
misconfigured SA on sending side? Or the admin added a fancy rbl list
which suddenly stops working and returns a hit for every query for an ip
or domain. Have been there, have seen that :-)
Thats brings us back to the FP question

Just my 5 cents: if someone trusts the spam assessment of a remote
system, then one should have the guts to reject straight-out on mta :-)
Or else ignore the spam assessment from remote.


Cheers

tobi

Am 27.11.19 um 17:56 schrieb Philipp Ewald:
> Hi Tobi,
>
> we only want to trust "X-Spam-Flag: YES" or why should someone (spammer,
> other mailserver with outgoing spamfilter) set this Flag to Yes?
>
> but like RW wrote:
>> If you want to
>> match on such a header you need to rewrite it before SA sees it.
>
> i thought shortcircuit will test before any other tests but header was
> remove before shortcircuit :(
> I have a lot to learn...
>
> Thanks for help maybe i try this again... later :-)
>
> Am 27.11.19 um 17:15 schrieb Tobi :
>> Philipp,
>>
>> Think you should ask yourself the following question: do I trust the
>> spam result from a remote server? If yes then why using a spamassassin
>> rule and not straight-out reject such mails on mta (header check)? And
>> if you do not trust the remote server then why using its spam decission
>> at all?
>>
>> Cheers
>>
>> tobi
>>
>> Am 26.11.19 um 14:06 schrieb Philipp Ewald:
>>> Hi guys,
>>>
>>> i want to bypas scanning mail if mail has already X-Spam-Flag: YES set.
>>> I found "clear_headers" in
>>> "/usr/share/spamassassin/10_default_prefs.cf".
>>>
>>> how can i override this setting? (include next update)
>>>
>>> Kind regards
>>> Philipp
>>>
>>>
>>>
>

Re: shortcircuit on alread x-spam-flag: yes

[Tobi]

Philipp,

Think you should ask yourself the following question: do I trust the
spam result from a remote server? If yes then why using a spamassassin
rule and not straight-out reject such mails on mta (header check)? And
if you do not trust the remote server then why using its spam decission
at all?

Cheers

tobi

Am 26.11.19 um 14:06 schrieb Philipp Ewald:
> Hi guys,
>
> i want to bypas scanning mail if mail has already X-Spam-Flag: YES set.
> I found "clear_headers" in "/usr/share/spamassassin/10_default_prefs.cf".
>
> how can i override this setting? (include next update)
>
> Kind regards
> Philipp
>
>
>

Re: LASTEXTERNALRDNS and LASTEXTERNALHELO not set in PerMsgStatus.pm ?

[Tobi]

Hi Kevin

sorry for sending offlist the first time :-)

> Have you been using that change in production?

Yes I changed it in my prod perl file today. Works so far for the last
couple of hours.

Issue could occur if trusted/internal networks is not set properly and
the message contains a non-smtp received header with a public IP. Like
for example could occur if the message is sent via lmtp. Then the
received header does not contain ptr info therefore _LASTEXTERNALRDNS_
cannot be set or only set with empty string. That could potentially lead
to fancy dns queries if using the tag in an askdns rule

From my point of view it would be very nice to have these two tags set
by default


Cheers

tobi

Am 27.11.19 um 16:18 schrieb Kevin A. McGrail:
> After a 10 minute or so study of the issue and comparing 3.4 and trunk,
> it definitely looks like the code is missing.  I am not 100% sure your
> fix works but it's better than it currently exists :-)
>
> Have you been using that change in production?
>
> Regards,
>
> KAM
>
> On 11/27/2019 6:59 AM, Tobi  wrote:
>> Hi,
>>
>> is there any specific reason why the two tags mentioned in subject are
>> not set in SA? It took me a while to find out why an askdns test was not
>> running. The test relies on _LASTEXTERNALRDNS_ but after running with
>> --debug I found that those tags are not set by SA. Although
>> _LASTEXTERNALIP_ is set. Also the man page states that the two tags
>> should exist.
>>
>> In PerMsgStatus.pm I saw that everything is prepared for the two tags
>> but they finally not set (around line 1721). So I changed to
>>
>>> if ($lasthop) {
>>>$self->set_tag('LASTEXTERNALIP',  $lasthop->{ip});
>>>$self->set_tag('LASTEXTERNALRDNS', $lasthop->{rdns});
>>>$self->set_tag('LASTEXTERNALHELO', $lasthop->{helo});
>>>  }
>>
>> and --debug shows that the tags are now set and the askdns test is run.
>>
>> So I wonder if the code has just been forgotten or if there are deeper
>> reasons to not set the tags?
>> If no deeper reasons exist it would be nice to have those two tags set
>> as default in PerMsgStatus.pm
>>
>>
>> Cheers
>>
>> --
>>
>> tobi
>

LASTEXTERNALRDNS and LASTEXTERNALHELO not set in PerMsgStatus.pm ?

[Tobi]

Hi,

is there any specific reason why the two tags mentioned in subject are
not set in SA? It took me a while to find out why an askdns test was not
running. The test relies on _LASTEXTERNALRDNS_ but after running with
--debug I found that those tags are not set by SA. Although
_LASTEXTERNALIP_ is set. Also the man page states that the two tags
should exist.

In PerMsgStatus.pm I saw that everything is prepared for the two tags
but they finally not set (around line 1721). So I changed to

> if ($lasthop) {
>$self->set_tag('LASTEXTERNALIP',  $lasthop->{ip});
>$self->set_tag('LASTEXTERNALRDNS', $lasthop->{rdns});
>$self->set_tag('LASTEXTERNALHELO', $lasthop->{helo});
>  }


and --debug shows that the tags are now set and the askdns test is run.

So I wonder if the code has just been forgotten or if there are deeper
reasons to not set the tags?
If no deeper reasons exist it would be nice to have those two tags set
as default in PerMsgStatus.pm


Cheers

--

tobi

Re: List of available query templates?

[Tobi]

Yes I mean the _tags_ like given below to construct dns queries with
askdns. Sorry if I was not precise enough :-)
I found several examples all over the docs but nowhere a central list of
those supported tags.

Am 04.10.19 um 15:27 schrieb Giovanni Bechis:
> On 10/4/19 3:01 PM, Bill Cole wrote:
>> On 4 Oct 2019, at 3:36, Tobi wrote:
>>
>>> Hi list
>>>
>>> is there any doc where one can find a list of supported DNS query
>>> templates?
>>
>> What does that even mean???
>>
>> SpamAssassin does many different sorts of DNS query. I am unaware of any 
>> "template" construct in SA used for its many possible DNS queries.
>>
>>
> I think the user is referring to rules such as:
> askdns __FROM_FMBLA_NEWDOM_AUTHORDOMAIN_.fresh.fmb.la. A 
> /^127\.2\.0\.2$/
>
> In Mail::SpamAssassin::Conf you have docs about what _AUTHORDOMAIN_ and other 
> tags means.
>
>  Giovanni
>

List of available query templates?

[Tobi]

Hi list

is there any doc where one can find a list of supported DNS query
templates? I mean except grep-ing through the whole source code? ;-)

Cheers

tobi

LOCALPART_IN_SUBJECT does not hit

[Tobi]

Hi list

We have the case that the LOCALPART_IN_SUBJECT rule should match upon a
message, but it doesn't. After checking the message we found that if we
replace the UTF8 encoded subject header with a plain ascii containing
the relevant string for LOCALPART_IN_SUBJECT it suddenly matches.

> Subject: =?utf-8?B?...?=

and localpart name.firstname cannot be found in subject

but if we change

> Subject: [Reminder] name.firstname Your email will

the localpart is found in subject.

We're using spamassassin 3.4.2 on CentOS 7.

We had to redact the subject and localpart for this report as they
contain customers personal details.

--

tobi

Re: Bug or feature? ;-)

[Tobi]

Thanks for pointing it out. Sorry did not get it in first point.
Changed the regex in the rule to expect the scheme too and now we get
the expected hits again.
Just one thing. Does this mean that email addresses found in body always
have a scheme (mailto://) too?

Thanks for your help and have a good one


Am 26.03.19 um 05:56 schrieb Henrik K:
> On Mon, Mar 25, 2019 at 08:42:50PM +0100, Axb wrote:
>>
>> seems to me everybody is making an effort in disregarding the fact
that the
>> URI rule was hitting on a header and imo, that should not happen.
>> This makes the whole uri behaviour even more unpredictable.
>
> As already was established, all body hits are guaranteed to have sceheme
>
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7440
>
> I see no problem
>


Am 26.03.19 um 05:56 schrieb Henrik K:
> On Mon, Mar 25, 2019 at 08:42:50PM +0100, Axb wrote:
>>
>> seems to me everybody is making an effort in disregarding the fact that the
>> URI rule was hitting on a header and imo, that should not happen.
>> This makes the whole uri behaviour even more unpredictable.
>
> As already was established, all body hits are guaranteed to have sceheme
>
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7440
>
> I see no problem
>

Re: Bug or feature? ;-)

[Tobi]

Am 25.03.19 um 15:18 schrieb Henrik K:
> On Mon, Mar 25, 2019 at 03:00:30PM +0100, Tobi  wrote:
>
> You are matching "any uri" and expect it to be "reliable"?  Perhaps consider
> first what you are trying to accomplish.  Your way will match mailto: and
> strings like perl.pl etc, but perhaps that's fine.
>

to account for mailto hits we use another rule like

uri __HAS_URI_AT/\@/

in our case we do not care if we hit perl.pl as domain in the body. In
that case the meta rule using the two __HAS rules just won't fire, which
is okay.
The meta tries to hit on that crappy "are you in the office ? Can we
transfers X EUR today"-spearphishes. Which as far as we can see
never contain a URI in body of the message (at least in our spamflow).

But now we got a case which slipped through although the criterias for
the meta to fire were given. But as they used dkim for their crap
spamassassin finds a URI in d= of the header and our meta does not fire
anymore.

So I hoped that "parse_dkim_uris = 0" would solve this but at least in
our 3.4.2 setup this config has no effect on the hit of the URI in the
dkim header

Re: Bug or feature? ;-)

[Tobi]

Hi

Am 25.03.19 um 13:25 schrieb Henrik K:
>
> Use /^https?:/ to find real uris.
>

what if the scheme is ftp (or something else) or fully missing just the URI?
Think that approach is not so much reliable ;-)

Found this config param in the docs "parse_dkim_uris" which defaults to
1. But set it to 0 in local.cf does not change anything. URI is still
taken from dkim header

Cheers

tobi




Am 25.03.19 um 13:25 schrieb Henrik K:
> On Mon, Mar 25, 2019 at 12:09:32PM +0100, Tobi  wrote:
>> Hello
>>
>> we're running spamassassin 3.4.2 and have the issue that one of our
>> rules which tests for existence of a url always sez url found for our
>> test message. Although the message body does not contain a url
>>
>> uri  __HAS_URI/\S/
>>
>> After running spamassassin -D against that message we found
>>
>> Mar 25 11:53:01.005 [7527] dbg: rules: ran uri rule __HAS_URI ==>
>> got hit: "g"
>>
>> So as the body did not contain a uri we started stripping out headers
>> until the match disappeared.
>
> Could have just used /.+/ to log it completely..
>
>> Which happend when we stripped out the DKIM header.  It seems that
>> spamassassin gets the url from this header
>>
>> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
>> d=gmail.com; s=20161025;
>> h=mime-version:from:date:message-id:subject:to;
>>
>>
>> if we replace d=gmail with d=example.com the debug tells us
>>
>> Mar 25 12:04:45.561 [8196] dbg: rules: ran uri rule __HAS_URI ==>
>> got hit: "e"
>>
>> Not sure to call it a bug or a feature but imho there should be no URI
>> found in a dkim header :-)
>
> Use /^https?:/ to find real uris.
>

Re: Bayes underperforming, HTML entities?

[Tobi]

Hi

I checked the first message on my SA and found multiple hits on
__SCC_SHORT_WORDS rule which resulted in hits on the metas

*  1.0 SCC_10_SHORT_WORD_LINES 10 lines with many short words
*  1.0 SCC_5_SHORT_WORD_LINES 5 lines with many short words
*  1.0 SCC_20_SHORT_WORD_LINES 20 lines with many short words

do you regularly perform sa-update on that box?
My bayes hit on BAYES_50

Furthermore I saw two hits on KAM rules [1]

As well as several dnsbl lookups hits (but its possible that these
listings are younger than the msg you received). On my SA the lookups
from IVM [2] (not free) hit very nice (on IPs and URIs). As well hits
from razor2


[1] https://www.pccc.com/downloads/SpamAssassin/contrib/KAM.cf
[2] https://www.invaluement.com/

Am 07.11.18 um 20:33 schrieb Amir Caspi:
> Hi all,
> 
>   In the past couple of weeks I've gotten a number of clearly-spam 
> messages that slipped past SA, and the only reason was because they were 
> getting low Bayes scores (BAYES_50 or even down to BAYES_00 or BAYES_05).  I 
> do my Bayes training manually on both ham and spam so there should not be any 
> mis-categorizations... and things worked fine until a few weeks ago, so I 
> don't know what's going on now.
> 
> Here's the magic dump:
> 
> -bash-3.2$ sa-learn --dump magic
> 0.000  0  3  0  non-token data: bayes db version
> 0.000  0 253112  0  non-token data: nspam
> 0.000  0 106767  0  non-token data: nham
> 0.000  0 150434  0  non-token data: ntokens
> 0.000  0 1536087614  0  non-token data: oldest atime
> 0.000  0 1541617125  0  non-token data: newest atime
> 0.000  0 1541614751  0  non-token data: last journal sync 
> atime
> 0.000  0 1541614749  0  non-token data: last expiry atime
> 0.000  05529600  0  non-token data: last expire atime 
> delta
> 0.000  0   1173  0  non-token data: last expire reduction 
> count
> 
> 
> I don't see any obvious problem but I'm not an expert at interpreting these...
> 
> Do I need to completely trash and rebuild my DB, or am I missing something 
> obvious?
> 
> In many cases, it would appear that these spams have either very little 
> (real) text (besides the usual attempt at Bayes poisoning) and/or are using 
> HTML-entity encoding to try to bypass Bayes.  Here are a couple of spamples:
> 
> https://pastebin.com/peiXZivJ
> https://pastebin.com/3h3r7r7j
> 
> Does SA decode HTML entities as part of normalize_charset?  If not ... can 
> this be added?
> 
> I'm using SA 3.4.1 (working on upgrading to 3.4.2 but have not had time to 
> build it yet).
> 
> Thanks!
> 
> --- Amir
> 

Re: Is there a way to perform selective full uri rbl lookups?

[Tobi]

Benny,

Maybe I don't see your point clearly ;-) But I don't want to whitelist
URIHOSTS.
Have this two rules now

urirhssub   URIBL_DOMAIN   my.rbl.tld. A 127.0.0.16
bodyURIBL_DOMAIN   eval:check_uridnsbl('MY_URIBL_DOMAIN')

askdns  URIBL_HOST _URIHOSTS_.my.rbl.tld. A 127.0.0.24

my.rbl.tld is based on mysql data which gets feeded more or less
automatically from different sources (like my own traps or external data
like phishtank etc ppt).

And I have a third rule

urirhssub   URIBL_DOMAIN_FU   my.rbl.tld. A 127.0.0.32
bodyURIBL_DOMAIN_FU   eval:check_uridnsbl('URIBL_DOMAIN_FU')
score   URIBL_DOMAIN_FU   200

where domains will be listed after too many entries in fullhost table.

Cheers

tobi


Am 19.02.2018 um 16:14 schrieb Benny Pedersen:
> Tobi skrev den 2018-02-19 14:43:
> 
>> no need for this as that case is covered by sa urirhssub queries.
>> I needed a way to perform www.sub.domain.tld AND domain.tld queries of
>> the uri www.sub.domain.tld
> 
> would you like to test?
> 
> blacklist _URIDOMAINS_
> whitelist _URIHOSTS_
> 
> :=)
> 
> if you score whitelist 50% of blacklist score there could be nice
> 
> that way spammers have higther burdon to jump over
> 
> and you dont need random listning of next subdomain spammer

Re: Is there a way to perform selective full uri rbl lookups?

[Tobi]

Am 19.02.2018 um 14:25 schrieb Benny Pedersen:
> Tobi skrev den 2018-02-19 11:45:
> add one more askdns to compensate on _URIDOMAINS_
> 

no need for this as that case is covered by sa urirhssub queries.
I needed a way to perform www.sub.domain.tld AND domain.tld queries of
the uri www.sub.domain.tld

Cheers

tobi

Re: Is there a way to perform selective full uri rbl lookups?

[Tobi]

Am 19.02.2018 um 15:04 schrieb Benny Pedersen:
> 
> yep got it, so if you only use URIHOSTS how do you know it does not miss
> in URIDOMAINS ?

I do not only use URIHOSTS but also a rhs lookup for just the domain.
So I have both bases covered :-)

Re: Is there a way to perform selective full uri rbl lookups?

[Tobi]

Am 19.02.2018 um 14:25 schrieb Benny Pedersen:
> Tobi skrev den 2018-02-19 11:45:
> add one more askdns to compensate on _URIDOMAINS_
>

no need for this as that case is covered by sa urirhssub queries.
I needed a way to perform www.sub.domain.tld AND domain.tld queries of
the uri www.sub.domain.tld against by own rbl.

Cheers

tobi

Re: Is there a way to perform selective full uri rbl lookups?

[Tobi]

Hi list

just as follow up: at least spamassassin 3.4.1 has the necessary stuff
in URIDNSBL.pm.
There _URIDOMAINS_ and _URIHOSTS_ are set so a fullhost lookup becomes a
simple one-liner

askdns  MY_FULL_TEST_URIHOSTS_.my.rbl.tld   A   127.0.0.4

which fires fullhost lookups according to spamassassin -D

Feb 19 10:44:50.821 [22500] dbg: async: calling callback on key
askdns:A:img.promio-mail.com.my.rbl.tld
Feb 19 10:44:50.821 [22500] dbg: askdns: answer received, rcode NOERROR,
query IN/A/img.promio-mail.com.my.rbl.tld, answer has 2 records

Will keep testing for some days, but this seems to be the solution for
me :-)

Cheers


tobi
Am 17.02.2018 um 12:52 schrieb Tobi:
> Hi Daniele (this time onlist, sorry for offlist I have a stupid mobile client 
> when it comes to replies to lists)
> 
> thanks a lot for your reply. As I'm really not the perl coder I think I will 
> keep it as I have my fullhost lookups currently :-)
> 
> Can anyone confirm that aux_tlds does not help if one want to perform rh 
> lookups and fulluri lookups on the same uri found?
> 
> Any chance that sa in future will support a urifullsub method to lookup 
> fullhost of an uri?
> 
> Cheers
> 
> Tobi
> 
> - Originale Nachricht -
> Von: Daniele Duca <d...@staff.spin.it>
> Gesendet: 17.02.18 - 09:04
> An: jahli...@gmx.ch, users@spamassassin.apache.org
> Betreff: Re: Is there a way to perform selective full uri rbl lookups?
> 
>> Hello,
>>
>> I do full uris dns lookups through a simple SA plugin. The core lines in 
>> the function are:
>>
>> sub check_fulluris {
>>      my ($self, $msg) = @_;
>>      my $pms = $msg->{permsgstatus};
>>      my $body = $msg->{msg}->get_pristine_body();
>>      foreach my $this_url (uniq( $body =~ 
>> /(http|https):\/\/(.*?)\//g )) {
>>
>>          # code to do dns lookups
>>
>>    }
>> }
>>
>> and in the .cf
>>
>> urirhssub   TEST_FULL_URIS     mypersonal.dnsbl.   A 127.0.0.2
>> body  TEST_FULL_URIS eval:check_fulluris('TEST_FULL_URIS')
>>
>> As for my personal reason of doing full hostnames lookups, I find it 
>> easier to mantain a rbldnsd zone with hacked websites/landing pages of 
>> marketers than to write uri rules in the .cf each time
>>
>> Hope it helps
>>
>> Daniele Duca
>>
>>
>>
>> On 16/02/2018 22:08, jahlives wrote:
>>> Hi list
>>>
>>> I'm looking for a way in spamassassin to run a full-uri-host rbl lookup
>>> for a specific rule. I do not want to discuss about sense or non-sense
>>> of full-uri-hosts lookups ;-)
>>>
>>> lets assume I have two rules which query my own rbl
>>>
>>> urirhssub HIT_DOMAINmy.rbl.tld. A 127.0.0.2
>>> bodyHIT_DOMAIN  eval:check_uridnsbl('HIT_DOMAIN')
>>>
>>> urifullsub HIT_FULL my.rbl.tld. A 127.0.0.4
>>> bodyHIT_FULLeval:check_uridnsbl('HIT_FULL')
>>>
>>> I know urifullsub does not exist, should just visualize what I try to
>>> achieve :-)
>>>
>>> now for a uri like www.sub.domain.tld both rules should be tested. The
>>> first one for domain.tld (which sa does with rh lookups) and the second
>>> one with the full-uri-host (www.sub.domain.tld)
>>>
>>> I read about aux_tlds but I think this does not help me as if I add
>>> domain.tld to aux_tlds the first query above would be fired with
>>> sub.domain.tld
>>>
>>> I thought that the second query could be solved using askdns plugin in a
>>> way like this
>>>
>>> askdns HIT_FULL _URIFULLHOST_.my.rbl.tld.   A   127.0.0.4
>>>
>>> But how to get access to urifullhost? :-)
>>>
>>> Currently I use a plugin of my antispam glue to perform the full uri
>>> host lookups on uris found. This plugin adds a X-Header upon hit on
>>> which spamassassin fires and scores.
>>> So I have a solution to this "problem" but it would be nice to do both
>>> queries from spamassassin :-)
>>>
>>> Cheers
>>>
>>> tobi
>>>
>>
> 

Re: Is there a way to perform selective full uri rbl lookups?

[Tobi]

Hi Daniele (this time onlist, sorry for offlist I have a stupid mobile client 
when it comes to replies to lists)

thanks a lot for your reply. As I'm really not the perl coder I think I will 
keep it as I have my fullhost lookups currently :-)

Can anyone confirm that aux_tlds does not help if one want to perform rh 
lookups and fulluri lookups on the same uri found?

Any chance that sa in future will support a urifullsub method to lookup 
fullhost of an uri?

Cheers

Tobi

- Originale Nachricht -
Von: Daniele Duca <d...@staff.spin.it>
Gesendet: 17.02.18 - 09:04
An: jahli...@gmx.ch, users@spamassassin.apache.org
Betreff: Re: Is there a way to perform selective full uri rbl lookups?

> Hello,
> 
> I do full uris dns lookups through a simple SA plugin. The core lines in 
> the function are:
> 
> sub check_fulluris {
>      my ($self, $msg) = @_;
>      my $pms = $msg->{permsgstatus};
>      my $body = $msg->{msg}->get_pristine_body();
>      foreach my $this_url (uniq( $body =~ 
> /(http|https):\/\/(.*?)\//g )) {
> 
>          # code to do dns lookups
> 
>    }
> }
> 
> and in the .cf
> 
> urirhssub   TEST_FULL_URIS     mypersonal.dnsbl.   A 127.0.0.2
> body  TEST_FULL_URIS eval:check_fulluris('TEST_FULL_URIS')
> 
> As for my personal reason of doing full hostnames lookups, I find it 
> easier to mantain a rbldnsd zone with hacked websites/landing pages of 
> marketers than to write uri rules in the .cf each time
> 
> Hope it helps
> 
> Daniele Duca
> 
> 
> 
> On 16/02/2018 22:08, jahlives wrote:
>> Hi list
>>
>> I'm looking for a way in spamassassin to run a full-uri-host rbl lookup
>> for a specific rule. I do not want to discuss about sense or non-sense
>> of full-uri-hosts lookups ;-)
>>
>> lets assume I have two rules which query my own rbl
>>
>> urirhssub HIT_DOMAIN my.rbl.tld. A 127.0.0.2
>> body HIT_DOMAIN  eval:check_uridnsbl('HIT_DOMAIN')
>>
>> urifullsub HIT_FULL  my.rbl.tld. A 127.0.0.4
>> body HIT_FULLeval:check_uridnsbl('HIT_FULL')
>>
>> I know urifullsub does not exist, should just visualize what I try to
>> achieve :-)
>>
>> now for a uri like www.sub.domain.tld both rules should be tested. The
>> first one for domain.tld (which sa does with rh lookups) and the second
>> one with the full-uri-host (www.sub.domain.tld)
>>
>> I read about aux_tlds but I think this does not help me as if I add
>> domain.tld to aux_tlds the first query above would be fired with
>> sub.domain.tld
>>
>> I thought that the second query could be solved using askdns plugin in a
>> way like this
>>
>> askdns HIT_FULL  _URIFULLHOST_.my.rbl.tld.   A   127.0.0.4
>>
>> But how to get access to urifullhost? :-)
>>
>> Currently I use a plugin of my antispam glue to perform the full uri
>> host lookups on uris found. This plugin adds a X-Header upon hit on
>> which spamassassin fires and scores.
>> So I have a solution to this "problem" but it would be nice to do both
>> queries from spamassassin :-)
>>
>> Cheers
>>
>> tobi
>>
> 

Re: URIBL_BLOCKED

[Tobi]

Am 15.02.2018 um 02:35 schrieb @lbutlr:
> On 2018-02-14 (09:55 MST), Tobi <jahli...@gmx.ch> wrote:
>>
>> Am 14.02.2018 um 17:16 schrieb @lbutlr:
>>> I can't imagine why i'd be over limit, my mail server is tiny.
>>
>> its not the mailserver that got blocked by limits, but the dns resolver
>> your mailserver uses!
>
> I use my own DNS on Bind 9.12, however the block error is not
appearing today, so...
>
>
>
and does your bind server use other forward servers? Or does it directly
resolve the queries from the authorative nameservers? All depends
whether you resolver is in forward mode or not. If it's in forward
mode then it sounds that the ips of those forwarders might got limited

Re: URIBL_BLOCKED

[Tobi]

Am 14.02.2018 um 17:16 schrieb @lbutlr:
> I can't imagine why i'd be over limit, my mail server is tiny.

its not the mailserver that got blocked by limits, but the dns resolver
your mailserver uses!
If you're using a 3rd party resolver (ex the ones from your provider or
8.8.8.8) you can hit the limits quite fast depending on how many other
users use the same resolver for their uribl queries.
I recommend to setup a local resolver (unbound or something similar) and
use that resolver for your mailserver(s).

Cheers

tobi

Re: Pretty good spoof of AmEx

[Tobi]

Not 100% sure about 168.100.1.4 ip but the 168.100.1.3 ip is used by the 
official postfix mailinglist. Pretty sure they should not be removed from dnswl 
:-)


- Originale Nachricht -
Von: David Jones 
Gesendet: 24.01.18 - 03:26
An: users@spamassassin.apache.org
Betreff: Re: Pretty good spoof of AmEx

> On 01/23/2018 07:11 PM, Alex wrote:
>> Hi,
>> 
>> On Tue, Jan 23, 2018 at 4:52 PM, David Jones  wrote:
>>> Here is a good example of a spoof that might get user clicks.  It didn't
>>> have good SPF or DKIM but it could have pretty easily making it look pretty
>>> clean in a default SA installation.
>>>
>>> https://pastebin.com/GTG8K56a
>>>
>>> Need to get this IP off of the HostKarma and dnswl.org whitelists if anyone
>>> from there is on this list.
>> 
> 
> Sounds like this is a shared IP with some good senders so this may need 
> to be reported to cloud9.net so they can find the source of this abuse 
> of their server.
> 
>> This appears to have hit on your side. Is this just an FYI?
>> 
> 
> Do you mean my SA (MailScanner) blocked it?  Yes it did.  Mostly due to 
> properly trained Bayes DB, DCC, Pyzor, and a local rules.  Just trying 
> to show my strategy for detecting and blocking spoofing as SPF, DKIM, 
> and DMARC are being properly implemented by companies that are common 
> targets of spoofing.
> 
> Safely whitelist_auth the Envelope-From domain and then setup 
> header/body rules to block the spoofing text.
> 
>> X-ENA-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (cached,
>> score=17.85, required 4, BAYES_99 5.20, BAYES_999 0.20,
>> 
>> Yeah, not good.
>> -2.5 RCVD_IN_HOSTKARMA_WRBL: Sender listed in HOSTKARMA-WHITE
>>   [168.100.1.4 listed in 
>> hostkarma.junkemailfilter.com]
>> -2.3 RCVD_IN_DNSWL_MED  RBL: Sender listed at http://www.dnswl.org/, 
>> medium
>>   trust [168.100.1.4 listed in list.dnswl.org]
>> 
>> Were there no EnvelopeFrom or Return-Path header?
>> 
> 
> EnvelopeFrom domain was welcome.aexp.com as you can see in the 
> Authentication-Results added by my MTA with OpenDMARC.  The legit email 
> has perfect DMARC alignment on both SPF and DKIM and they run with p=reject.
> 
> No Return-Path header in the original.
> 
>> This hits a local rule involving undisclosed-recips and/or not to my
>> domain and "urgent" messages. It also now hits pyzor and dcc
>> 
> 
> Is this Bcc'd recipients?  That can be helpful information but probably 
> not a high scoring rule unless you are combining it in a meta with other 
> hits.
> 
>> I also have a rule that adds 1.2 points to emails that hit hostkarma
>> with no domain security.
>> 
> 
> How is this a sign of spam?  Have you noticed a pattern?  I will search 
> my logs (actually run a SQL query) for this to see if you are onto 
> something here.
> 
>>> Kevin already had something similar to this in KAM.cf checking for SPF_FAIL
>>> from aexp.com but it wouldn't help with that spoofed one at the top with the
>>> "m" in the domain.
>> 
>> Should we try to do something about "american express" with a faked
>> domain (amexp.com)?
>> 
> 
> We could setup a 60_blacklist_from.cf file in the SA ruleset for 
> definite bad domains but that's probably not the best place to maintain 
> that.  It really should be in major DBLs that SA already knows to check.
> 
> -- 
> David Jones

AW: dns-blocklist aren't used but should be

[Tobi]

Use spamassassin -D 
Gesendet: 07.01.18 - 16:26
An: users@spamassassin.apache.org
Betreff: dns-blocklist aren't used but should be

> Hi.
> 
> For work I am investigating an issue where none of the dns blacklists
> are used.
> We are using the current spamassassin version and also current version
> of Net::DNS.
> 
> It is installed on a current version debian system.
> We run a local nameserver using bind.
> We invoke spamassassin via "spamassassin -t < testmail" where testmail
> is a spam mail.
> 
> The weird thing is that a "dig" command works fine on the debian system,
> so name resolving is actually working outside of spamassassin. And after
> using the dig command to check the origin of the mail: dig
> xxx.xxx.xxx.xxx.zen.spamhaus.org
> Then after using that command, spamassassin will then consider spamhaus
> when checking the testmail. Probably because the dns entry is cached for
> a while or something. It will work for some minutes. Same thing with
> other blacklists. After a dig command spamassassin will start using the
> respective rule.
> 
> What is going on? It seems to be DNS related. I've read that Net::DNS is
> responsible for dns resolving for spamassassin. How can I check if it is
> working correctly? In my /etc/resolv.conf there is only one entry:
> 127.0.0.1 since we are running a local nameserver (again: dig or host
> command work just fine for name resolving ).
> 

Re: Flakey spam email. How to filter?

[Tobi]

@Dave
you're sure that trusted_networks must be changed in case of fetching mails? I 
fetch mines from gmail too and sa always has the correct first non trusted 
relay. Without changing *_networks. With fetching you do not get an smtp 
received header so sa jumps to the next relay. And (at least from what I see in 
my gmail mails) the first smtp received header without a private ip address is 
the one that handsoff to gmail aka the one to feed to sa

Chees

tobi

- Originale Nachricht -
Von: David Jones <djo...@ena.com>
Gesendet: 11.12.17 - 17:27
An: users@spamassassin.apache.org
Betreff: Re: Flakey spam email. How to filter?

> On 12/11/2017 09:44 AM, Mark London wrote:
>> I'm getting a lot of flakey spam messages,  that don't trigger any 
>> significant spamassassin rules, even though it obviously looks really 
>> bogus.
>> 
>> Here's an example.   Any suggestions?
>> 
>> https://pastebin.com/bZUt0ThS
>> 
>> These spams are being sent to my gmail account, and then forwarded to my 
>> work address  I tried stripping off all the forwarding headers, but it 
>> doesn't trigger any RBLs
>> 
>> Thanks for any help.
>> 
>> - Mark
>> 
>> 
>> 
> 
> It's going to be very difficult to filter mail properly that has been 
> forwarded from Gmail.  Why would you want to do this anyway?  Report it 
> as Spam at Gmail and let Google block it for you and everyone else on 
> Gmail and G-Suite.
> 
> If you want to continue this mail flow and use Spamassassin, I would 
> recommend using POP to pull the email from Google and not forward it 
> which breaks a lot of stuff like SPF.  You will need to setup your 
> trusted_networks to cover all of Google's mail servers IPs listed in 
> their SPF record to get RBLs to work correctly which could be challenging.
> 
> I ran that email through my filters and it scored a 12.5 for me.  Make 
> sure you have DCC installed and working.  I realize that time has passed 
> so DCC may not have hit the original SMTP receive time but still it 
> should have scored well above 6.0 based on properly trained Bayes and 
> some other SA hits:
> 
>   0.9 DKIM_ADSP_NXDOMAIN No valid author signature and domain not in DNS
>   0.0 HTML_MESSAGE   BODY: HTML included in message
>   1.2 BAYES_50   BODY: Bayes spam probability is 40 to 60%
>  [score: 0.5000]
>   0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
>   0.8 HTML_TAG_BALANCE_HEAD  BODY: HTML has unbalanced "head" tags
>   1.5 BODY_8BITS BODY: Body includes 8 consecutive 8-bit 
> characters
>   2.2 DCC_CHECK  Detected as bulk mail by DCC (dcc-servers.net)
>   0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not 
> necessarily valid
>   0.4 HTML_MIME_NO_HTML_TAG  HTML-only message, but there is no HTML tag
>   0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
>   0.2 KAM_HUGEIMGSRC Message contains many image tags with huge 
> http urls
>   2.3 S25R_4 T_S25R: Bottom of rDNS ends w/ num, next 
> lvl has num-num
> 
> That IP of 158.69.185.128 is not listed on any RBLs so it's pretty much 
> left to SA content-based rules like DCC, Bayes, and a few others above.
> 
> -- 
> David Jones

Re: Scoring mails from "not mynetworks" but using my domain in the headers?

[Tobi]

ALL_TRUSTED should fire if msg is only transported via trusted hosts, so
you can do && !ALL_TRUSTED
But would it not be better to not accept such messages in first place
and reject them on your border mta?

Am 27.11.2017 um 13:57 schrieb Ralf Hildebrandt:
> How can I distinguish my internal networks from the evil internet in a
> spamassassin rule?
> 
> I want to give messages coming from "not mynetworks" but using my
> domain in the From: header some additional points:
> 
> header  MY_FROM   From =~ /charite.de/i
> describeMY_FROM   Sender is from charite.de
> 
> # Now you create a rule to combine them:
> meta MY_FROM_FROM_OUTSIDE  MY_FROM && HOW_DO_I_QUERY_TRUSTED_NETWORKS
> describe MY_FROM_FROM_OUTSIDE  Sender is from my domain, but comes from the 
> outside
> scoreMY_FROM_FROM_OUTSIDE  1.0
> 
> 

How does spamassassin see multiple of the same headers?

[Tobi]

Hi list

I have an "issue" with my spamassassin where a test hits although it
shouldn't
I use fuglu as mail scanning daemon which has plugins for spamassassin
and the av-scanners I use. The av-scanner-plugins set a header with the
status of its findings, so the later spamassassin plugin can test for
this header

The header of a clean message look like this

X-Virus-SophosAV-Status: Clean

and my spamassassin tests

header  __SOPHOS_HITX-Virus-SophosAV-Status !~
/^clean$/i
header  __SOPHOS_HEADER exists:X-Virus-SophosAV-Status

header  SOPHOS_EICAR_TEST   X-Virus-SophosAV-Status =~ /Infected
\(EICAR-AV-Test\)/
describeSOPHOS_EICAR_TEST   eicar test file found

metaSOPHOS_HIT  ( !SOPHOS_EICAR_TEST &&
__SOPHOS_HIT && __SOPHOS_HEADER )
describeSOPHOS_HIT  hit by sophos antivirus

The issue arises when I reprocess a message which already has the sophos
clean header through fuglu.
The fuglu plugin adds another "X-Virus-SophosAV-Status: Clean" header
(so they appear twice) but then the following spamassassin plugin
matches SOHPOS_HIT
Does spamassassin somehow concatenate the values of headers with the
same name so the regexp does not match "/^clean$/i" anymore?

Cheers

tobi

Re: Header tests shown

[Tobi]

I currently add bayes token information and relay information as headers to 
each msg processed. Especially relay information can be helpful ex if you have 
a script that parses received headers. With such headers thats much more easy, 
just look for the first untrusted hop

Re: Random word spams and wiki spams

[Tobi]

I use quite low scoring for this plugin. I know its outdated, but it does 
(imho) some helpful checks on rdns which sa cannot do alone as it relies on the 
rdns result from mta. And my mta (postfix) only logs rdns if its a fcrdns :-)
And thxs for the hint to check sa baserules first before defining own ones. 
Will do it this evening

Cheers

tobi

- Originale Nachricht -
Von: Alex <mysqlstud...@gmail.com>
Gesendet: 08.07.2017 - 05:05
An: jahli...@gmx.ch, SA Mailing list <users@spamassassin.apache.org>
Betreff: Re: Random word spams and wiki spams

> Hi,
> 
>> Without that rule it might have flown below my sa-radar.
>> Got some scoring on it by using this plugin:
>> https://github.com/eilandert/Botnet.pm
> 
> Be careful with the botnet plugin - it's terribly out of date and very
> prone to false-positives. It's just not effective anymore.

Re: Random word spams and wiki spams

[Tobi]

> typo?

Ups thats a c error :-)
I score the HAS_LIST_ UNSUB with 0.1 As I need this test to show up in sa 
headers for my dovecot sieve rules to act upon, therefore I cannot use __RULE.
I'll check the built in rules to ensure that I do not reinvent the wheel :-)

Cheers

tobi


- Originale Nachricht -
Von: Benny Pedersen <m...@junc.eu>
Gesendet: 08.07.2017 - 03:27
An: users@spamassassin.apache.org
Betreff: Re: Random word spams and wiki spams

> Tobi skrev den 2017-07-07 19:40:
> 
>>> https://pastebin.com/innRFvZt
> 
>> __HAS_LIST_ID  exists:exists:List-Id
> 
> typo ?
> 
> imho it should be exists:headername
> 
>> HAS_LIST_UNSUB exists:List-Unsubscribe
> 
> that would score 1.0, intended ?
> 
> if not change to __HAS_LIST_UNSUB
> 
> but check spamasassin own rules if that is not already defined, else you 
> really redefine it
> 
>> metaRED_PILL   (MIME_BASE64_TEXT && FROM_EXCESS_BASE64 && __HAS_URI 
>> && !__HAS_LIST_ID && !HAS_LIST_UNSUB)
> 
> ok, renema HAS_LIST_UNSUB to match above

Re: Random word spams and wiki spams

[Tobi]

Am 07.07.2017 um 19:04 schrieb Alex:
>
> I'm interested in how your system would have (or currently does)
> handle this email I received some days ago:
> https://pastebin.com/innRFvZt
>
that one triggers one of my redpill meta rules and scores at 24.1 :-)

__HAS_LIST_ID  exists:exists:List-Id
HAS_LIST_UNSUB exists:List-Unsubscribe
metaRED_PILL   (MIME_BASE64_TEXT && FROM_EXCESS_BASE64 && __HAS_URI
&& !__HAS_LIST_ID && !HAS_LIST_UNSUB)

Without that rule it might have flown below my sa-radar.
Got some scoring on it by using this plugin:
https://github.com/eilandert/Botnet.pm

and with the built in rules MIME_BASE64_TEXT and FROM_EXCESS_BASE64. As
well RCVD_DOUBLE_IP_SPAM hit on that sample

Regards

tobi

Re: updates.spamassassin.org gone?

[Tobi]

afaik updates.spamassassin.org does not need to be resolvable

The two important records for updates exist and resolve:

$dig mirrors.updates.spamassassin.org txt +short
"http://spamassassin.apache.org/updates/MIRRORED.BY;

$dig 0.4.3.updates.spamassassin.org txt +short
"1799552"

Am 06.07.2017 um 10:06 schrieb Rainer Sokoll:
> Hi,
>
> for at least the last 2 days, updates.spamassin.org does not resolve anymore:
>
> ~$ host updates.spamassassin.org.
> ~$
>
> But note:
>
> ~$ host nonexistent.spamassassin.org.
> Host nonexistent.spamassassin.org. not found: 3(NXDOMAIN)
> ~$
>
> Meaning: no updates for SA anymore?
>
> Rainer

Re: Why both DNS lookup checks fire?

[Tobi]

Problem solved :-)
After changing the urirhssub lines to

urirhssub   XXX_RCVD_MY_URIBL_DOMAIN  multi.mydomain.tld. A 
127.0.0.16
urirhssub   XXX_RCVD_MY_URIBL_HOSTmulti.mydomain.tld. A 
127.0.0.24 

only the XXX_RCVD_MY_URIBL_DOMAIN check fires

Regards

tobi

Am 01.06.2017 um 14:33 schrieb Tobi:
> Hello list
>
> I'm running Spamassassin 3.4.0 on a Centos 7 (64bit) with latest updates. 
> My goal is to have an own dnsbl list for lookups in Spamassassin. 
> The lookup zone is multi.mydomain.tld and I have the following to checks for 
> SA:
>
> urirhssub XXX_RCVD_MY_URIBL_DOMAIN  multi.mydomain.tld. A 16
> bodyXXX_RCVD_MY_URIBL_DOMAIN  
> eval:check_uridnsbl('XXX_RCVD_MY_URIBL_DOMAIN')
> tflags  XXX_RCVD_MY_URIBL_DOMAIN  net
> describeXXX_RCVD_MY_URIBL_DOMAIN  contains URI domain listed
> reuse   XXX_RCVD_MY_URIBL_DOMAIN
>
> urirhssub XXX_RCVD_MY_URIBL_HOST  multi.mydomain.tld. A 24
> bodyXXX_RCVD_MY_URIBL_HOST  
> eval:check_uridnsbl('XXX_RCVD_MY_URIBL_HOST')
> tflags  XXX_RCVD_MY_URIBL_HOST  net
> describeXXX_RCVD_MY_URIBL_HOST  contains URI host listed
> reuse   XXX_RCVD_MY_URIBL_HOST
>
> The zone returns 127.0.0.16 for domains (without any hostpart) listed and 
> 127.0.0.24 for hosts (domain + hostpart) listed
> So far so good :-)
> Problem is that both checks do fire although only 127.0.0.16 is returned by 
> lookup
>
>   *  2.3 XXX_RCVD_MY_URIBL_DOMAIN contains URI domain listed
>   *  [URIs: kelasalbaghdadi.com]
>   *  3.8 XXX_RCVD_MY_URIBL_HOST contains URI host listed
>   *  [URIs: kelasalbaghdadi.com]
>
>
> $ dig kelasalbaghdadi.com.multi.mydomain.tld
> [...]
> ;; QUESTION SECTION:
> ;kelasalbaghdadi.com.multi.mydomain.tld. IN   A
>
> ;; ANSWER SECTION:
> kelasalbaghdadi.com.multi.mydomain.tld. 6052 IN A 127.0.0.16
>
> There is no mention of 127.0.0.24 which would be required for 
> XXX_RCVD_MY_URIBL_HOST to fire.
>
> Any idea how to avoid that both checks fire up? Did I mess something up in 
> config? 
>
> Thanks for any idea on how to solve that
>
> tobi
>

Why both DNS lookup checks fire?

[Tobi]

Hello list

I'm running Spamassassin 3.4.0 on a Centos 7 (64bit) with latest updates. 
My goal is to have an own dnsbl list for lookups in Spamassassin. 
The lookup zone is multi.mydomain.tld and I have the following to checks for SA:

urirhssub   XXX_RCVD_MY_URIBL_DOMAIN  multi.mydomain.tld. A 16
bodyXXX_RCVD_MY_URIBL_DOMAIN  
eval:check_uridnsbl('XXX_RCVD_MY_URIBL_DOMAIN')
tflags  XXX_RCVD_MY_URIBL_DOMAIN  net
describeXXX_RCVD_MY_URIBL_DOMAIN  contains URI domain listed
reuse   XXX_RCVD_MY_URIBL_DOMAIN

urirhssub   XXX_RCVD_MY_URIBL_HOST  multi.mydomain.tld. A 24
bodyXXX_RCVD_MY_URIBL_HOST  
eval:check_uridnsbl('XXX_RCVD_MY_URIBL_HOST')
tflags  XXX_RCVD_MY_URIBL_HOST  net
describeXXX_RCVD_MY_URIBL_HOST  contains URI host listed
reuse   XXX_RCVD_MY_URIBL_HOST

The zone returns 127.0.0.16 for domains (without any hostpart) listed and 
127.0.0.24 for hosts (domain + hostpart) listed
So far so good :-)
Problem is that both checks do fire although only 127.0.0.16 is returned by 
lookup

*  2.3 XXX_RCVD_MY_URIBL_DOMAIN contains URI domain listed
*  [URIs: kelasalbaghdadi.com]
*  3.8 XXX_RCVD_MY_URIBL_HOST contains URI host listed
*  [URIs: kelasalbaghdadi.com]


$ dig kelasalbaghdadi.com.multi.mydomain.tld
[...]
;; QUESTION SECTION:
;kelasalbaghdadi.com.multi.mydomain.tld. IN A

;; ANSWER SECTION:
kelasalbaghdadi.com.multi.mydomain.tld. 6052 IN A   127.0.0.16

There is no mention of 127.0.0.24 which would be required for 
XXX_RCVD_MY_URIBL_HOST to fire.

Any idea how to avoid that both checks fire up? Did I mess something up in 
config? 

Thanks for any idea on how to solve that

tobi