Spam from addresses where full name mirrors left-hand side of address

[Rich Wales]

[I tried asking this question a couple of days ago, but I've seen no
signs that it made it out to the list -- possibly because the sample
e-mail addresses I included in my question might have caused it to be
flagged as spam.  So here goes again, this time with the addresses
mangled a bit.]

I see a lot of spam with "From:" lines where the left-hand side of the
address is essentially the same (modulo punctuation) as the "full name"
portion of the address.  The right-hand side, on the other hand, is a
random gibberish domain.

A few examples currently sitting in my local server's spam quarantine
(with the addresses edited so they hopefully won't trigger any spam checks):

    Adding To Human Lifespan 
    "Eliminate Fat Fast" 
    "Home Warranty Special" 
    Smartphone Screen Protector 

Two questions:

Is it *technically possible* to create a Spamassassin rule which would
match this sort of "From:" line?

And assuming it can be done, is it *worthwhile* to do it?  I do realize
some perfectly legitimate "From:" lines conform to this same pattern,
and the only way to really tell the difference may be via AI or a real
human brain.
-- 
*Rich Wales*
ri...@richw.org

Re: This sucks

[Michael Brunnbauer]

Hello Bill,

On Sun, Apr 01, 2018 at 03:55:48PM -0400, Bill Cole wrote:
> This is a critical fact. It indicates that your spamd and the spamassassin
> script you are running are definitely using different SpamAssassin
> configurations, possibly different versions of the SpamAssassin
> distribution, and or possibly even different versions of Perl.

I am very sure that there are no other versions of SpamAssassin, Perl and
Net::DNS lying around.

> Determining what config the spamassassin script is using is fairly easy:
> 'spamassassin -D generic,config,diag  --lint' will give you all the details.
> Figuring out what spamd is using is less simple (and system-specific) but
> since you've been maintaining a system by hand for a long time I expect
> you'll be able to figure out how to do so safely.

This does not sound very helpful of you so I did some debugging on my own and 
have more information:

The problem only occurs only when spamd is started in the homedir of root. If
I start it in any other directory (including subdirs of /root), Net:DNS
behaves like it should: $answer->rdatastr in dnsbl_uri in Dns.pm contains IP 
addresses in dotted quad notation, like 127.0.0.3. If I start spamd in /root,
$answer->rdatastr contains strings like "\# 4 7f03" instead. This occurs 
regardless of any -x or -u flags to spamd.

So being in /root when started changes the behavior of spamd. Is it possible
that this is a timing issue? Could "\# 4 7f03" be some unprocessed
response that would be converted to 127.0.0.3 a moment later? Or is there
some other explanation for this?

Regards,

Michael Brunnbauer

-- 
++  Michael Brunnbauer
++  netEstate GmbH
++  Geisenhausener Straße 11a
++  81379 München
++  Tel +49 89 32 19 77 80
++  Fax +49 89 32 19 77 89 
++  E-Mail bru...@netestate.de
++  http://www.netestate.de/
++
++  Sitz: München, HRB Nr.142452 (Handelsregister B München)
++  USt-IdNr. DE221033342
++  Geschäftsführer: Michael Brunnbauer, Franz Brunnbauer
++  Prokurist: Dipl. Kfm. (Univ.) Markus Hendel

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Rob McEwen]

On 4/1/2018 7:10 PM, Kevin A. McGrail wrote:
No, I don't think it's an April Fool's trick though it is possible. 
They announced this a day or 2 ago.
See 
https://www.cloudconnectcommunity.com/ccc/ls/community/g-suite-feature-ideas/post/6320666165116928 
and https://firebase.google.com/docs/dynamic-links/



I have been in talks with Google about this, sharing with them specific 
data and stats. My contact there sent me an email about this a day or 
two ago referencing that announcement - I just hadn't had time to review 
it yet. So this is for real.


The data I had compiled and shared with them that I gathered about a 
month ago... was devastating. I can't make that report public because 
the spammers were OFTEN sending a one-to-one ratio of a single 
individual google shortener per recipient address. Therefore, making 
that data public would have revealed many spamtrap addresses (both mine 
and those of 3rd party spam feeds). Another problem with that data... is 
that I was finding 10s of thousands of google shortners that were 
banging on the same dozens spammer's domains - and it would be so 
trivial for Google to look for that pattern and then just wipe out all 
of THOSE many shortners that were redirecting to one of these dozen 
spammers' domains. At the same time, the vast majority of these 
shortners were persisting for weeks and weeks with no sign that Google 
was hardly lifting a finger to terminate them. In fact, in that initial 
report, 95+% of shortners used by spammers were persisting for weeks on 
end, and 80% of those were using one of those 12 spammer' domains.


HOWEVER... I started compiling new data few days ago, and it looks MUCH 
better now. I'm not done putting that report together, but early 
indications show that something has recently changed for the better - 
but that there is still significant room for improvement, but they are 
at least headed in the right direction. But that report is still under 
construction.


So I think the pressure on Google, along with some of the data I 
provided them... might have helped? Or maybe that was just "one straw 
that broke the camel's back"? Either way, I'm happy that this seems to 
be getting fixed, or they are at least headed in the right direction.


--
Rob McEwen
https://www.invaluement.com
+1 (478) 475-9032

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Kevin A. McGrail]

No, I don't think it's an April Fool's trick though it is possible.  They
announced this a day or 2 ago.

See
https://www.cloudconnectcommunity.com/ccc/ls/community/g-suite-feature-ideas/post/6320666165116928
and https://firebase.google.com/docs/dynamic-links/

Regards,
KAM

--
Kevin A. McGrail
Asst. Treasurer & VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

On Sun, Apr 1, 2018 at 5:02 PM, Benny Pedersen  wrote:

> @lbutlr skrev den 2018-04-01 22:46:
>
>> On 2018-02-20 (08:30 MST), Rob McEwen  wrote:
>>
>>>
>>> RE: The "goo.gl
>>> " shortner is OUT OF CONTROL (+ invaluement's response)
>>>
>>
>> 
>>
>
> april fools day :=)
>

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Benny Pedersen]

@lbutlr skrev den 2018-04-01 22:46:

On 2018-02-20 (08:30 MST), Rob McEwen  wrote:


RE: The "goo.gl
" shortner is OUT OF CONTROL (+ invaluement's response)





april fools day :=)

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[@lbutlr]

On 2018-02-20 (08:30 MST), Rob McEwen  wrote:
> 
> RE: The "goo.gl
> " shortner is OUT OF CONTROL (+ invaluement's response)




-- 
Technically, Aziraphale was a Principality, but people made jokes about
that these days

Re: This sucks

[Bill Cole]

On 1 Apr 2018, at 12:26 (-0400), Michael Brunnbauer wrote:


So let's look at
my problem again: running my example spam through spamassassin gets it
marked as spam while using spamc+spamd does not.


This is a critical fact. It indicates that your spamd and the 
spamassassin script you are running are definitely using different 
SpamAssassin configurations, possibly different versions of the 
SpamAssassin distribution, and or possibly even different versions of 
Perl.


Determining what config the spamassassin script is using is fairly easy: 
'spamassassin -D generic,config,diag  --lint' will give you all the 
details. Figuring out what spamd is using is less simple (and 
system-specific) but since you've been maintaining a system by hand for 
a long time I expect you'll be able to figure out how to do so safely.



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Currently Seeking Steady Work: https://linkedin.com/in/billcole

Re: This sucks

[Michael Brunnbauer]

Hello Amir,

On Sun, Apr 01, 2018 at 01:17:03PM -0600, Amir Caspi wrote:
> On Apr 1, 2018, at 10:26 AM, Michael Brunnbauer  wrote:
> > 
> > running my example spam through spamassassin gets it marked as spam while 
> > using spamc+spamd does not.
> 
> I know this is the equivalent of ???did you plug it in??? but... did you 
> restart spamd after rebuilding Net::DNS?

Yes. Actually I'm not testing on the production system any more. I use a 
different system where I have to start spamd manually to test. So spamassassin 
and spamd use the same setup.

Regards,

Michael Brunnbauer

-- 
++  Michael Brunnbauer
++  netEstate GmbH
++  Geisenhausener Straße 11a
++  81379 München
++  Tel +49 89 32 19 77 80
++  Fax +49 89 32 19 77 89 
++  E-Mail bru...@netestate.de
++  http://www.netestate.de/
++
++  Sitz: München, HRB Nr.142452 (Handelsregister B München)
++  USt-IdNr. DE221033342
++  Geschäftsführer: Michael Brunnbauer, Franz Brunnbauer
++  Prokurist: Dipl. Kfm. (Univ.) Markus Hendel


signature.asc
Description: PGP signature

Re: This sucks

[Amir Caspi]

On Apr 1, 2018, at 10:26 AM, Michael Brunnbauer  wrote:
> 
> running my example spam through spamassassin gets it marked as spam while 
> using spamc+spamd does not.

I know this is the equivalent of “did you plug it in” but... did you restart 
spamd after rebuilding Net::DNS?

Just making sure. Sometimes it’s the obvious that still gets all of us.

--- Amir
thumbed via iPhone

Re: This sucks

[Michael Brunnbauer]

hi

On Sun, Apr 01, 2018 at 12:55:33PM -0500, David Jones wrote:
> Can you provide an example message lightly redacted via pastebin.com?

I use this message for testing: https://pastebin.com/9h9d62UW

The relay IP 185.207.8.210 is in several blacklists but spamd won't notice.
spamassassin does.

> Please
> tell us more details about your environment like OS version, Perl version,
> etc.  We know you are using spamd as the SA glue but what version of SA?

Linux Kernel 3.16.56
Glibc 2.27
Perl 5.24.1
SpamAssassin 3.4.1
Net::DNS 0.83

> How is your DNS setup?  Do you have a local recursive resolver that is not
> forwarding?

The first Nameserver in /etc/resolv.conf is on the local network and is
a non-forwarding recursive resolver. I tried this in local.cf but it does not 
change anything:

 dns_available yes
 dns_server 

>  What type and version of a local recursor are you running?  See
> this article for more details:

It's bind 9.10.4-P8

Regards,

Michael Brunnbauer

-- 
++  Michael Brunnbauer
++  netEstate GmbH
++  Geisenhausener Straße 11a
++  81379 München
++  Tel +49 89 32 19 77 80
++  Fax +49 89 32 19 77 89 
++  E-Mail bru...@netestate.de
++  http://www.netestate.de/
++
++  Sitz: München, HRB Nr.142452 (Handelsregister B München)
++  USt-IdNr. DE221033342
++  Geschäftsführer: Michael Brunnbauer, Franz Brunnbauer
++  Prokurist: Dipl. Kfm. (Univ.) Markus Hendel

Re: BODY custom rule not working if text and html parts are different?

[John Hardin]

On Sun, 1 Apr 2018, John Hardin wrote:


On Sun, 1 Apr 2018, Matus UHLAR - fantomas wrote:


On 01.04.18 05:47, Pedro David Marco wrote:

This is a problem i see oftenly...
what if the URL is only in the TEXT part  and not in the HTML?  many email 
aplications show those URLs as clickable as if they were valid HTML HREFs 
when they are not...


in this case, body rule matches, but uri does not.


I think there are hueristics to pull (non-obfuscated) URIs out of body text.


Yeah, just confirmed. A non-obfuscated URI in plain-text body part is 
recognized and extracted for uri rules.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  The reason it took so long to get Bin Laden is that it took the
  SEALs five years to swim that far into the desert.  -- anon
---
 Today: April Fools' day

Re: BODY custom rule not working if text and html parts are different?

[John Hardin]

On Sun, 1 Apr 2018, Matus UHLAR - fantomas wrote:


On 01.04.18 05:47, Pedro David Marco wrote:

This is a problem i see oftenly...
what if the URL is only in the TEXT part  and not in the HTML?  many email 
aplications show those URLs as clickable as if they were valid HTML HREFs 
when they are not...


in this case, body rule matches, but uri does not.


I think there are hueristics to pull (non-obfuscated) URIs out of body 
text.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  The reason it took so long to get Bin Laden is that it took the
  SEALs five years to swim that far into the desert.  -- anon
---
 Today: April Fools' day

Re: This sucks

[David Jones]

On 04/01/2018 11:26 AM, Michael Brunnbauer wrote:


hi all,

Reindl Harald wrote:

but your distribution sucks too when you need to "So I downgraded to
Net-DNS-0.83 today and got spamassassin working but not spamd"


I don't use a distribution and build everything myself (since I bootstrapped
my system in the 1990ies). I seldom have problems to get things running.

I deleted the .so and .pm files and directories belonging to the newer
Net::DNS in /usr/lib/perl5 before downgrading to the older one.

David Jones wrote:

What is your MTA?


qmail.


Enable greylisting


I have.

[lot's of other useful tips]

I'd like to start to improve things by getting DNS blacklist in Spamassassin
to work again. I think it would improve things drastically. So let's look at
my problem again: running my example spam through spamassassin gets it
marked as spam while using spamc+spamd does not.

Benny Pedersen wrote:


add trusted ips that should not reject to trusted_networks
or stop REJECT based on spamassassin ips blacklists


I have my trusted ips in trusted_networks. I also have checked with

  add_header all RelaysUntrusted _RELAYSUNTRUSTED_

that the relevant untrusted relays get checked. This is also clear from the
output I sent. Here is is again:

spamassassin -D looks like:

  Apr  1 15:30:18.733 [22195] dbg: dns: hit 
 127.0.0.3

spamd -D looks like:

  Apr  1 15:10:51 merlot spamd[6505]: dns: hit 
 \# 4 7f03

spamassassin reports RCVD_IN_SBL_CSS while spamd -D does not. The output
from spamassassin contains a normal IP while that from spamd prints the IP
as integer. This is extremely suspicious. I'd like to focus on that.

And sorry for being negative. It's easter sunday and I'm working because a
customer is drowning in spam - spam that would be filtered out with working
DNS blacklisting.

Regards,

Michael Brunnbauer



Can you provide an example message lightly redacted via pastebin.com? 
Please tell us more details about your environment like OS version, Perl 
version, etc.  We know you are using spamd as the SA glue but what 
version of SA?


How is your DNS setup?  Do you have a local recursive resolver that is 
not forwarding?  What type and version of a local recursor are you 
running?  See this article for more details:


https://wiki.apache.org/spamassassin/CachingNameserver

If someone can reproduce the problem with spamd then we need to open a 
bug at https://bz.apache.org/SpamAssassin/ with all the proper details 
to reproduce the problem to get a developer to take a look at this.


In the mean time, there might be other ways for you to call SA from 
qmail to get around this spamd problem.  I am not familiar with qmail 
enough to help but maybe there are others on this list that can.


--
David Jones

Re: This sucks

[Michael Brunnbauer]

hi all,

Reindl Harald wrote:
> but your distribution sucks too when you need to "So I downgraded to
> Net-DNS-0.83 today and got spamassassin working but not spamd"

I don't use a distribution and build everything myself (since I bootstrapped
my system in the 1990ies). I seldom have problems to get things running.

I deleted the .so and .pm files and directories belonging to the newer 
Net::DNS in /usr/lib/perl5 before downgrading to the older one.

David Jones wrote:
> What is your MTA?

qmail.

> Enable greylisting

I have.

[lot's of other useful tips]

I'd like to start to improve things by getting DNS blacklist in Spamassassin 
to work again. I think it would improve things drastically. So let's look at
my problem again: running my example spam through spamassassin gets it
marked as spam while using spamc+spamd does not.

Benny Pedersen wrote:

> add trusted ips that should not reject to trusted_networks
> or stop REJECT based on spamassassin ips blacklists

I have my trusted ips in trusted_networks. I also have checked with

 add_header all RelaysUntrusted _RELAYSUNTRUSTED_

that the relevant untrusted relays get checked. This is also clear from the 
output I sent. Here is is again:

spamassassin -D looks like:

 Apr  1 15:30:18.733 [22195] dbg: dns: hit  
127.0.0.3

spamd -D looks like:

 Apr  1 15:10:51 merlot spamd[6505]: dns: hit 
 \# 4 7f03

spamassassin reports RCVD_IN_SBL_CSS while spamd -D does not. The output
from spamassassin contains a normal IP while that from spamd prints the IP
as integer. This is extremely suspicious. I'd like to focus on that.

And sorry for being negative. It's easter sunday and I'm working because a
customer is drowning in spam - spam that would be filtered out with working
DNS blacklisting.

Regards,

Michael Brunnbauer

-- 
++  Michael Brunnbauer
++  netEstate GmbH
++  Geisenhausener Straße 11a
++  81379 München
++  Tel +49 89 32 19 77 80
++  Fax +49 89 32 19 77 89 
++  E-Mail bru...@netestate.de
++  http://www.netestate.de/
++
++  Sitz: München, HRB Nr.142452 (Handelsregister B München)
++  USt-IdNr. DE221033342
++  Geschäftsführer: Michael Brunnbauer, Franz Brunnbauer
++  Prokurist: Dipl. Kfm. (Univ.) Markus Hendel

Re: This sucks

[David Jones]

On 04/01/2018 09:25 AM, Michael Brunnbauer wrote:


hi

I think I lost quite a few customers in the last months because DNS-lookups
are fucked up with Spamassassin so all DNSBL tests won't trigger while not
reporting errors. A problem with newer versions of Net::DNS that has been
known for months without any consequences - like a new release. This sucks.

So I downgraded to Net-DNS-0.83 today and got spamassassin working but not
spamd.

spamassassin -D looks like:

  Apr  1 15:30:18.733 [22195] dbg: dns: hit 
 127.0.0.3

spamd -D looks like:

  Apr  1 15:10:51 merlot spamd[6505]: dns: hit 
 \# 4 7f03

One time the result is an IP as integer and one time it's a normal IP. The
integer result is not recognized and the DNSBL tests do not trigger.

What can I do?



What is your MTA?  You should do as much as possible in the MTA like RBL 
checks and other basic DNS checks.  If you are using Postfix, enable 
postscreen to help a lot with defaults.  Then enable weighted RBL checks 
in postscreen like we have mentioned often on this mailling list in the 
past year or so.  Make sure you add postwhite from github.com along with 
with the postscreen weighted RBLs.


Enable greylisting, rate limiting, connection limits, pipelining limits, 
etc. in the MTA too.  Setup a high MX that simply tempfails everything 
to attract botnets that won't retry.


Setup OpenDMARC, OpenDKIM, and policyd-spf to improve SA's ability to 
allow through trusted senders.  Add dkimwl.org rules along with other 
custom rules that have been discussed the past year or so on this 
mailing list like:


DecodeShortURLs.cf & pm
iXhash2.cf & pm
dwl.dnswl.org
dkimwl.org
KAM.cf
b.barracudacentral.org
ubl.unsubscore.com (Lashback)
score.senderscore.com
UNOFFICIAL ClamAV sigs from sanesecurity.com
Invaluement (subscription required but it's not expensive and worth 
every bit)


--
David Jones

Re: This sucks

[Benny Pedersen]

Michael Brunnbauer skrev den 2018-04-01 16:25:

hi

I think I lost quite a few customers in the last months because 
DNS-lookups
are fucked up with Spamassassin so all DNSBL tests won't trigger while 
not
reporting errors. A problem with newer versions of Net::DNS that has 
been
known for months without any consequences - like a new release. This 
sucks.


since spamassassin does NOT reject dont blame it

So I downgraded to Net-DNS-0.83 today and got spamassassin working but 
not

spamd.

spamassassin -D looks like:

 Apr  1 15:30:18.733 [22195] dbg: dns: hit
 127.0.0.3

spamd -D looks like:

 Apr  1 15:10:51 merlot spamd[6505]: dns: hit
 \# 4 7f03

One time the result is an IP as integer and one time it's a normal IP. 
The

integer result is not recognized and the DNSBL tests do not trigger.

What can I do?


say it sucks again :)

add trusted ips that should not reject to trusted_networks

or stop REJECT based on spamassassin ips blacklists

P.S.: This is the third time I try to send mail to the list - I hope it 
works

this time. I'll try without PGP sig.


poor manns dkim :=)

i uaw poardix clamav spampd opendkim opendmarc pypolicyd-spf i never 
rejected a maillist or wanted mail

This sucks

[Michael Brunnbauer]

hi

I think I lost quite a few customers in the last months because DNS-lookups
are fucked up with Spamassassin so all DNSBL tests won't trigger while not
reporting errors. A problem with newer versions of Net::DNS that has been
known for months without any consequences - like a new release. This sucks.

So I downgraded to Net-DNS-0.83 today and got spamassassin working but not
spamd.

spamassassin -D looks like:

 Apr  1 15:30:18.733 [22195] dbg: dns: hit  
127.0.0.3

spamd -D looks like:

 Apr  1 15:10:51 merlot spamd[6505]: dns: hit 
 \# 4 7f03

One time the result is an IP as integer and one time it's a normal IP. The
integer result is not recognized and the DNSBL tests do not trigger.

What can I do?

P.S.: This is the third time I try to send mail to the list - I hope it works
this time. I'll try without PGP sig.

Regards,

Michael Brunnbauer

-- 
++  Michael Brunnbauer
++  netEstate GmbH
++  Geisenhausener Straße 11a
++  81379 München
++  Tel +49 89 32 19 77 80
++  Fax +49 89 32 19 77 89 
++  E-Mail bru...@netestate.de
++  http://www.netestate.de/
++
++  Sitz: München, HRB Nr.142452 (Handelsregister B München)
++  USt-IdNr. DE221033342
++  Geschäftsführer: Michael Brunnbauer, Franz Brunnbauer
++  Prokurist: Dipl. Kfm. (Univ.) Markus Hendel

Re: BODY custom rule not working if text and html parts are different?

[Sebastian Arcus]

On 01/04/18 07:10, Matus UHLAR - fantomas wrote:

On 01.04.18 05:47, Pedro David Marco wrote:

This is a problem i see oftenly...
what if the URL is only in the TEXT part  and not in the HTML?  many 
email aplications show those URLs as clickable as if they were valid 
HTML HREFs when they are not...


in this case, body rule matches, but uri does not.


I wonder if RAWBODY would match the url both in the text part and in the 
html part? Does anybody know?

Re: BODY custom rule not working if text and html parts are different?

[Leandro]

2018-04-01 2:47 GMT-03:00 Pedro David Marco :

> This is a problem i see oftenly...
>
> what if the URL is only in the TEXT part  and not in the HTML?  many email
> aplications show those URLs as clickable as if they were valid HTML HREFs
> when they are not...
>

We have a script that can extract URLs at text part. Lines 998-1016:

https://www.dropbox.com/s/5aorrijafw5ygk0/uribl.pl?dl=0

You can use it as model to your own script or use it as is.


>
> -
> PedroD
>

Re: BODY custom rule not working if text and html parts are different?

[Matus UHLAR - fantomas]

On 01.04.18 05:47, Pedro David Marco wrote:

This is a problem i see oftenly...
what if the URL is only in the TEXT part  and not in the HTML?  many email 
aplications show those URLs as clickable as if they were valid HTML HREFs when 
they are not...


in this case, body rule matches, but uri does not.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Microsoft dick is soft to do no harm