Re: Update: mail/greyscanner to version 2.00
On Jan 07 17:06:52, skin...@britvault.co.uk wrote:
If DNS_SOCK_MAX is defined in the config,
greyscanner checks that the domain part of every sender
has an A and an MX record, and blacklists everything else.
That itself is surely a good thing, but:
(2) I am getting a lot of false negatives, such as
Jan 6 01:15:53 www greyscanner[10017]: Trapped 115.67.162.38: Mailed from
sender google.com with no MX or A
Needless to say, there is an MX and an A for google.com.
The fact that 115.67.162.38 itself does not have an A
and is not actually google's outgoing SMTP server does
not come into it: this is not checked in greyscanner.
Here's a sample of greyscanner trapped no MX lines from my logs
(most of it looks spamish, others not so.):
for line in $(fgrep greyscanner /var/log/maillog | awk '/ MX / { print $7$11
}'); do print -n $line\nPTR: ; dig -x $(print $line | cut -d: -f1) +short;
print; done
173.85.227.74:jaytronautomation.com
PTR:
37.6.249.99:hol.gr
PTR: adsl-99.37.6.249.tellas.gr.
50.193.227.41:gmail.com
PTR: 50-193-227-41-static.hfc.comcastbusiness.net.
209.85.220.50:gmail.com
PTR: mail-pa0-f50.google.com.
This is what I'm talking about:
there is an MX and A for 'gmail.com',
plus this host resolves there and back,
(and apparently is an outgoing smtp of google.com).
Yet greyscanner blacklists it, with 'no MX or A for gmail.com'.
110.175.80.66:gmail.com
PTR: 110-175-80-66.static.tpgi.com.au.
This is a spammer alright, but still,
there _is_ an MX and A for 'gmail.com',
which is all that greyscanner checks for.
Somehow those DNS lookups must be failing,
but so far I haven't found time to look into Net::DNS
Jan
Re: Update: mail/greyscanner to version 2.00
On 2014-01-08 Wed 16:04 PM |, Jan Stary wrote: This is what I'm talking about: there is an MX and A for 'gmail.com', plus this host resolves there and back, (and apparently is an outgoing smtp of google.com). Yet greyscanner blacklists it, with 'no MX or A for gmail.com'. 110.175.80.66:gmail.com PTR: 110-175-80-66.static.tpgi.com.au. This is a spammer alright, but still, there _is_ an MX and A for 'gmail.com', which is all that greyscanner checks for. Somehow those DNS lookups must be failing, Yes. This diff is not a final solution, but enables users to see syslog warnings of what would be trapped if DNS checks were fully enabled: https://bitbucket.org/bonetruck/greyscanner/pull-request/4/disable-dns-checks-by-default-enable-warn/diff but so far I haven't found time to look into Net::DNS Cheers, -- Craig Skinner | http://www.bbc.co.uk/programmes/b03mtrg9/clips
Re: Update: mail/greyscanner to version 2.00
On Dec 29 19:18:07, h...@stare.cz wrote: On Dec 29 10:55:16, j...@bonetruck.org wrote: * Jim Razmus j...@bonetruck.org [131229 10:18]: I've just released greyscanner version 2.00. Notable changes include: -config file moved from /etc to /etc/mail -now runs with privilege separation using the _greyscanner user Anyone using the external address checking feature should ensure those programs are executable by _greyscanner. The attached patch updates the port. ok to commit? jim@ Now with the correct group id. Sorry for the noise. Works well for me. Thanks! After some days of use, and discussions with Jim, I think that while the update to 2.00 is an improvement, the DNS checks in greyscanner are broken. If DNS_SOCK_MAX is defined in the config, greyscanner checks that the domain part of every sender has an A and an MX record, and blacklists everything else. That itself is surely a good thing, but: (1) the check performed is actually if ( $q[0]-qtype eq 'A' || $q[0]-qtype eq 'MX' ) i.e. a sender gets a pass if there was an A _or_ and MX for it (2) I am getting a lot of false negatives, such as Jan 3 11:43:11 www greyscanner[5816]: Trapped 82.113.55.75: Mailed from sender xnet.cz with no MX or A This is a valid sender, and xnet.cz has both MX and A. Quite a few of my valid senders got blacklisted like this. Jan 6 01:15:53 www greyscanner[10017]: Trapped 115.67.162.38: Mailed from sender google.com with no MX or A Needless to say, there is an MX and an A for google.com. The fact that 115.67.162.38 itself does not have an A and is not actually google's outgoing SMTP server does not come into it: this is not checked in greyscanner. Also, there is a hardcode limit of 4 (four) timeouts; once that happens, everything not resolved so far is blacklisted. So while the other features of greyscanner (such as MAX_SENDERS_RATIO) do a lot of good, I believe the DNS checks should be turned of by default, until fixed. Jan --- greyscanner.conf.orig Tue Jan 7 13:54:02 2014 +++ greyscanner.confTue Jan 7 13:54:08 2014 @@ -9,7 +9,7 @@ $SCAN_INTERVAL = 600; # suggestion which works for a busy site is 50. Don't crank it too high # or you'll hit maxfiles, etc. Setting this to 0 will *disable* the DNS # checks for MX and A records. -$DNS_SOCK_MAX = 50; +$DNS_SOCK_MAX = 0; # Perfom count checks on hosts with more than this many tuples $SUSPECT_TUPLES = 5;
Re: Update: mail/greyscanner to version 2.00
(1) the check performed is actually if ( $q[0]-qtype eq 'A' || $q[0]-qtype eq 'MX' ) i.e. a sender gets a pass if there was an A _or_ and MX for it Come to think of it, this might be intended. Some domains do have an MX for domain.org, and do not have an A for 'domain.org' - but do have an A for www.domain.org etc; which is perfectly OK. Right?
Re: Update: mail/greyscanner to version 2.00
Currently, the greyscanner manpage is generated by Pod::Man; Here is an mdoc(7) manpage and a diff that removes the doc from the actual Perl script. Jan .Dd $Mdocdate$ .Dt GREYSCANNER 1 .Os .Sh NAME .Nm greyscanner .Nd greytrapping daemon for spamd .Sh SYNOPSIS .Nm greyscanner .Sh DESCRIPTION .Nm complements OpenBSD .Xr spamd 8 greylisting by applying additional heuristics to the greylisted hosts. The heuristics include: .Bl -bullet -compact .It confirm senders email address is valid .It confirm recipient address(es) is valid .It confirm existence of the senders MX or A record in DNS .El and more. Offending hosts are flagged as TRAPPED in the spamd database. .Sh FILES .Pa /etc/mail/greyscanner.conf optional config file .Sh SEE ALSO .Xr spamd 8 , .Xr spamdb 8 , .Lk http://bitbucket.org/bonetruck/greyscanner .Sh HISTORY .An Bob Beck Aq Mt b...@cvs.openbsd.org created .Nm in 2006. .Pp .An Jim Razmus II Aq Mt j...@bonetruck.org revised the program, added documentation, and packaged it for .Ox OpenBSD in 2009. .Sh BUGS The DNS checks triggered by setting .Dv DNS_SOCK_MAX can give false negatives. 448,479d447 __END__ =head1 NAME greyscanner - Grey trapping daemon for OpenBSD spamd =head1 SYNOPSIS Bgreyscanner =head1 DESCRIPTION greyscanner complements OpenBSD spamd(8) greylisting by applying additional heuristics to greylisted hosts. Additional heuristics include: confirm senders email address is valid, confirm existence of the senders MX or A record in DNS, confirm recipient address(es) is valid, and more. Offending hosts are flagged as 'trapped' in the spamd database. =head1 FILES F/etc/mail/greyscanner.conf optional config file =head1 SEE ALSO spamd(8), spamdb(8), Lhttp://bitbucket.org/bonetruck/greyscanner =head1 HISTORY Bob Beck created greyscanner in 2006. Jim Razmus II revised the program, added documentation, and packaged it for OpenBSD in 2009.
Re: Update: mail/greyscanner to version 2.00
On 2014-01-07 Tue 13:56 PM |, Jan Stary wrote:
If DNS_SOCK_MAX is defined in the config,
greyscanner checks that the domain part of every sender
has an A and an MX record, and blacklists everything else.
That itself is surely a good thing, but:
(1) the check performed is actually
if ( $q[0]-qtype eq 'A' || $q[0]-qtype eq 'MX' )
i.e. a sender gets a pass if there was an A _or_ and MX for it
(2) I am getting a lot of false negatives, such as
Jan 3 11:43:11 www greyscanner[5816]: Trapped 82.113.55.75: Mailed from
sender xnet.cz with no MX or A
This is a valid sender, and xnet.cz has both MX and A.
Quite a few of my valid senders got blacklisted like this.
Maybe not directly relevant, but this IP seems to have an
invalid DNS PTR record ('/' is not a legal character):
$ dig -x 82.113.55.75 +short
75.64/26.55.113.82.in-addr.arpa.
smtp-out.xnet.cz.
Jan 6 01:15:53 www greyscanner[10017]: Trapped 115.67.162.38: Mailed from
sender google.com with no MX or A
Needless to say, there is an MX and an A for google.com.
The fact that 115.67.162.38 itself does not have an A
and is not actually google's outgoing SMTP server does
not come into it: this is not checked in greyscanner.
Not directly relevant, but like you said,
that's probably not a valid google outgoing host:
$ dig -x 115.67.162.38 +short
$ whois 115.67.162.38
[:snip: not google]
person: NSP CORE IP
address:Total Access Communication PLC
address:319 Chamchuri Square Building
address:40 Floor,
address:Phayathai Road, Pathumwan,
address:Bangkok Thailand 10330
country:TH
person: NSP CORE IP
address:Total Access Communication PLC
address:319 Chamchuri Square Building
address:40 Floor,
address:Phayathai Road, Pathumwan,
address:Bangkok Thailand 10330
country:TH
Here's a sample of greyscanner trapped no MX lines from my logs
(most of it looks spamish, others not so.):
for line in $(fgrep greyscanner /var/log/maillog | awk '/ MX / { print $7$11
}'); do print -n $line\nPTR: ; dig -x $(print $line | cut -d: -f1) +short;
print; done
173.85.227.74:jaytronautomation.com
PTR:
37.6.249.99:hol.gr
PTR: adsl-99.37.6.249.tellas.gr.
50.193.227.41:gmail.com
PTR: 50-193-227-41-static.hfc.comcastbusiness.net.
209.85.220.50:gmail.com
PTR: mail-pa0-f50.google.com.
209.85.220.44:gmail.com
PTR: mail-pa0-f44.google.com.
110.175.80.66:gmail.com
PTR: 110-175-80-66.static.tpgi.com.au.
204.68.32.6:gmail.com
PTR: tmcip6.tmcaz.com.
209.85.192.169:gmail.com
PTR: mail-pd0-f169.google.com.
86.13.0.228:betrix.co.uk
PTR: cpc14-ward9-2-0-cust227.10-2.cable.virginm.net.
72.27.229.180:yahoo.com
PTR: host-72-27-229-180.ccvn.com.
118.136.168.2:yahoo.com
PTR: fm-dyn-118-136-168-2.fast.net.id.
110.175.80.66:yahoo.com
PTR: 110-175-80-66.static.tpgi.com.au.
204.68.32.6:yahoo.com
PTR: tmcip6.tmcaz.com.
220.135.175.84:yahoo.com
PTR: 220-135-175-84.HINET-IP.hinet.net.
92.234.53.187:yahoo.com
PTR: cpc65447-grth8-2-0-cust442.16-4.cable.virginmedia.com.
173.12.79.250:yahoo.com
PTR: 173-12-79-250-miami.hfc.comcastbusiness.net.
172.56.27.86:yahoo.com
PTR:
190.147.176.31:cable.net.co
PTR: Static-IP-cr19014717631.cable.net.co.
195.137.219.217:zadzarip.my-addr.com
PTR: zadzarip.my-addr.com.
194.247.60.73:evilstone.com
PTR: 073ip.my-addr.com.
212.22.72.199:evilstone.com
PTR: arrqdzpi.my-addr.com.
24.148.237.105:mindspring.com
PTR: user-0c99rb9.cable.mindspring.com.
186.43.89.67:edatel.net.co
PTR: adsl-186-43-89-67.edatel.net.co.
201.233.68.3:epm.net.co
PTR: cable201-233-68-3.epm.net.co.
97.68.220.138:biz.bhn.net
PTR: 97-68-220-138.biz.bhn.net.
97.68.220.138:biz.bhn.net
PTR: 97-68-220-138.biz.bhn.net.
78.83.132.25:portalvedra.com
PTR: home-78-83-132-25.optinet.bg.
88.249.199.49:ttnet.com.tr
PTR: 88.249.199.49.dynamic.ttnet.com.tr.
78.83.132.25:portalvedra.com
PTR: home-78-83-132-25.optinet.bg.
46.109.196.26:dsldevice.lan
PTR:
46.109.196.26:dsldevice.lan
PTR:
94.111.78.41:speedtouch.lan
PTR: cust-41-78-111-94.dyn.as47377.net.
46.41.213.217:betsfreebets.com
PTR:
94.111.78.41:speedtouch.lan
PTR: cust-41-78-111-94.dyn.as47377.net.
46.41.213.217:betsfreebets.com
PTR:
202.29.54.58:menbl.com
PTR:
Cheers,
--
Craig Skinner | http://twitter.com/Craig_Skinner | http://linkd.in/yGqkv7
Re: Update: mail/greyscanner to version 2.00
On 2013-12-29 Sun 19:18 PM |, Jan Stary wrote: Works well for me. Me too, -- Craig Skinner | http://www.bbc.co.uk/programmes/b03mtrg9/clips
Update: mail/greyscanner to version 2.00
I've just released greyscanner version 2.00. Notable changes include: -config file moved from /etc to /etc/mail -now runs with privilege separation using the _greyscanner user Anyone using the external address checking feature should ensure those programs are executable by _greyscanner. The attached patch updates the port. ok to commit? jim@ Index: greyscanner//Makefile === RCS file: /cvs/ports/mail/greyscanner/Makefile,v retrieving revision 1.9 diff -u -p -r1.9 Makefile --- greyscanner//Makefile 5 Dec 2013 00:27:50 - 1.9 +++ greyscanner//Makefile 29 Dec 2013 16:05:45 - @@ -2,7 +2,7 @@ COMMENT = greytrapping daemon to complement OpenBSD spamd -DISTNAME = greyscanner-1.02 +DISTNAME = greyscanner-2.00 CATEGORIES = mail HOMEPAGE = https://bitbucket.org/bonetruck/greyscanner/ Index: greyscanner//distinfo === RCS file: /cvs/ports/mail/greyscanner/distinfo,v retrieving revision 1.2 diff -u -p -r1.2 distinfo --- greyscanner//distinfo 5 Dec 2013 00:27:50 - 1.2 +++ greyscanner//distinfo 29 Dec 2013 16:05:45 - @@ -1,2 +1,2 @@ -SHA256 (greyscanner-1.02.tar.gz) = +VUp/IRTWFVqVpZ9s4tYyy+zmo3XYixD1m6iYSKVSwg= -SIZE (greyscanner-1.02.tar.gz) = 6919 +SHA256 (greyscanner-2.00.tar.gz) = Hkl+Gv6Vt+kE7Np8chx4BwVHk0WEQHNhPB8tPpPQexI= +SIZE (greyscanner-2.00.tar.gz) = 7684 Index: greyscanner//pkg/PLIST === RCS file: /cvs/ports/mail/greyscanner/pkg/PLIST,v retrieving revision 1.2 diff -u -p -r1.2 PLIST --- greyscanner//pkg/PLIST 1 May 2011 20:05:54 - 1.2 +++ greyscanner//pkg/PLIST 29 Dec 2013 16:05:45 - @@ -1,4 +1,6 @@ @comment $OpenBSD: PLIST,v 1.2 2011/05/01 20:05:54 sthen Exp $ +@newgroup _greyscanner:297 +@newuser _greyscanner:729:729:daemon:greyscanner daemon:/nonexistent:/sbin/nologin @man man/man1/greyscanner.1 sbin/greyscanner share/examples/greyscanner/
Re: Update: mail/greyscanner to version 2.00
* Jim Razmus j...@bonetruck.org [131229 10:18]: I've just released greyscanner version 2.00. Notable changes include: -config file moved from /etc to /etc/mail -now runs with privilege separation using the _greyscanner user Anyone using the external address checking feature should ensure those programs are executable by _greyscanner. The attached patch updates the port. ok to commit? jim@ Now with the correct group id. Sorry for the noise. Index: greyscanner//Makefile === RCS file: /cvs/ports/mail/greyscanner/Makefile,v retrieving revision 1.9 diff -u -p -r1.9 Makefile --- greyscanner//Makefile 5 Dec 2013 00:27:50 - 1.9 +++ greyscanner//Makefile 29 Dec 2013 16:51:28 - @@ -2,7 +2,7 @@ COMMENT = greytrapping daemon to complement OpenBSD spamd -DISTNAME = greyscanner-1.02 +DISTNAME = greyscanner-2.00 CATEGORIES = mail HOMEPAGE = https://bitbucket.org/bonetruck/greyscanner/ Index: greyscanner//distinfo === RCS file: /cvs/ports/mail/greyscanner/distinfo,v retrieving revision 1.2 diff -u -p -r1.2 distinfo --- greyscanner//distinfo 5 Dec 2013 00:27:50 - 1.2 +++ greyscanner//distinfo 29 Dec 2013 16:51:28 - @@ -1,2 +1,2 @@ -SHA256 (greyscanner-1.02.tar.gz) = +VUp/IRTWFVqVpZ9s4tYyy+zmo3XYixD1m6iYSKVSwg= -SIZE (greyscanner-1.02.tar.gz) = 6919 +SHA256 (greyscanner-2.00.tar.gz) = Hkl+Gv6Vt+kE7Np8chx4BwVHk0WEQHNhPB8tPpPQexI= +SIZE (greyscanner-2.00.tar.gz) = 7684 Index: greyscanner//pkg/PLIST === RCS file: /cvs/ports/mail/greyscanner/pkg/PLIST,v retrieving revision 1.2 diff -u -p -r1.2 PLIST --- greyscanner//pkg/PLIST 1 May 2011 20:05:54 - 1.2 +++ greyscanner//pkg/PLIST 29 Dec 2013 16:51:28 - @@ -1,4 +1,6 @@ @comment $OpenBSD: PLIST,v 1.2 2011/05/01 20:05:54 sthen Exp $ +@newgroup _greyscanner:729 +@newuser _greyscanner:729:729:daemon:greyscanner daemon:/nonexistent:/sbin/nologin @man man/man1/greyscanner.1 sbin/greyscanner share/examples/greyscanner/
Re: Update: mail/greyscanner to version 2.00
On Dec 29 10:55:16, j...@bonetruck.org wrote: * Jim Razmus j...@bonetruck.org [131229 10:18]: I've just released greyscanner version 2.00. Notable changes include: -config file moved from /etc to /etc/mail -now runs with privilege separation using the _greyscanner user Anyone using the external address checking feature should ensure those programs are executable by _greyscanner. The attached patch updates the port. ok to commit? jim@ Now with the correct group id. Sorry for the noise. Works well for me. Thanks! Jan Index: greyscanner//Makefile === RCS file: /cvs/ports/mail/greyscanner/Makefile,v retrieving revision 1.9 diff -u -p -r1.9 Makefile --- greyscanner//Makefile 5 Dec 2013 00:27:50 - 1.9 +++ greyscanner//Makefile 29 Dec 2013 16:51:28 - @@ -2,7 +2,7 @@ COMMENT =greytrapping daemon to complement OpenBSD spamd -DISTNAME = greyscanner-1.02 +DISTNAME = greyscanner-2.00 CATEGORIES = mail HOMEPAGE = https://bitbucket.org/bonetruck/greyscanner/ Index: greyscanner//distinfo === RCS file: /cvs/ports/mail/greyscanner/distinfo,v retrieving revision 1.2 diff -u -p -r1.2 distinfo --- greyscanner//distinfo 5 Dec 2013 00:27:50 - 1.2 +++ greyscanner//distinfo 29 Dec 2013 16:51:28 - @@ -1,2 +1,2 @@ -SHA256 (greyscanner-1.02.tar.gz) = +VUp/IRTWFVqVpZ9s4tYyy+zmo3XYixD1m6iYSKVSwg= -SIZE (greyscanner-1.02.tar.gz) = 6919 +SHA256 (greyscanner-2.00.tar.gz) = Hkl+Gv6Vt+kE7Np8chx4BwVHk0WEQHNhPB8tPpPQexI= +SIZE (greyscanner-2.00.tar.gz) = 7684 Index: greyscanner//pkg/PLIST === RCS file: /cvs/ports/mail/greyscanner/pkg/PLIST,v retrieving revision 1.2 diff -u -p -r1.2 PLIST --- greyscanner//pkg/PLIST1 May 2011 20:05:54 - 1.2 +++ greyscanner//pkg/PLIST29 Dec 2013 16:51:28 - @@ -1,4 +1,6 @@ @comment $OpenBSD: PLIST,v 1.2 2011/05/01 20:05:54 sthen Exp $ +@newgroup _greyscanner:729 +@newuser _greyscanner:729:729:daemon:greyscanner daemon:/nonexistent:/sbin/nologin @man man/man1/greyscanner.1 sbin/greyscanner share/examples/greyscanner/
