Re: spamDB - blacklist mode
On Tue, Mar 03, 2020 at 04:46:11AM +, s...@skolma.com wrote: > Boudewijn, > Thank you for your reply, and clarification. > > The man pages for SPAMD and SPAMDB do not directly state this relationship / > behavior, and therefore I had made the assumption that spamd would capture > and feed all entries into the spamdb, in all operational modes. > ..hopefully i have not overlooked vital documentation. > hi. there's a few parts to spamd, and i think it can be initially confusing until you work out how it's put together. but there has been effort put into the spamd(8) page to make it clear what happens by default, and how that differs in blacklisting mode: SPAMD(8) System Manager's Manual SPAMD(8) ... When spamd is run in default mode, it will greylist connections from new hosts. Depending on its configuration, it may choose to blacklist the host or, if the checks described below are met, eventually whitelist it. When spamd is run in blacklist-only mode, using the -b flag, it will consult a pre-defined set of blacklist addresses to decide whether to tarpit the host or not. ... When run in default mode, connections receive the pleasantly innocuous temporary failure of: 451 Temporary failure, please try again later. This happens in the SMTP dialogue immediately after the DATA command is received from the client. spamd will use the db file in /var/db/spamd to track these connections to spamd by connecting IP address, HELO/EHLO, envelope-from, and envelope-to, or tuple for short. Hosts which connect but do not attempt to deliver mail will not generate a tuple and always be ignored. ... spamd-setup(8) should be run periodically by cron(8) to update the blacklists configured in spamd.conf(5). Use crontab(1) to uncomment the entry in root's crontab. When run in blacklist-only mode, the -b flag should be specified. ... BLACKLIST-ONLY MODE When running in default mode, the pf.conf(5) rules described above are sufficient. However when running in blacklist- only mode, a slightly modified pf.conf(5) ruleset is required, diverting any addresses found in the table to spamd. Any other addresses are passed to the real MTA. ... /var/db/spamd Greylisting database. i hope it's clear that /var/db/spamdb is used in default mode, but not in blacklist-only mode. jmc
Re: spamDB - blacklist mode
Boudewijn, Thank you for your reply, and clarification. The man pages for SPAMD and SPAMDB do not directly state this relationship / behavior, and therefore I had made the assumption that spamd would capture and feed all entries into the spamdb, in all operational modes. ..hopefully i have not overlooked vital documentation. in any event, Standard logging continues to capture as required/expected: Mar 3 00:32:54 myserver spamd[3205]: xx.xx.xx.xx: disconnected after 9 seconds. Mar 3 00:32:59 myserver spamd[3205]: xx.xx.xx.xx: disconnected after 7 seconds. Mar 3 00:33:05 myserver spamd[3205]: xx.xx.xx.xx: disconnected after 7 seconds. Mar 3 00:33:11 myserver spamd[3205]: xx.xx.xx.xx: disconnected after 7 seconds. Mar 3 00:35:22 myserver spamd[3205]: xx.xx.xx.xx: disconnected after 132 seconds. thank you all. -sub > Op Thu, 27 Feb 2020 00:19:59 +0100 schreef : > >> Questions: > >> Does the spamDB play a role at all in pure Black listing mode ? > > No, that DB is used for bookkeeping and decision-making. In blacklist-only > > mode, there is none of that. > >> Does the spamDB only get created/configured when running in Normal/Grey > >> mode ? > > It should. > >> Does is require Manual creation ? > > No. > >> Issue: > >> When Attempting to review SPAMDB entries i get an error: > >> spamdb: cannot open /var/db/spamd for reading: No such file or directory > > What kind of entries did you expect to find?
Re: spamDB - blacklist mode
Op Thu, 27 Feb 2020 00:19:59 +0100 schreef : Questions: Does the spamDB play a role at all in pure Black listing mode ? No, that DB is used for bookkeeping and decision-making. In blacklist-only mode, there is none of that. Does the spamDB only get created/configured when running in Normal/Grey mode ? It should. Does is require Manual creation ? No. Issue: When Attempting to review SPAMDB entries i get an error: spamdb: cannot open /var/db/spamd for reading: No such file or directory What kind of entries did you expect to find? Setup: [...] -- Gemaakt met Opera's e-mailprogramma: http://www.opera.com/mail/
spamDB - blacklist mode
Hi misc, Questions: Does the spamDB play a role at all in pure Black listing mode ? Does the spamDB only get created/configured when running in Normal/Grey mode ? Does is require Manual creation ? Issue: When Attempting to review SPAMDB entries i get an error: spamdb: cannot open /var/db/spamd for reading: No such file or directory Setup: My aim is to just trap & annoy spammers and build my own blacklist. Connection to this host will only ever tarpit and log. (it's the last mx record) With a fresh install of 6.6 deployed, i have setup PF and SPAMD in pure blacklist mode, following the manpage and various guides from PeterH etc.. $ grep spamd /etc/rc.conf.local spamd_flags= spamd_black=YES pf.conf # == SPAMD redirect == pass in on $ext_if proto tcp to any port smtp \ divert-to 127.0.0.1 port spamd Tarpit is working, logging is working: Feb 26 10:01:15 myhost spamd[3205]: xx.xx.xx.xx: disconnected after 6 seconds. Feb 26 10:01:20 myhost spamd[3205]: xx.xx.xx.xx: disconnected after 5 seconds. Feb 26 10:01:25 myhost spamd[3205]: xx.xx.xx.xx: disconnected after 5 seconds. Feb 26 10:01:30 myhost spamd[3205]: xx.xx.xx.xx: disconnected after 5 seconds. Feb 26 10:01:36 myhost spamd[3205]: xx.xx.xx.xx: disconnected after 6 seconds. Feb 26 10:01:42 myhost spamd[3205]: xx.xx.xx.xx: disconnected after 6 seconds. Feb 26 13:00:20 myhost spamd[3205]: xx.xx.xx.xx: disconnected after 13 seconds. Feb 26 13:00:30 myhost spamd[3205]: xx.xx.xx.xx: disconnected after 12 seconds. Feb 26 13:00:40 myhost spamd[3205]: xx.xx.xx.xx: disconnected after 12 seconds. Feb 26 13:38:28 myhost spamd[3205]: xx.xx.xx.xx: disconnected after 12 seconds. I have also run /usr/libexec/spamd-setup with default config. (no errors) thanks in advance. -sub
Re: could use some spamdb output
Op Fri, 21 Dec 2018 17:10:46 +0100 schreef Gilles Chehade : spamdb | grep -E '^(GREY|WHITE)\|' | cut -d\| -f1,2 -- Gemaakt met Opera's e-mailprogramma: http://www.opera.com/mail/
could use some spamdb output
hello misc@, If you are comfortable with sharing your spamdb output with me, it would be very helpful in confirming or not some theories I have. I do not need the sender/recipient parts, only the first two fields that disclose if the connection is in GREY or WHITE list and IP address of MX that initated the connection: $ spamdb | grep -E '^(GREY|WHITE)\|' | cut -d\| -f1,2 Do not spam misc@ with that output, send it directly to me. Thanks ! -- Gilles Chehade @poolpOrg https://www.poolp.org tip me: https://paypal.me/poolpOrg
Re: spamdb
On Tue, 15 Sep 2015 19:26:22 +0200 "Fran. J Ballesteros" <n...@lsub.org> wrote: > just FYI. our spamd indeed had problems leading to corrupt db entries > so some where never white listed. > > I changed it to use a simple in memory db and it now white lists as > it should. the change is ok for us but not for openbsd, so I didn't > submit any patch anywhere. > > the symptom of the problem is that some mails get rejected with > temporary failures forever. I would pay attention to the logs and the > db if using the stock spamd. > > Also, in case it affects, we are using the software raid. > > hth others googling for spamd. > > > El 10/9/2015, a las 15:41, Peter N. M. Hansteen <pe...@bsdly.net> > > escribió: > > > >> On Thu, Sep 10, 2015 at 03:04:26PM +0200, Fran. J Ballesteros > >> wrote: > >> > >> with 5.7 our spamdb becomes corrupt after a while. Are we the only > >> ones with this problem? Anyone else using it? > > > > using spamd with related tools including spamdb through the 5.7 > > cycle and past, yes. > > > > seeing spamdb corrupted, not that I've noticed. What are the > > symptoms more specifically? > > > > > > -- > > Peter N. M. Hansteen, member of the first RFC 1149 implementation > > team http://bsdly.blogspot.com/ http://www.bsdly.net/ > > http://www.nuug.no/ "Remember to set the evil bit on all malicious > > network traffic" delilah spamd[29949]: 85.152.224.147: disconnected > > after 42673 seconds. > I have been hitting issues with spamdb since April. Ended up disabling it as it's just a private server for 2 people so not really hurt much by being disabled. You can see the problem I reported on the mailing list here: http://marc.info/?t=14301581823=1=2 I do wonder if you are having the same issues as I did back then.
Re: spamdb
just FYI. our spamd indeed had problems leading to corrupt db entries so some where never white listed. I changed it to use a simple in memory db and it now white lists as it should. the change is ok for us but not for openbsd, so I didn't submit any patch anywhere. the symptom of the problem is that some mails get rejected with temporary failures forever. I would pay attention to the logs and the db if using the stock spamd. Also, in case it affects, we are using the software raid. hth others googling for spamd. > El 10/9/2015, a las 15:41, Peter N. M. Hansteen <pe...@bsdly.net> escribió: > >> On Thu, Sep 10, 2015 at 03:04:26PM +0200, Fran. J Ballesteros wrote: >> >> with 5.7 our spamdb becomes corrupt after a while. Are we the only ones with >> this problem? Anyone else using it? > > using spamd with related tools including spamdb through the 5.7 cycle and > past, yes. > > seeing spamdb corrupted, not that I've noticed. What are the symptoms more > specifically? > > > -- > Peter N. M. Hansteen, member of the first RFC 1149 implementation team > http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ > "Remember to set the evil bit on all malicious network traffic" > delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: spamdb
On Thu, Sep 10, 2015 at 03:04:26PM +0200, Fran. J Ballesteros wrote: > with 5.7 our spamdb becomes corrupt after a while. Are we the only ones with > this problem? Anyone else using it? using spamd with related tools including spamdb through the 5.7 cycle and past, yes. seeing spamdb corrupted, not that I've noticed. What are the symptoms more specifically? -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ "Remember to set the evil bit on all malicious network traffic" delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
spamdb
hi with 5.7 our spamdb becomes corrupt after a while. Are we the only ones with this problem? Anyone else using it?
Re: odd behaviour of spamdb
Op Mon, 13 Jul 2015 09:14:37 +0200 schreef Markus Rosjat ros...@ghweb.de:
hi there,
I have a script the following script to delete spam mx ip from the spamd
whitelist and write them in my own blacklist. After that I reload the
blacklist with spamd- setup. This seems to work but I noticed when the
same ip has another mail in the greylist the ip becomes whitelisted if
the delivery attempt is successful (which it shouldnt in the first place
because I trapped the ip and put it in my blacklist). This seems like an
odd behaviour to me, its not the end of the world but it feels kinda
wrong :)
here is the script:
ip_range=$1
for i in `spamdb | grep $ip_range | grep WHITE | awk -F | '{print $2}'`;
do
echo $i
/usr/sbin/spamdb -d $i
/usr/sbin/spamdb -a -t $i
echo $i /etc/mail/blacksheep.txt
done
/usr/libexec/spamd-setup
maybe someone give me some hints for improvement
The explicit deletion (-d) is unnecessary and may have race condition problems.
An IP that is both trapped and grey will not be re-added to the whitelist,
unless spamlogd interferes.
--
(Remove the obvious prefix to reply privately.)
Gemaakt met Opera's e-mailprogramma: http://www.opera.com/mail/
odd behaviour of spamdb
hi there,
I have a script the following script to delete spam mx ip from the spamd
whitelist and write them in my own blacklist. After that I reload the
blacklist with spamd- setup. This seems to work but I noticed when the
same ip has another mail in the greylist the ip becomes whitelisted if
the delivery attempt is successful (which it shouldnt in the first place
because I trapped the ip and put it in my blacklist). This seems like an
odd behaviour to me, its not the end of the world but it feels kinda
wrong :)
here is the script:
ip_range=$1
for i in `spamdb | grep $ip_range | grep WHITE | awk -F | '{print $2}'`;
do
echo $i
/usr/sbin/spamdb -d $i
/usr/sbin/spamdb -a -t $i
echo $i /etc/mail/blacksheep.txt
done
/usr/libexec/spamd-setup
maybe someone give me some hints for improvement
regards
--
Markus Rosjatfon: +49 351 8107223mail: ros...@ghweb.de
G+H Webservice GbR Gorzolla, Herrmann
Königsbrücker Str. 70, 01099 Dresden
http://www.ghweb.de
fon: +49 351 8107220 fax: +49 351 8107227
Bitte prüfen Sie, ob diese Mail wirklich ausgedruckt werden muss! Before you
print it, think about your responsibility and commitment to the ENVIRONMENT
Re: odd behaviour of spamdb
On 7/13/15, Markus Rosjat ros...@ghweb.de wrote:
hi there,
I have a script the following script to delete spam mx ip from the spamd
whitelist and write them in my own blacklist. After that I reload the
blacklist with spamd- setup. This seems to work but I noticed when the
same ip has another mail in the greylist the ip becomes whitelisted if
the delivery attempt is successful (which it shouldnt in the first place
because I trapped the ip and put it in my blacklist). This seems like an
odd behaviour to me, its not the end of the world but it feels kinda
wrong :)
If i understand your message correctly, after removal of the ip
from the WHITE list, it still remains in the GREY, which will be
WHITE-listed again, on the next spamd scan (60 second interval),
thus, allowing for the successful delivery.
To see this with an IP that has been WHITE-listed, but still in
the GREY, do:
$ spamdb | grep $ip
WHITE|$ip|...
GREY|$ip|...
$ spamdb -d $ip
$ spamdb | grep $ip
GREY|$ip|...
$ sleep 60
$ spamdb | grep $ip
WHITE|$ip|...
GREY|$ip|...
As a side note, your awk bit can be replaced by a `cut -d \| -f 2'.
--patrick
here is the script:
ip_range=$1
for i in `spamdb | grep $ip_range | grep WHITE | awk -F | '{print $2}'`;
do
echo $i
/usr/sbin/spamdb -d $i
/usr/sbin/spamdb -a -t $i
echo $i /etc/mail/blacksheep.txt
done
/usr/libexec/spamd-setup
maybe someone give me some hints for improvement
regards
--
Markus Rosjatfon: +49 351 8107223mail: ros...@ghweb.de
G+H Webservice GbR Gorzolla, Herrmann
Königsbrücker Str. 70, 01099 Dresden
http://www.ghweb.de
fon: +49 351 8107220 fax: +49 351 8107227
Bitte prüfen Sie, ob diese Mail wirklich ausgedruckt werden muss! Before you
print it, think about your responsibility and commitment to the ENVIRONMENT
Re: odd behaviour of spamdb
Am 13.07.2015 um 10:07 schrieb patrick keshishian:
On 7/13/15, Markus Rosjat ros...@ghweb.de wrote:
hi there,
I have a script the following script to delete spam mx ip from the spamd
whitelist and write them in my own blacklist. After that I reload the
blacklist with spamd- setup. This seems to work but I noticed when the
same ip has another mail in the greylist the ip becomes whitelisted if
the delivery attempt is successful (which it shouldnt in the first place
because I trapped the ip and put it in my blacklist). This seems like an
odd behaviour to me, its not the end of the world but it feels kinda
wrong :)
If i understand your message correctly, after removal of the ip
from the WHITE list, it still remains in the GREY, which will be
WHITE-listed again, on the next spamd scan (60 second interval),
thus, allowing for the successful delivery.
well after the first run of the script the ip should be trapped and in
my opinion the grey mail shouldnt white list the ip again. I just saw
this behaviour 2 times with the same ip because they sent the mail to 3
different mailaddresses.
To see this with an IP that has been WHITE-listed, but still in
the GREY, do:
$ spamdb | grep $ip
WHITE|$ip|...
GREY|$ip|...
$ spamdb -d $ip
$ spamdb | grep $ip
GREY|$ip|...
$ sleep 60
$ spamdb | grep $ip
WHITE|$ip|...
GREY|$ip|...
As a side note, your awk bit can be replaced by a `cut -d \| -f 2'.
thanks for the hint :)
--patrick
here is the script:
ip_range=$1
for i in `spamdb | grep $ip_range | grep WHITE | awk -F | '{print $2}'`;
do
echo $i
/usr/sbin/spamdb -d $i
/usr/sbin/spamdb -a -t $i
echo $i /etc/mail/blacksheep.txt
done
/usr/libexec/spamd-setup
maybe someone give me some hints for improvement
regards
--
Markus Rosjatfon: +49 351 8107223mail: ros...@ghweb.de
G+H Webservice GbR Gorzolla, Herrmann
Königsbrücker Str. 70, 01099 Dresden
http://www.ghweb.de
fon: +49 351 8107220 fax: +49 351 8107227
Bitte prüfen Sie, ob diese Mail wirklich ausgedruckt werden muss! Before you
print it, think about your responsibility and commitment to the ENVIRONMENT
--
Markus Rosjatfon: +49 351 8107223mail: ros...@ghweb.de
G+H Webservice GbR Gorzolla, Herrmann
Königsbrücker Str. 70, 01099 Dresden
http://www.ghweb.de
fon: +49 351 8107220 fax: +49 351 8107227
Bitte prüfen Sie, ob diese Mail wirklich ausgedruckt werden muss! Before you
print it, think about your responsibility and commitment to the ENVIRONMENT
spamdb log question
Hi there, just a simple question, is there a way to seperate the spamdb logs into logs for white-, grey- and blacklist entries? It would make the lookup make much easier when something goes wrong :) regards -- Markus Rosjatfon: +49 351 8107223mail: ros...@ghweb.de G+H Webservice GbR Gorzolla, Herrmann Königsbrücker Str. 70, 01099 Dresden http://www.ghweb.de fon: +49 351 8107220 fax: +49 351 8107227 Bitte prüfen Sie, ob diese Mail wirklich ausgedruckt werden muss! Before you print it, think about your responsibility and commitment to the ENVIRONMENT
Re: spamdb log question
hi, well I have scripts to find some evil spammer and stuff when they manage to climb over the greywall :) I was just thinking maybe there is a way with pf or so to channel the entries in diffrent logfiles. But like I said befor I can live without it :) Am 01.07.2015 um 14:58 schrieb Chris Bennett: On Wed, Jul 01, 2015 at 11:01:18AM +0200, Markus Rosjat wrote: Hi there, just a simple question, is there a way to seperate the spamdb logs into logs for white-, grey- and blacklist entries? It would make the lookup make much easier when something goes wrong :) I just use: alias G='spamdb|grep SP;spamdb|grep PED;spamdb|grep G' when I want to peek at what is NOT WHITE on the log. This lets me quickly whitelist something new, like I just signed up or registered for a site. No way for me to know its IP address beforehand. You can write a very small perl or shell program to scan the database and append to your new logs. Keep this script running in the background. Have it regularly run spamdb, split off each WHITE, GREY, BLACK line. Check your new logfiles to see if any of these lines already exists, if not, append the new entry to appropriate log. There may also be other ways to accomplish this, see what others say. Chris Bennett -- Markus Rosjatfon: +49 351 8107223mail: ros...@ghweb.de G+H Webservice GbR Gorzolla, Herrmann Königsbrücker Str. 70, 01099 Dresden http://www.ghweb.de fon: +49 351 8107220 fax: +49 351 8107227 Bitte prüfen Sie, ob diese Mail wirklich ausgedruckt werden muss! Before you print it, think about your responsibility and commitment to the ENVIRONMENT
Re: spamdb log question
On Wed, Jul 01, 2015 at 11:01:18AM +0200, Markus Rosjat wrote: Hi there, just a simple question, is there a way to seperate the spamdb logs into logs for white-, grey- and blacklist entries? It would make the lookup make much easier when something goes wrong :) I just use: alias G='spamdb|grep SP;spamdb|grep PED;spamdb|grep G' when I want to peek at what is NOT WHITE on the log. This lets me quickly whitelist something new, like I just signed up or registered for a site. No way for me to know its IP address beforehand. You can write a very small perl or shell program to scan the database and append to your new logs. Keep this script running in the background. Have it regularly run spamdb, split off each WHITE, GREY, BLACK line. Check your new logfiles to see if any of these lines already exists, if not, append the new entry to appropriate log. There may also be other ways to accomplish this, see what others say. Chris Bennett
Re: spamdb - can't delete spam db entry (Error 22)
On Mon, Apr 27, 2015, at 11:45 PM, Adam Wolk wrote: On Mon, Apr 27, 2015, at 10:52 PM, Adam Wolk wrote: On Mon, Apr 27, 2015, at 10:43 PM, Adam Wolk wrote: On Mon, Apr 27, 2015, at 10:22 PM, Todd C. Miller wrote: On Mon, 27 Apr 2015 20:06:59 +0200, Adam Wolk wrote: Apr 27 19:54:55 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Does anyone know how serious that error is (should I be worried) and what might have caused it? Error 22 is EINVAL. I'm not sure how that can happen in this case though. Have you tried restating spamd? Hi Todd, Indeed I tried restarting spamd and the issue is the same each time. With a spamd restart the error happens immediately startup: Apr 27 22:27:52 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 22:28:51 tintagel spamd[25915]: listening for incoming connections. Apr 27 22:28:51 tintagel spamd[7233]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Just noticed, that right after a previous restart it's no longer Error 22 but Error 0 Apr 27 21:50:27 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 21:51:27 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 21:52:18 tintagel spamd[8450]: listening for incoming connections. Apr 27 21:52:18 tintagel spamd[20180]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:52:25 tintagel spamd[6924]: listening for incoming connections. Apr 27 21:52:25 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:53:26 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:54:26 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:55:27 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) # ls -l /var/db/spamd -rw-r--r-- 1 _spamd _spamd 6881280 Apr 27 22:51 /var/db/spamd here's my process output limited to spamd # ps aux | grep -i spamd root 30279 0.0 3.3 68000 67956 ?? SsSun11PM1:22.68 perl: /usr/local/bin/spamd -d -u _spamdaemon -P (perl) _spamdaemon 10621 0.0 0.4 68016 8872 ?? S Sun11PM0:00.59 perl: spamd child (perl) _spamdaemon 29838 0.0 0.4 68016 8936 ?? S Sun11PM0:00.83 perl: spamd child (perl) _spamd7233 0.0 0.1 9860 1704 ?? Is10:28PM0:00.73 spamd: (pf spamd-white update) (spamd) _spamd 25915 0.0 0.3 10308 5220 ?? I 10:28PM0:00.12 spamd: [priv] (greylist) (spamd) _spamd 14894 0.0 0.0 9656 1020 ?? I 10:28PM0:00.00 spamd: (/var/db/spamd update) (spamd) root 30162 0.0 0.0 636 4 p7 R+10:52PM0:00.00 grep -i spamd (ksh) # You might also try running: $ spamdb | fgrep 66.111.4.25 Here is the output: $ spamdb | fgrep 66.111.4.25 WHITE|66.111.4.25|||1430096342|1430098533|1433208963|4|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|adam.w...@tintagel.pl|1430142855|1430145035|1430157255|4|0 to see if that entry is really in the database and if so see if spamdb -d can remove it. # spamdb -d 66.111.4.25 # echo $? 0 # spamdb | fgrep 66.111.4.25 WHITE|66.111.4.25|||1430096342|1430098533|1433208963|4|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|adam.w...@tintagel.pl|1430142855|1430145035|1430157255|4|0 - todd The weird thing is - it just started happening. I did see other weird issues like
Re: spamdb - can't delete spam db entry (Error 22)
On Mon, Apr 27, 2015, at 10:52 PM, Adam Wolk wrote: On Mon, Apr 27, 2015, at 10:43 PM, Adam Wolk wrote: On Mon, Apr 27, 2015, at 10:22 PM, Todd C. Miller wrote: On Mon, 27 Apr 2015 20:06:59 +0200, Adam Wolk wrote: Apr 27 19:54:55 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Does anyone know how serious that error is (should I be worried) and what might have caused it? Error 22 is EINVAL. I'm not sure how that can happen in this case though. Have you tried restating spamd? Hi Todd, Indeed I tried restarting spamd and the issue is the same each time. With a spamd restart the error happens immediately startup: Apr 27 22:27:52 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 22:28:51 tintagel spamd[25915]: listening for incoming connections. Apr 27 22:28:51 tintagel spamd[7233]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Just noticed, that right after a previous restart it's no longer Error 22 but Error 0 Apr 27 21:50:27 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 21:51:27 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 21:52:18 tintagel spamd[8450]: listening for incoming connections. Apr 27 21:52:18 tintagel spamd[20180]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:52:25 tintagel spamd[6924]: listening for incoming connections. Apr 27 21:52:25 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:53:26 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:54:26 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:55:27 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) # ls -l /var/db/spamd -rw-r--r-- 1 _spamd _spamd 6881280 Apr 27 22:51 /var/db/spamd here's my process output limited to spamd # ps aux | grep -i spamd root 30279 0.0 3.3 68000 67956 ?? SsSun11PM1:22.68 perl: /usr/local/bin/spamd -d -u _spamdaemon -P (perl) _spamdaemon 10621 0.0 0.4 68016 8872 ?? S Sun11PM0:00.59 perl: spamd child (perl) _spamdaemon 29838 0.0 0.4 68016 8936 ?? S Sun11PM0:00.83 perl: spamd child (perl) _spamd7233 0.0 0.1 9860 1704 ?? Is10:28PM0:00.73 spamd: (pf spamd-white update) (spamd) _spamd 25915 0.0 0.3 10308 5220 ?? I 10:28PM0:00.12 spamd: [priv] (greylist) (spamd) _spamd 14894 0.0 0.0 9656 1020 ?? I 10:28PM0:00.00 spamd: (/var/db/spamd update) (spamd) root 30162 0.0 0.0 636 4 p7 R+10:52PM0:00.00 grep -i spamd (ksh) # You might also try running: $ spamdb | fgrep 66.111.4.25 Here is the output: $ spamdb | fgrep 66.111.4.25 WHITE|66.111.4.25|||1430096342|1430098533|1433208963|4|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|adam.w...@tintagel.pl|1430142855|1430145035|1430157255|4|0 to see if that entry is really in the database and if so see if spamdb -d can remove it. # spamdb -d 66.111.4.25 # echo $? 0 # spamdb | fgrep 66.111.4.25 WHITE|66.111.4.25|||1430096342|1430098533|1433208963|4|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|adam.w...@tintagel.pl|1430142855|1430145035|1430157255|4|0 - todd The weird thing is - it just started happening. I did see other weird issues like spamdb not showing any entries in short periods of time but I assumed that it was expired hosts and they always came back after a while. I did make a copy
Re: spamdb - can't delete spam db entry (Error 22)
On Mon, Apr 27, 2015, at 10:43 PM, Adam Wolk wrote: On Mon, Apr 27, 2015, at 10:22 PM, Todd C. Miller wrote: On Mon, 27 Apr 2015 20:06:59 +0200, Adam Wolk wrote: Apr 27 19:54:55 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Does anyone know how serious that error is (should I be worried) and what might have caused it? Error 22 is EINVAL. I'm not sure how that can happen in this case though. Have you tried restating spamd? Hi Todd, Indeed I tried restarting spamd and the issue is the same each time. With a spamd restart the error happens immediately startup: Apr 27 22:27:52 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 22:28:51 tintagel spamd[25915]: listening for incoming connections. Apr 27 22:28:51 tintagel spamd[7233]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Just noticed, that right after a previous restart it's no longer Error 22 but Error 0 Apr 27 21:50:27 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 21:51:27 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 21:52:18 tintagel spamd[8450]: listening for incoming connections. Apr 27 21:52:18 tintagel spamd[20180]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:52:25 tintagel spamd[6924]: listening for incoming connections. Apr 27 21:52:25 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:53:26 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:54:26 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) Apr 27 21:55:27 tintagel spamd[3732]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 0) # ls -l /var/db/spamd -rw-r--r-- 1 _spamd _spamd 6881280 Apr 27 22:51 /var/db/spamd here's my process output limited to spamd # ps aux | grep -i spamd root 30279 0.0 3.3 68000 67956 ?? SsSun11PM1:22.68 perl: /usr/local/bin/spamd -d -u _spamdaemon -P (perl) _spamdaemon 10621 0.0 0.4 68016 8872 ?? S Sun11PM0:00.59 perl: spamd child (perl) _spamdaemon 29838 0.0 0.4 68016 8936 ?? S Sun11PM0:00.83 perl: spamd child (perl) _spamd7233 0.0 0.1 9860 1704 ?? Is10:28PM0:00.73 spamd: (pf spamd-white update) (spamd) _spamd 25915 0.0 0.3 10308 5220 ?? I 10:28PM0:00.12 spamd: [priv] (greylist) (spamd) _spamd 14894 0.0 0.0 9656 1020 ?? I 10:28PM0:00.00 spamd: (/var/db/spamd update) (spamd) root 30162 0.0 0.0 636 4 p7 R+10:52PM0:00.00 grep -i spamd (ksh) # You might also try running: $ spamdb | fgrep 66.111.4.25 Here is the output: $ spamdb | fgrep 66.111.4.25 WHITE|66.111.4.25|||1430096342|1430098533|1433208963|4|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|adam.w...@tintagel.pl|1430142855|1430145035|1430157255|4|0 to see if that entry is really in the database and if so see if spamdb -d can remove it. # spamdb -d 66.111.4.25 # echo $? 0 # spamdb | fgrep 66.111.4.25 WHITE|66.111.4.25|||1430096342|1430098533|1433208963|4|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|adam.w...@tintagel.pl|1430142855|1430145035|1430157255|4|0 - todd The weird thing is - it just started happening. I did see other weird issues like spamdb not showing any entries in short periods of time but I assumed that it was expired hosts and they always came back after a while. I did make a copy of my /var/db/spamd in case it's a corrupt db. Though nothing specific was happening with the host when the errors started. I started looking at /usr/src which makes me
Re: spamdb - can't delete spam db entry (Error 22)
On Mon, 27 Apr 2015 20:06:59 +0200, Adam Wolk wrote: Apr 27 19:54:55 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Does anyone know how serious that error is (should I be worried) and what might have caused it? Error 22 is EINVAL. I'm not sure how that can happen in this case though. Have you tried restating spamd? You might also try running: $ spamdb | fgrep 66.111.4.25 to see if that entry is really in the database and if so see if spamdb -d can remove it. - todd
Re: spamdb - can't delete spam db entry (Error 22)
On Mon, Apr 27, 2015, at 10:22 PM, Todd C. Miller wrote:
On Mon, 27 Apr 2015 20:06:59 +0200, Adam Wolk wrote:
Apr 27 19:54:55 tintagel spamd[27724]: can't delete 66.111.4.25
out1-smtp.messagingengine.com adam.w...@koparo.com
adam.w...@tintagel.pl from spamd db (Error 22)
Does anyone know how serious that error is (should I be worried) and
what might have caused it?
Error 22 is EINVAL. I'm not sure how that can happen in this case
though. Have you tried restating spamd?
Hi Todd,
Indeed I tried restarting spamd and the issue is the same each time.
With a spamd restart the error happens immediately startup:
Apr 27 22:27:52 tintagel spamd[3732]: can't delete 66.111.4.25
out1-smtp.messagingengine.com adam.w...@koparo.com
adam.w...@tintagel.pl from spamd db (Error 0)
Apr 27 22:28:51 tintagel spamd[25915]: listening for incoming
connections.
Apr 27 22:28:51 tintagel spamd[7233]: can't delete 66.111.4.25
out1-smtp.messagingengine.com adam.w...@koparo.com
adam.w...@tintagel.pl from spamd db (Error 0)
You might also try running:
$ spamdb | fgrep 66.111.4.25
Here is the output:
$ spamdb | fgrep 66.111.4.25
WHITE|66.111.4.25|||1430096342|1430098533|1433208963|4|0
GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0
GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0
GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|adam.w...@tintagel.pl|1430142855|1430145035|1430157255|4|0
to see if that entry is really in the database and if so see if
spamdb -d can remove it.
# spamdb -d 66.111.4.25
# echo $?
0
# spamdb | fgrep 66.111.4.25
WHITE|66.111.4.25|||1430096342|1430098533|1433208963|4|0
GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0
GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0
GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|adam.w...@tintagel.pl|1430142855|1430145035|1430157255|4|0
- todd
The weird thing is - it just started happening. I did see other weird
issues
like spamdb not showing any entries in short periods of time but I
assumed
that it was expired hosts and they always came back after a while.
I did make a copy of my /var/db/spamd in case it's a corrupt db. Though
nothing
specific was happening with the host when the errors started.
I started looking at /usr/src which makes me think the issue comes from:
/usr/src/libexec/spamd/grey.c
case DBC_DEL:
memset(dbk, 0, sizeof(dbk));
dbk.size = strlen(dbc-key);
dbk.data = dbc-key;
if (db-del(db, dbk, 0)) {
syslog_r(LOG_ERR, sdata,
can't delete %s from spamd db
(%m),
dbc-key);
ret = -1;
}
break;
which uses the hash version of db.h
/usr/src/lib/libc/db/hash/hash.c
hash_delete(const DB *dbp, const DBT *key,
u_int32_t flag) /* Ignored */
{
HTAB *hashp;
hashp = (HTAB *)dbp-internal;
if (flag flag != R_CURSOR) {
hashp-err = errno = EINVAL;
return (ERROR);
}
if ((hashp-flags O_ACCMODE) == O_RDONLY) {
hashp-err = errno = EPERM;
return (ERROR);
}
return (hash_access(hashp, HASH_DELETE, (DBT *)key, NULL));
}
The line with EINVAL like you correctly pointed out.
While here, why is flag marked as /* ignored */ and the error I'm
hitting looks like code which verifies if that parameter was properly
set?
The if shouldn't have a way to trigger since del is passed 0 as the flag
parameter so I'm a bit dumbfounded here. I couldn't find any other
part of the code that could result in the exact same error message.
I am trying to write a small C program to open the db file to try and
delete
the entry from a reduced use case - so far it's being going really slow
to get
a useful test case. Not sure if I will be able to whip it up in a
reasonable time frame.
Regards,
Adam
spamdb - can't delete spam db entry (Error 22)
Hi all, I spent part of the weekend setting up a private OpenSMTPD server using spamd. Everything seems to be working great but I'm now starting to see some weird behaviour. The server is running an amd64 snapshot from Apr 25 using a default spamd configuration. Does anyone know how serious that error is (should I be worried) and what might have caused it? I did my testing by sending email from this address my gmail account to my server and I am now seeing the following messages in /var/log/daemon Apr 27 19:54:55 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 19:55:56 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 19:56:57 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 19:57:58 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 19:58:58 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 19:59:59 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 20:01:01 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) Apr 27 20:02:02 tintagel spamd[27724]: can't delete 66.111.4.25 out1-smtp.messagingengine.com adam.w...@koparo.com adam.w...@tintagel.pl from spamd db (Error 22) ... and so on They keep repeating every minute. Current spamdb entry as of 19:58:58 in the timestamp # spamdb WHITE|66.111.4.25|||1430096342|1430098533|1433208963|4|0 GREY|209.85.218.48|mail-oi0-f48.google.com|netpr...@gmail.com|mulan...@tintagel.pl|1430145364|1430159764|1430159764|1|0 GREY|209.85.214.175|mail-ob0-f175.google.com|netpr...@gmail.com|mulan...@tintagel.pl|1430152660|1430167060|1430167060|1|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|209.85.214.175|mail-ob0-f175.google.com|netpr...@gmail.com|mulan...@tintagel.pl|1430152660|1430167060|1430167060|1|0 GREY|209.85.214.175|mail-ob0-f175.google.com|netpr...@gmail.com|mulan...@tintagel.pl|1430152660|1430167060|1430167060|1|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|mulan...@tintagel.pl|1430146234|1430148635|1430160634|3|0 GREY|209.85.218.41|mail-oi0-f41.google.com|netpr...@gmail.com|mulan...@tintagel.pl|1430102722|1430117122|1430117122|1|0 GREY|66.111.4.25|out1-smtp.messagingengine.com|adam.w...@koparo.com|adam.w...@tintagel.pl|1430142855|1430145035|1430157255|4|0 GREY|209.85.214.175|mail-ob0-f175.google.com|netpr...@gmail.com|mulan...@tintagel.pl|1430152660|1430167060|1430167060|1|0 spamd rc.conf.local entry: spamd_flags=-v spampd_flags=--port=10035 --relayhost=127.0.0.1:10036 --tagall -aw My pf spamd-white list # pfctl -t spamd-white -T show 66.111.4.25 My pf setup regarding spamd allow email # pass in on $ext_if proto tcp to any port smtp pass in on $ext_if proto tcp to any port submission # allow imaps port 993 pass in on $ext_if proto tcp to any port imaps # rules for spamd(8) table spamd-white persist table nospamd persist file /etc/mail/nospamd pass in on $ext_if proto tcp from any to any port smtp \ rdr-to 127.0.0.1 port spamd pass in on $ext_if proto tcp from nospamd to any port smtp pass in log on $ext_if proto tcp from spamd-white to any port smtp pass out log on $ext_if proto tcp to any port smtp Regards, -- Adam Wolk adam.w...@koparo.com
n00b spamd/spamdb question
I've finally started using spamd on a new mail server, and am seeing some results that I don't understand. (I'm also using smtpd(8) now, so this is all new software to me...) 1 - spamdb(8) shows nothing but WHITE-listed entries 2 - but spamd(8) (running with -v -G 2:4:864) logs almost every one of those WHITE IP addresses as either being greylisted or blacklisted. Shouldn't those be showing up as BLACK in spamdb, not WHITE ? Some, but not all, of these are making it through to smtpd, which then rejects most of those emails based on syntax errors, bad recipients, etc. My best guess so far is that I've got the -G passtime too low, and everyone talking to me so far is really aggressive and actually retries correctly...? This server is still only a secondary MX for the domains that get hit with lots of spam, so that's actually plausible. -- -Adam Thompson athom...@athompso.net
Re: n00b spamd/spamdb question
Date: Thu, 21 Aug 2014 13:11:23 -0500 From: Adam Thompson athom...@athompso.net To: OpenBSD-misc list misc@openbsd.org I've finally started using spamd on a new mail server, and am seeing some results that I don't understand. (I'm also using smtpd(8) now, so this is all new software to me...) That is exciting. spamd and smtpd are excellent imho. I recommend you continue to read the man pages until you have a better understanding of how they work. 1 - spamdb(8) shows nothing but WHITE-listed entries 2 - but spamd(8) (running with -v -G 2:4:864) logs almost every one of those WHITE IP addresses as either being greylisted or blacklisted. Shouldn't those be showing up as BLACK in spamdb, not WHITE ? spamdb(8) indicates 4 different entry types. BLACK is not an entry type. My best guess so far is that I've got the -G passtime too low, and everyone talking to me so far is really aggressive and actually retries correctly...? This server is still only a secondary MX for the domains that get hit with lots of spam, so that's actually plausible. I would recommend using the default spamd values. Cheers.
Re: n00b spamd/spamdb question
On 14-08-21 01:50 PM, Steven Roberts wrote:
I recommend you continue to read the man pages until you have
a better understanding of how they work.
I've already read them several times. The problem, as in all our man
pages, is that once I know what I'm looking for, it's (usually)well
documented.
However, if I don't know what I'm looking for yet, they don't tell me
what questions to ask - manpages don't solve second-order ignorance[1].
For that, I've been reading Peter H{ansteen,essler}'s blog entries and
presentations, but of course that only produces limited results until
backfilled with operational experience.
spamdb(8) indicates 4 different entry types.
BLACK is not an entry type.
Oops. I see that now. Then how do I see what IPs are blacklisted
without becoming a human version of spamd-setup(8)?
I would recommend using the default spamd values.
Easy enough. We'll see what happens when this becomes the primary MX.
Absent content filtering, I anticipate a large upswing in the amount of
spam landing in my inbox...
[1] http://www-plan.cs.colorado.edu/diwan/3308-07/p17-armour.pdf
--
-Adam Thompson
athom...@athompso.net
Re: n00b spamd/spamdb question
Oops. I see that now. Then how do I see what IPs are blacklisted without becoming a human version of spamd-setup(8)? If running spamd in default mode ... 1. spamdb(8), TRAPPED entries. 2. The spamd.conf(5) file is read by spamd-setup(8) to configure blacklists for spamd(8). I am not aware of a way to fetch the blacklist directly from spamd. I believe 'spamd-cfg' services(5) is for sending data only. P.S. The sender does not need to be blacklisted in order to prevent their spam from reaching your smtpd. Check out your /var/log/daemon to monitor connections/activity etc.
Re: remove entry from spamdb greylist
Op Tue, 13 Aug 2013 17:49:51 +0200 schreef ML mail mlnos...@yahoo.com: I am using spamd in greylisting mode and would like to delete the following entry: GREY|207.126.144.121|eu1sys200aog106.obsmtp.com|no_reply@sender|recipient@domain|1376398715|1376400232|1376413115|4|0 I tried the following command: spamdb -d 207.126.144.121 Unfortunately it does not remove the entry as it is still there. Any ideas what could be wrong? An IP address can only be used as a key for WHITE and TRAPPED entries. The spamdb(8) utility was not designed to remove GREY entries, but if you are clever, you might be able to use it for this purpose. Anyway, the important question is: what are you trying to accomplish? -- Gemaakt met Opera's revolutionaire e-mailprogramma: http://www.opera.com/mail/ (Remove the obvious prefix to reply privately.)
Re: remove entry from spamdb greylist
!-- On Tue 13.Aug'13 at 16:49:51 BST, ML mail (mlnos...@yahoo.com), wrote: Hello, I am using spamd in greylisting mode and would like to delete the following entry: GREY|207.126.144.121|eu1sys200aog106.obsmtp.com|no_reply@sender|recipient@domain|1376398715|1376400232|1376413115|4|0 I tried the following command: spamdb -d 207.126.144.121 Unfortunately it does not remove the entry as it is still there. Any ideas what could be wrong? Regards, M.L. I don't think anything is wrong. It will be removed but might not show straight away. man 8 spamdb explains very clearly. I personally use the -t switch as well as -d. spamdb -t -d ... so the address is trapped. I then usually add it to my blacklist database and/or my spam anchor using pfctl -- James Griffin: jmz at kontrol.kode5.net A4B9 E875 A18C 6E11 F46D B788 BEE6 1251 1D31 DC38
Re: remove entry from spamdb greylist
This mail server has been somehow wrongly marked as grey and would stay stuck like in this state not allowing mails flowing in. Somehow after 3 attempts it should have whitelisted this IP so it didn't. What I now did is to manually add this IP my PF spamd whitelist. The final goal being simply to allow mail coming in from this mail server. Regards, M.L. From: Boudewijn Dijkstra sp4mtr4p.boudew...@indes.com To: misc misc@openbsd.org Sent: Wednesday, August 14, 2013 12:39 PM Subject: Re: remove entry from spamdb greylist Op Tue, 13 Aug 2013 17:49:51 +0200 schreef ML mail mlnos...@yahoo.com: I am using spamd in greylisting mode and would like to delete the following entry: GREY|207.126.144.121|eu1sys200aog106.obsmtp.com|no_reply@sender|recipient@ domain|1376398715|1376400232|1376413115|4|0 I tried the following command: spamdb -d 207.126.144.121 Unfortunately it does not remove the entry as it is still there. Any ideas what could be wrong? An IP address can only be used as a key for WHITE and TRAPPED entries. The spamdb(8) utility was not designed to remove GREY entries, but if you are clever, you might be able to use it for this purpose. Anyway, the important question is: what are you trying to accomplish? -- Gemaakt met Opera's revolutionaire e-mailprogramma: http://www.opera.com/mail/ (Remove the obvious prefix to reply privately.)
Re: remove entry from spamdb greylist
On Wed, Aug 14, 2013 at 06:38:50AM -0700, ML mail wrote: This mail server has been somehow wrongly marked as grey and would stay stuck like in this state not allowing mails flowing in. Somehow after 3 attempts it should have whitelisted this IP so it didn't. What I now did is to manually add this IP my PF spamd whitelist. The final goal being simply to allow mail coming in from this mail server. spamdb -a 207.126.144.121 should set it to state WHITE, and the GREY entry (which will be overridden by the WHITE) will expire sooner or later. If it doesn't behave that way, I'd think reporting a bug would be in order, if you have sufficient logging going on at least. - P -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: remove entry from spamdb greylist
If that PF table is spamd-white, then it will get reset when you run spamd-setup(8) or reboot. Maybe a better way is to manually add this IP to the spamdb whitelist: spamdb -a 207.126.144.121 In this case the grey entry will be ignored and stay in the database until it expires. Or, even better, regularly add/update all OBSMTP addresses in your whitelist via SPF. Search the archives for people who made scripts for this kind of thing. Greylisting is great, but some mail delivery networks need a little help in the form of manual whitelisting of their netblocks. Op Wed, 14 Aug 2013 15:38:50 +0200 schreef ML mail mlnos...@yahoo.com: This mail server has been somehow wrongly marked as grey and would stay stuck like in this state not allowing mails flowing in. Somehow after 3 attempts it should have whitelisted this IP so it didn't. What I now did is to manually add this IP my PF spamd whitelist. The final goal being simply to allow mail coming in from this mail server. Regards, M.L. From: Boudewijn Dijkstra sp4mtr4p.boudew...@indes.com To: misc misc@openbsd.org Sent: Wednesday, August 14, 2013 12:39 PM Subject: Re: remove entry from spamdb greylist Op Tue, 13 Aug 2013 17:49:51 +0200 schreef ML mail mlnos...@yahoo.com: I am using spamd in greylisting mode and would like to delete the following entry: GREY|207.126.144.121|eu1sys200aog106.obsmtp.com|no_reply@sender|recipient@ domain|1376398715|1376400232|1376413115|4|0 I tried the following command: spamdb -d 207.126.144.121 Unfortunately it does not remove the entry as it is still there. Any ideas what could be wrong? An IP address can only be used as a key for WHITE and TRAPPED entries. The spamdb(8) utility was not designed to remove GREY entries, but if you are clever, you might be able to use it for this purpose. Anyway, the important question is: what are you trying to accomplish? -- Gemaakt met Opera's revolutionaire e-mailprogramma: http://www.opera.com/mail/ (Remove the obvious prefix to reply privately.)
Re: remove entry from spamdb greylist
Dear Peter, Thanks for your input too! Actually yesterday I have also tried afterwards to do a spamdb -a and as I didn't see any immediate effect (IP still listed under GREY), I simply assumed that it didn't work. From your mail I understand that it stays for a while as GREY until it expires, so maybe I wasn't patient enough. Anyhow today as I checked this IP is listed under WHITE so it looks like it finally worked as I wanted. Regards, M.L. From: Peter N. M. Hansteen pe...@bsdly.net To: ML mail mlnos...@yahoo.com Cc: misc misc@openbsd.org Sent: Wednesday, August 14, 2013 4:07 PM Subject: Re: remove entry from spamdb greylist On Wed, Aug 14, 2013 at 06:38:50AM -0700, ML mail wrote: This mail server has been somehow wrongly marked as grey and would stay stuck like in this state not allowing mails flowing in. Somehow after 3 attempts it should have whitelisted this IP so it didn't. What I now did is to manually add this IP my PF spamd whitelist. The final goal being simply to allow mail coming in from this mail server. spamdb -a 207.126.144.121 should set it to state WHITE, and the GREY entry (which will be overridden by the WHITE) will expire sooner or later. If it doesn't behave that way, I'd think reporting a bug would be in order, if you have sufficient logging going on at least. - P -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
remove entry from spamdb greylist
Hello, I am using spamd in greylisting mode and would like to delete the following entry: GREY|207.126.144.121|eu1sys200aog106.obsmtp.com|no_reply@sender|recipient@domain|1376398715|1376400232|1376413115|4|0 I tried the following command: spamdb -d 207.126.144.121 Unfortunately it does not remove the entry as it is still there. Any ideas what could be wrong? Regards, M.L.
Re: spamdb [-t] -x hours -a keys
Hi!
Sorry.
Emilio Lucena
--- spamdb.c.orig Sat May 17 07:48:06 2008
+++ spamdb.cMon Jul 18 08:04:29 2011
@@ -39,10 +39,10 @@
#define SPAMTRAP 2
intdblist(DB *);
-intdbupdate(DB *, char *, int, int);
+intdbupdate(DB *, char *, int, int, time_t);
int
-dbupdate(DB *db, char *ip, int add, int type)
+dbupdate(DB *db, char *ip, int add, int type, time_t exp)
{
DBT dbk, dbd;
struct gdatagd;
@@ -97,10 +97,10 @@
switch (type) {
case WHITE:
gd.pass = now;
- gd.expire = now + WHITEEXP;
+ gd.expire = now + exp;
break;
case TRAPHIT:
- gd.expire = now + TRAPEXP;
+ gd.expire = now + exp;
gd.pcount = -1;
break;
case SPAMTRAP:
@@ -139,10 +139,10 @@
switch (type) {
case WHITE:
gd.pass = now;
- gd.expire = now + WHITEEXP;
+ gd.expire = now + exp;
break;
case TRAPHIT:
- gd.expire = now + TRAPEXP;
+ gd.expire = now + exp;
gd.pcount = -1;
break;
case SPAMTRAP:
@@ -261,7 +261,7 @@
static int
usage(void)
{
- fprintf(stderr, usage: %s [[-Tt] -a keys] [[-Tt] -d keys]\n,
__progname);
+ fprintf(stderr, usage: %s [[-Tt] -x hours -a keys] [[-Tt] -d keys]\n,
__progname);
exit(1);
/* NOTREACHED */
}
@@ -270,10 +270,12 @@
main(int argc, char **argv)
{
int i, ch, action = 0, type = WHITE, r = 0, c = 0;
+ time_t exp = 0;
HASHINFOhashinfo;
DB *db;
+ const char *errstr;
- while ((ch = getopt(argc, argv, adtT)) != -1) {
+ while ((ch = getopt(argc, argv, adtTx:)) != -1) {
switch (ch) {
case 'a':
action = 1;
@@ -287,6 +289,14 @@
case 'T':
type = SPAMTRAP;
break;
+case 'x':
+ /* limit exp to 2160 hours (90 days) */
+ exp = strtonum(optarg, 1, (24 * 90), errstr);
+ if (errstr)
+ usage();
+ /* convert to seconds from hours */
+ exp *= (60 * 60);
+ break;
default:
usage();
break;
@@ -294,7 +304,7 @@
}
argc -= optind;
argv += optind;
- if (action == 0 type != WHITE)
+ if ((action == 0 type != WHITE) || (action != 1 exp != 0))
usage();
memset(hashinfo, 0, sizeof(hashinfo));
@@ -314,10 +324,12 @@
case 0:
return dblist(db);
case 1:
+ if (exp == 0)
+ exp = type ? TRAPEXP : WHITEEXP;
for (i=0; iargc; i++)
if (argv[i][0] != '\0') {
c++;
- r += dbupdate(db, argv[i], 1, type);
+ r += dbupdate(db, argv[i], 1, type, exp);
}
if (c == 0)
errx(2, no addresses specified);
@@ -326,7 +338,7 @@
for (i=0; iargc; i++)
if (argv[i][0] != '\0') {
c++;
- r += dbupdate(db, argv[i], 0, type);
+ r += dbupdate(db, argv[i], 0, type, exp);
}
if (c == 0)
errx(2, no addresses specified);
spamdb [-t] -x hours -a keys
Hi,
I've been using OpenBSD spamd for a while now and I find it
really great.
One feature I wanted to try was the ability to trap and
whitelist hosts for different periods of time, depending
on the host's behaviour. As you well know, there are some
really bad guys out there.
So I wrote this little patch that allows the user to
specify how many hours the keys should be trapped or
whitelisted.
I've being using it for some time, and I thought someone
else might have the same need.
Regards,
Emilio Lucena
--- spamdb.c.origSat May 17 07:48:06 2008
+++ spamdb.cMon Jul 18 08:04:29 2011
@@ -39,10 +39,10 @@
#define SPAMTRAP 2
intdblist(DB *);
-intdbupdate(DB *, char *, int, int);
+intdbupdate(DB *, char *, int, int, time_t);
int
-dbupdate(DB *db, char *ip, int add, int type)
+dbupdate(DB *db, char *ip, int add, int type, time_t exp)
{
DBTdbk, dbd;
struct gdatagd;
@@ -97,10 +97,10 @@
switch (type) {
case WHITE:
gd.pass = now;
-gd.expire = now + WHITEEXP;
+gd.expire = now + exp;
break;
case TRAPHIT:
-gd.expire = now + TRAPEXP;
+gd.expire = now + exp;
gd.pcount = -1;
break;
case SPAMTRAP:
@@ -139,10 +139,10 @@
switch (type) {
case WHITE:
gd.pass = now;
-gd.expire = now + WHITEEXP;
+gd.expire = now + exp;
break;
case TRAPHIT:
-gd.expire = now + TRAPEXP;
+gd.expire = now + exp;
gd.pcount = -1;
break;
case SPAMTRAP:
@@ -261,7 +261,7 @@
static int
usage(void)
{
-fprintf(stderr, usage: %s [[-Tt] -a keys] [[-Tt] -d keys]\n,
__progname);
+fprintf(stderr, usage: %s [[-Tt] -x hours -a keys] [[-Tt] -d
keys]\n, __progname);
exit(1);
/* NOTREACHED */
}
@@ -270,10 +270,12 @@
main(int argc, char **argv)
{
int i, ch, action = 0, type = WHITE, r = 0, c = 0;
+time_t exp = 0;
HASHINFOhashinfo;
DB*db;
+const char *errstr;
-while ((ch = getopt(argc, argv, adtT)) != -1) {
+while ((ch = getopt(argc, argv, adtTx:)) != -1) {
switch (ch) {
case 'a':
action = 1;
@@ -287,6 +289,14 @@
case 'T':
type = SPAMTRAP;
break;
+case 'x':
+/* limit exp to 2160 hours (90 days) */
+exp = strtonum(optarg, 1, (24 * 90),errstr);
+if (errstr)
+usage();
+/* convert to seconds from hours */
+exp *= (60 * 60);
+break;
default:
usage();
break;
@@ -294,7 +304,7 @@
}
argc -= optind;
argv += optind;
-if (action == 0 type != WHITE)
+if ((action == 0 type != WHITE) || (action != 1 exp != 0))
usage();
memset(hashinfo, 0, sizeof(hashinfo));
@@ -314,10 +324,12 @@
case 0:
return dblist(db);
case 1:
+if (exp == 0)
+exp = type ? TRAPEXP : WHITEEXP;
for (i=0; iargc; i++)
if (argv[i][0] != '\0') {
c++;
-r += dbupdate(db, argv[i], 1, type);
+r += dbupdate(db, argv[i], 1, type, exp);
}
if (c == 0)
errx(2, no addresses specified);
@@ -326,7 +338,7 @@
for (i=0; iargc; i++)
if (argv[i][0] != '\0') {
c++;
-r += dbupdate(db, argv[i], 0, type);
+r += dbupdate(db, argv[i], 0, type, exp);
}
if (c == 0)
errx(2, no addresses specified);
Re: spamdb [-t] -x hours -a keys
I'm not personally interested in this, but your mailer broke the diff.
Fix and resend if you want serious feedback on this.
/Alexander
On 07/30/11 14:51, Emilio Lucena wrote:
Hi,
I've been using OpenBSD spamd for a while now and I find it
really great.
One feature I wanted to try was the ability to trap and
whitelist hosts for different periods of time, depending
on the host's behaviour. As you well know, there are some
really bad guys out there.
So I wrote this little patch that allows the user to
specify how many hours the keys should be trapped or
whitelisted.
I've being using it for some time, and I thought someone
else might have the same need.
Regards,
Emilio Lucena
--- spamdb.c.origSat May 17 07:48:06 2008
+++ spamdb.cMon Jul 18 08:04:29 2011
@@ -39,10 +39,10 @@
#define SPAMTRAP 2
intdblist(DB *);
-intdbupdate(DB *, char *, int, int);
+intdbupdate(DB *, char *, int, int, time_t);
int
-dbupdate(DB *db, char *ip, int add, int type)
+dbupdate(DB *db, char *ip, int add, int type, time_t exp)
{
DBTdbk, dbd;
struct gdatagd;
@@ -97,10 +97,10 @@
switch (type) {
case WHITE:
gd.pass = now;
-gd.expire = now + WHITEEXP;
+gd.expire = now + exp;
break;
case TRAPHIT:
-gd.expire = now + TRAPEXP;
+gd.expire = now + exp;
gd.pcount = -1;
break;
case SPAMTRAP:
@@ -139,10 +139,10 @@
switch (type) {
case WHITE:
gd.pass = now;
-gd.expire = now + WHITEEXP;
+gd.expire = now + exp;
break;
case TRAPHIT:
-gd.expire = now + TRAPEXP;
+gd.expire = now + exp;
gd.pcount = -1;
break;
case SPAMTRAP:
@@ -261,7 +261,7 @@
static int
usage(void)
{
-fprintf(stderr, usage: %s [[-Tt] -a keys] [[-Tt] -d keys]\n,
__progname);
+fprintf(stderr, usage: %s [[-Tt] -x hours -a keys] [[-Tt] -d
keys]\n, __progname);
exit(1);
/* NOTREACHED */
}
@@ -270,10 +270,12 @@
main(int argc, char **argv)
{
int i, ch, action = 0, type = WHITE, r = 0, c = 0;
+time_t exp = 0;
HASHINFOhashinfo;
DB*db;
+const char *errstr;
-while ((ch = getopt(argc, argv, adtT)) != -1) {
+while ((ch = getopt(argc, argv, adtTx:)) != -1) {
switch (ch) {
case 'a':
action = 1;
@@ -287,6 +289,14 @@
case 'T':
type = SPAMTRAP;
break;
+case 'x':
+/* limit exp to 2160 hours (90 days) */
+exp = strtonum(optarg, 1, (24 * 90),errstr);
+if (errstr)
+usage();
+/* convert to seconds from hours */
+exp *= (60 * 60);
+break;
default:
usage();
break;
@@ -294,7 +304,7 @@
}
argc -= optind;
argv += optind;
-if (action == 0 type != WHITE)
+if ((action == 0 type != WHITE) || (action != 1 exp != 0))
usage();
memset(hashinfo, 0, sizeof(hashinfo));
@@ -314,10 +324,12 @@
case 0:
return dblist(db);
case 1:
+if (exp == 0)
+exp = type ? TRAPEXP : WHITEEXP;
for (i=0; iargc; i++)
if (argv[i][0] != '\0') {
c++;
-r += dbupdate(db, argv[i], 1, type);
+r += dbupdate(db, argv[i], 1, type, exp);
}
if (c == 0)
errx(2, no addresses specified);
@@ -326,7 +338,7 @@
for (i=0; iargc; i++)
if (argv[i][0] != '\0') {
c++;
-r += dbupdate(db, argv[i], 0, type);
+r += dbupdate(db, argv[i], 0, type, exp);
}
if (c == 0)
errx(2, no addresses specified);
Re: A bad entry in the spamdb kills pfctl
Op Tue, 23 Nov 2010 18:05:14 +0100 schreef Peter Fraser p...@thinkage.ca: Somehow I have an bad entry in my /var/db/spamdb the entry in question is a follows. GREY|kadorken.thspamdb -t -a itroll.03092...@thinkage.chinkage.on.ca|spamdb -t -a kgdykesb...@thinkage.on.ca|spamdb -t -a kgdykesb...@thinkage.on.ca|spamdb -t -a kgdykescxspamdb|1160168514|0|0|1|-2 I have no idea how the entry got there. How can I delete it? I cannot get the spamdb -d to work, it is expecting an IP address The entry ends with 0|0|1|-2 so it is actually a spamtrap entry that contains newlines. That's why it didn't expire although you'd think it should have. I could successfully use spamdb(8) to add spamtrap entries containing a few newlines and then remove them (phew!). So try this: # spamdb -Td 'kadorken.thspamdb -t -a itroll.03092...@thinkage.chinkage.on.ca spamdb -t -a kgdykesb...@thinkage.on.ca spamdb -t -a kgdykesb...@thinkage.on.ca spamdb -t -a kgdykescxspamdb' If that doesn't work, then possibly the real key is longer and you need a modified version of spamdb(8) to extract it. -- Gemaakt met Opera's revolutionaire e-mailprogramma: http://www.opera.com/mail/
Re: A bad entry in the spamdb kills pfctl
That worked thanks -Original Message- From: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] On Behalf Of Boudewijn Dijkstra Sent: Wednesday, November 24, 2010 6:08 AM To: OpenBSD misc Subject: Re: A bad entry in the spamdb kills pfctl Op Tue, 23 Nov 2010 18:05:14 +0100 schreef Peter Fraser p...@thinkage.ca: Somehow I have an bad entry in my /var/db/spamdb the entry in question is a follows. GREY|kadorken.thspamdb -t -a itroll.03092...@thinkage.chinkage.on.ca|spamdb -t -a kgdykesb...@thinkage.on.ca|spamdb -t -a kgdykesb...@thinkage.on.ca|spamdb -t -a kgdykescxspamdb|1160168514|0|0|1|-2 I have no idea how the entry got there. How can I delete it? I cannot get the spamdb -d to work, it is expecting an IP address The entry ends with 0|0|1|-2 so it is actually a spamtrap entry that contains newlines. That's why it didn't expire although you'd think it should have. I could successfully use spamdb(8) to add spamtrap entries containing a few newlines and then remove them (phew!). So try this: # spamdb -Td 'kadorken.thspamdb -t -a itroll.03092...@thinkage.chinkage.on.ca spamdb -t -a kgdykesb...@thinkage.on.ca spamdb -t -a kgdykesb...@thinkage.on.ca spamdb -t -a kgdykescxspamdb' If that doesn't work, then possibly the real key is longer and you need a modified version of spamdb(8) to extract it. -- Gemaakt met Opera's revolutionaire e-mailprogramma: http://www.opera.com/mail/
A bad entry in the spamdb kills pfctl
Somehow I have an bad entry in my /var/db/spamdb the entry in question is a follows. GREY|kadorken.thspamdb -t -a itroll.03092...@thinkage.chinkage.on.ca|spamdb -t -a kgdykesb...@thinkage.on.ca|spamdb -t -a kgdykesb...@thinkage.on.ca|spamdb -t -a kgdykescxspamdb|1160168514|0|0|1|-2 I have no idea how the entry got there. How can I delete it? I cannot get the spamdb -d to work, it is expecting an IP address spamd seems to still work, but seems to use a much more memory to operate. The memory usage caused a problem that was hard to track down. spamd calls pfctl to update the table spamd-white, pfclt would fail with insufficient memory. No error message (that I could find) was reported. The side effect of the failure of pfctl was the spamd-white table in pf was not updated, which mean that no mail from new locations could be received. Eventually there were complaints. I found the problem after booting the system and noticing the spamd-white table had no entries. If spamd was run in debug mode, then the error message did appear. My workaround was to add more memory to the system.
Re: A bad entry in the spamdb kills pfctl
On Tue, Nov 23, 2010 at 05:05:14PM +, Peter Fraser wrote: Somehow I have an bad entry in my /var/db/spamdb the entry in question is a follows. GREY|kadorken.thspamdb -t -a itroll.03092...@thinkage.chinkage.on.ca|spamdb -t -a kgdykesb...@thinkage.on.ca|spamdb -t -a kgdykesb...@thinkage.on.ca|spamdb -t -a kgdykescxspamdb|1160168514|0|0|1|-2 I have no idea how the entry got there. How can I delete it? I cannot get the spamdb -d to work, it is expecting an IP address hi, don;t know if there's an official correct way, but you could just replace the database with an empty file. if starting from scratch is too awful a thought, you could easily repopulate the database using the output of (the current) spamdb command and some small script. there might be a nicer way to do it though. jmc
spamdb -t -a w.x.y.z add trap, but do not remove grey listing and end up passing it.
One question here. Shouldn't the spamdb -t -a w.x.y.z when adding a trap manually in the spamd database also removed any grey listing for the same IP's? What happened is of in the same 4 hours window I manually add an IP to the trap, but I also see source for that IP in the greg listing, they are not removed and even if the trap entry is there, it will end up being white listed regardless. example: # spamdb | grep 41.199.130.240 GREY|41.199.130.240|ZGMTAVHVGN|unquestionable...@regiecazouls.com|tosa...@realconnect.com|1282601097|1282615497|1282615497|1|0 # spamdb -t -a 41.199.130.240 # spamdb | grep 41.199.130.240 GREY|41.199.130.240|ZGMTAVHVGN|unquestionable...@regiecazouls.com|tosa...@realconnect.com|1282601097|1282615497|1282615497|1|0 TRAPPED|41.199.130.240|1282694618 If I still receive retrial from that IP before the expiring time that should be here for this example: # date -r 1282615497 Mon Aug 23 22:04:57 EDT 2010 it will pass and being listed as white, even of the trap is pretty clear that it should be trap until: # date -r 1282694618 Tue Aug 24 20:03:38 EDT 2010 Is that really intended to be that way? I don't think so, but that's what is going on. Any thoughts? Daniel
Re: spamdb -t -a w.x.y.z add trap, but do not remove grey listing and end up passing it.
On Mon, Aug 23, 2010 at 5:07 PM, Daniel Ouellet dan...@presscom.net wrote: One question here. Shouldn't the spamdb -t -a w.x.y.z when adding a trap manually in the spamd database also removed any grey listing for the same IP's? What happened is of in the same 4 hours window I manually add an IP to the trap, but I also see source for that IP in the greg listing, they are not removed and even if the trap entry is there, it will end up being white listed regardless. I had a similar issue but with the -d option to delete an offending spam source. Entry is GREY-listed and eventually WHITE-listed. Spam is detected in user inbox followed by an urge to remove the offender from the WHITE-list using `spamdb -d'. Guess what? Since it is in the GREY-list still, it is WHITE-listed w/in a minute. I offered a fairly simple patch to spamdb to remove all entries of argument (IP) back in 2008[1], but the idea was shutdown. That thread continues to give advice of using the -t -a options (like you are using), but as you have seen, and also pointed out in follow-up to the advice, the method is not as effective a initially thought. If the patch to -d option was accepted, following `spamdb -d IP' with either your `spamdb -t -a IP' or my current scheme of BLACK-listing the offenders would work like a charm. Alas... --patrick [1] http://marc.info/?l=openbsd-techm=121146640814228w=2 example: # spamdb | grep 41.199.130.240 GREY|41.199.130.240|ZGMTAVHVGN|unquestionable...@regiecazouls.com|tosa...@realconnect.com|1282601097|1282615497|1282615497|1|0 # spamdb -t -a 41.199.130.240 # spamdb | grep 41.199.130.240 GREY|41.199.130.240|ZGMTAVHVGN|unquestionable...@regiecazouls.com|tosa...@realconnect.com|1282601097|1282615497|1282615497|1|0 TRAPPED|41.199.130.240|1282694618 If I still receive retrial from that IP before the expiring time that should be here for this example: # date -r 1282615497 Mon Aug 23 22:04:57 EDT 2010 it will pass and being listed as white, even of the trap is pretty clear that it should be trap until: # date -r 1282694618 Tue Aug 24 20:03:38 EDT 2010 Is that really intended to be that way? I don't think so, but that's what is going on. Any thoughts? Daniel
Re: spamdb: is it my eyes or do TRAPPED addresses still manage to get through?
Traplists do not go into tables. (for this exact reason) only the
whitelisted
hosts go into tables guys.
Bob
* Peter N. M. Hansteen pe...@bsdly.net [2009-07-28 15:31]:
Renaud Allard ren...@allard.it writes:
It happened to me also with servers with huge white/black lists. If
it's happening for new connections, ensure that pf is configured with
enough maximum table entries (set limit table-entries).
That's interesting. Hitting table size limits would explain the
symptoms. The blacklists we used were uatraps, nixspam and bsdly --
at the time we observed this, size about 60,000, 40,000 and 3000ish
respectively -- for a total of just over 100,000. IIRC the greylist
had just grown to somewhat more than 100,000 too. This with no
non-default settings with respect to tables.
All the best,
Peter
--
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
Remember to set the evil bit on all malicious network traffic
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
--
#!/usr/bin/perl
if ((not 0 not 1) != (! 0 ! 1)) {
print Larry and Tom must smoke some really primo stuff...\n;
}
Re: spamdb: is it my eyes or do TRAPPED addresses still manage to get through?
Renaud Allard ren...@allard.it writes: It happened to me also with servers with huge white/black lists. If it's happening for new connections, ensure that pf is configured with enough maximum table entries (set limit table-entries). That's interesting. Hitting table size limits would explain the symptoms. The blacklists we used were uatraps, nixspam and bsdly -- at the time we observed this, size about 60,000, 40,000 and 3000ish respectively -- for a total of just over 100,000. IIRC the greylist had just grown to somewhat more than 100,000 too. This with no non-default settings with respect to tables. All the best, Peter -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: spamdb: is it my eyes or do TRAPPED addresses still manage to get through?
On 7/24/09 3:03 PM, Peter N. M. Hansteen wrote: setting up a new spamd plus various content filtering at a client site we were kind of baffled to see that apparently manually setting an address to TRAPPED with spamdb, ie spamdb -a -t 211.49.57.32 for some reason seems porous, in that messages received from that IP address still hits the content filter a few minutes after the manual intervention. I just wonder what it is I'm seeing here - spamdb lookups cached or something? It happened to me also with servers with huge white/black lists. If it's happening for new connections, ensure that pf is configured with enough maximum table entries (set limit table-entries).
Re: spamdb: is it my eyes or do TRAPPED addresses still manage to get through?
Trapping an address only affects new connections that are looked up in
the
database. it does not affect existing passed connections. spamd only updates
the
tables on it's scan of the database so these will not take effect immediately.
-Bob
* Peter N. M. Hansteen pe...@bsdly.net [2009-07-24 07:10]:
setting up a new spamd plus various content filtering at a client site
we were kind of baffled to see that apparently manually setting an
address to TRAPPED with spamdb, ie
spamdb -a -t 211.49.57.32
for some reason seems porous, in that messages received from that IP
address still hits the content filter a few minutes after the manual
intervention. I just wonder what it is I'm seeing here - spamdb
lookups cached or something?
- Peter
--
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
Remember to set the evil bit on all malicious network traffic
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
--
#!/usr/bin/perl
if ((not 0 not 1) != (! 0 ! 1)) {
print Larry and Tom must smoke some really primo stuff...\n;
}
spamdb: is it my eyes or do TRAPPED addresses still manage to get through?
setting up a new spamd plus various content filtering at a client site we were kind of baffled to see that apparently manually setting an address to TRAPPED with spamdb, ie spamdb -a -t 211.49.57.32 for some reason seems porous, in that messages received from that IP address still hits the content filter a few minutes after the manual intervention. I just wonder what it is I'm seeing here - spamdb lookups cached or something? - Peter -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: spamdb: is it my eyes or do TRAPPED addresses still manage to get through?
On Fri, Jul 24, 2009 at 03:03:51PM +0200, Peter N. M. Hansteen wrote: setting up a new spamd plus various content filtering at a client site we were kind of baffled to see that apparently manually setting an address to TRAPPED with spamdb, ie spamdb -a -t 211.49.57.32 for some reason seems porous, in that messages received from that IP address still hits the content filter a few minutes after the manual intervention. I just wonder what it is I'm seeing here - spamdb lookups cached or something? I noticed such behavior some time ago. It makes sense if you think about it: spamd has a db, and now and then updates pf tables. Until the update in pf occurs... During normal, automatic operation this isn't an issue since IPs are going from greylist to either whilelist or blacklist, and presumably (I haven't read the code) spamd will update pf immediately since it knows there's a change. If you want immediate response then dump the IP in the pf table as well as adding it with spamdb. -- Darrin Chandler| Phoenix BSD User Group | MetaBUG dwchand...@stilyagin.com | http://phxbug.org/ | http://metabug.org/ http://www.stilyagin.com/ | Daemons in the Desert | Global BUG Federation
Some mystic with one spamdb WHITE record
Hi. ...working on with spamd. # uname -a OpenBSD mx.chics.ru 4.4 GENERIC#0 i386 Today I see some annoying spammer from 200.162.44.162. He was greylisted but then go thought it. # spamdb |fgrep '200.162.44.162' GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru|o...@email|1231915016|1231917033|1231929416|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru|o...@email|1231915016|1231917033|1231929416|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru|o...@email|1231915014|1231917030|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917031|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917031|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915013|1231917030|1231929413|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917031|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915015|1231917032|1231929415|6|0 WHITE|200.162.44.162|||1231915016|1231917033|1235034664|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915016|1231917033|1231929416|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915013|1231917030|1231929413|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915015|1231917032|1231929415|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915013|1231917030|1231929413|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915015|1231917032|1231929415|6|0 As you can see, WHITE entry. # date -r 1231915016 Wed Jan 14 11:36:56 YEKT 2009 So I start to try to blacklist him. I wrote 200.162.44.162 to my /etc/postfix/spamd_black.txt: $ cat /etc/postfix/spamd_black.txt b 200.162.44.162 some-spammer b $ fgrep -v '#' /etc/mail/spamd.conf |grep -v '^$' all:\ :myblack:uatraps:nixspam:china:korea:mywhite: myblack:\ :black:\ :msg=Your address %A has sent spam to me:\ :method=file:\ :file=/etc/postfix/spamd_black.txt: mywhite:\ :white:\ :method=file:\ :file=/etc/postfix/spamd_white.txt: b My /etc/postfix/spamd_white.txt is now empty file. Then I # /usr/libexec/spamd-setup -d blacklist myblack 23 entries b # spamdb -d '200.162.44.162' So, WHITE entry eliminated: # spamdb |fgrep '200.162.44.162' GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915016|1231917033|1231929416|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915016|1231917033|1231929416|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917030|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917031|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917031|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915013|1231917030|1231929413|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917031|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915015|1231917032|1231929415|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915016|1231917033|1231929416|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915013|1231917030|1231929413|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915015|1231917032|1231929415|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915013|1231917030|1231929413|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915015|1231917032|1231929415|6|0 I thought that from now spammer will be blacklisted. Wait some secondsb And what I see? # spamdb |fgrep '200.162.44.162' GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915016|1231917033|1231929416|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915016|1231917033|1231929416|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917030|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917031|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917031|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915013|1231917030|1231929413|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915014|1231917031|1231929414|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email |1231915015|1231917032|1231929415|6|0 WHITE|200.162.44.162|||1231915016|1231917033|1235035571|6|0 GREY|200.162.44.162|mail.plusoft.com.br|v...@chinatown.ru| o...@email
spamdb with '0' as pass
I have difficulties in understanding why a minority of IP's of a huge set of WHITE entries of our spamdb do not have a 'pass' date set: # spamdb | grep 128.1x8.50.xxx WHITE|128.1x8.50.xxx|||1218625388|0|1221750240|1|1 spamdb(8) says: time the entry passed from being GREY to being WHITE. Since it is WHITE, how can pass be 0? Puzzled, Stephan
can't remove greytrapped entry from spamdb
(On 4.3 recent snapshot) I began receiving mail for a certain email address and forgot to adjust my /etc/mail/spamd.alloweddomains file (where I have a list of all valid email addresses). So I found the following spamd logging reasonable: spamd[5771]: 10.10.10.10: disconnected after 386 seconds. lists: spamd-greytrap Along with its spamdb entry: TRAPPED|10.10.10.10|1214679171 However, after including the offending email address and stopping and restarting spamd; and removing the greytrapped/blacklisted host from spamdb like so $ sudo spamdb -T -d 10.10.10.10 I continue to get the same logging message and the address is again found in spamdb: $ sudo spamdb | grep 10.10.10.10 Password: GREY|10.10.10.10|... Granted that the last time it showed up as TRAPPED and now it shows GREY. But why does the log message say greytrap? /juan
Re: can't remove greytrapped entry from spamdb
2008/6/27 Juan Miscaro [EMAIL PROTECTED]: (On 4.3 recent snapshot) I began receiving mail for a certain email address and forgot to adjust my /etc/mail/spamd.alloweddomains file (where I have a list of all valid email addresses). So I found the following spamd logging reasonable: spamd[5771]: 10.10.10.10: disconnected after 386 seconds. lists: spamd-greytrap Along with its spamdb entry: TRAPPED|10.10.10.10|1214679171 However, after including the offending email address and stopping and restarting spamd; and removing the greytrapped/blacklisted host from spamdb like so $ sudo spamdb -T -d 10.10.10.10 I continue to get the same logging message and the address is again found in spamdb: $ sudo spamdb | grep 10.10.10.10 Password: GREY|10.10.10.10|... Granted that the last time it showed up as TRAPPED and now it shows GREY. But why does the log message say greytrap? Disregard, it was greyscanner.pl that didn't like the private address and greytrapped it. /juan
WHITE and GREY spamdb entries from the same host
Hi, Is it normal to have white and grey entries from the same IP address showing up in the output of spamdb? Should the GREY entries not be deleted once the IP address is whitelisted? GREY|217.130.91.233|qanr.comunitel.net|from-email|to-email| 1205058895|1205060468|1205073295|6|0 WHITE|217.130.91.233|||1205058895|1205060468|1208171170|6|1 Thanks for the explanation. Regards, Jose -- Want an e-mail address like mine? Get a free e-mail account today at www.mail.com!
Re: WHITE and GREY spamdb entries from the same host
On Sun, Mar 09, 2008 at 09:17:51AM -0500, Jose Fragoso wrote: Hi, Is it normal to have white and grey entries from the same IP address showing up in the output of spamdb? Should the GREY entries not be deleted once the IP address is whitelisted? GREY|217.130.91.233|qanr.comunitel.net|from-email|to-email| 1205058895|1205060468|1205073295|6|0 WHITE|217.130.91.233|||1205058895|1205060468|1208171170|6|1 Thanks for the explanation. AFAIK this is normal. The WHITE entry gets added and will be the important part. The GREY entry should expire on its own soon. In other words this is a temporary and harmless condition, by design, seen for a short while after a host is whitelisted. -- Darrin Chandler| Phoenix BSD User Group | MetaBUG [EMAIL PROTECTED] | http://phxbug.org/ | http://metabug.org/ http://www.stilyagin.com/ | Daemons in the Desert | Global BUG Federation
Re: WHITE and GREY spamdb entries from the same host
On 2008-03-09, Jose Fragoso [EMAIL PROTECTED] wrote: Is it normal to have white and grey entries from the same IP address showing up in the output of spamdb? Yes Should the GREY entries not be deleted once the IP address is whitelisted? No
spamdb replication?
Hi, reading about spamd having changed the database format (recently?), how do I best achieve replicating and merging the spamdb database(s) across a number of machines, maintaining consistent white- and greylisting entries? Or is this not yet supported (the docs suggest so)? Best, --Toni++
Re: spamdb replication?
Toni Mueller wrote: Hi, reading about spamd having changed the database format (recently?), how do I best achieve replicating and merging the spamdb database(s) across a number of machines, maintaining consistent white- and greylisting entries? Or is this not yet supported (the docs suggest so)? if the machines are all the same arch, you can copy the db file to the other hosts, best done while spamd not running and then restart using the -y and -Y options appropriately. cheers, jake Best, --Toni++
Re: spamdb output
On Mon, 19 Nov 2007 14:47:37 -0700, Bob Beck wrote: RW [EMAIL PROTECTED] [2007-11-11 22:39]: It seems that the migrated database works but new entries go on the end - no SORT of order, and SPAMTRAP entries (that I entered using a script) ended up showing in two bunches in the midst of other unordered entries. My question is: Is this normal with spamd a la 4.2 or is it because I migrated a database? This is normal in 4.2 - the change happened post 4.0 when spamdb stopped using DB_BTREE Thanks Bob. I'm already using a script to sort the list to emulate the previous behaviour but at least I know I'll have to keep a copy for any future wipe and re-install upgrade. Looking at today's output showed me another puzzle which you will probably shoot down, but here goes. Here is one line fro spamdb: GREY|69.28.223.134|mta5br.cmpgnr.com|gotb1103621_1102728_683443_1138134 [EMAIL PROTECTED] |[EMAIL PROTECTED]|1195673789|1195675648|1195688189|2|0 but here is a line from my spamlog: Nov 22 07:08:14 mail spamd[28826]: whitelisting 69.28.223.134 in /var/db/spamd Why does the spamdb output show GREY instead of WHITE three hours later? It does show the 2 knocks which date -r will show were more than a half hour apart and so the whitelisting should have happened. Colour me puzzled. BTW the envelope recipient address shown is a spamtrap and is my only edit of the output. Thanks again for spamd. I absolutely love it. I have never known of it causing loss of genuine mail and also grepping the mail log daily for reject has only shown two emails in the last six months being blocked by zen.spamhaus having passed spamd. Both were really spammers anyway so apmd has an extremely good batting average. Two domains hosted on that box and zero customer complaints = mail admin happiness. In the beginning was The Word and The Word was Content-type: text/plain The Word of Rod.
Re: spamdb output
* RW [EMAIL PROTECTED] [2007-11-11 22:39]: It seems that the migrated database works but new entries go on the end - no SORT of order, and SPAMTRAP entries (that I entered using a script) ended up showing in two bunches in the midst of other unordered entries. My question is: Is this normal with spamd a la 4.2 or is it because I migrated a database? This is normal in 4.2 - the change happened post 4.0 when spamdb stopped using DB_BTREE -Bob
spamdb output
I just got through updating a mailserver that had been running 4.0 to 4.2 using a new HDD, fresh install of OS and required packages. All old scripts settings etc preserved on original HDD now sitting in an accessible older box so I can grab anything forgotten. The one thing that hit me was the output of spamdb. Back on 4.0 all the entries came out (sort of) sorted. All the SPAMTRAP entries last but sorted on the trap address field. All the GREY, WHITE or TRAPPED entries first sorted on the IP field (but sorted alphabetically i.e. 101.x.y.z precedes 99.x.y.z) All that was fine because I could easily see if there were two entries for the one IP which happened when a script that runs every few minutes evaluates a GREY entry and enters it as TRAPPED. It seems that the migrated database works but new entries go on the end - no SORT of order, and SPAMTRAP entries (that I entered using a script) ended up showing in two bunches in the midst of other unordered entries. My question is: Is this normal with spamd a la 4.2 or is it because I migrated a database? I can always use: spamdb |sort -n -t | -k 2 |less to get a fully sorted list if I have to, but curiosity makes me ask about expected behaviour. Of course (to cut off pedants) I could have used: spamdb |sort -t | -k 2 -n|less to get the output looking like that from 4.0. Thanx, Rod/ /earth: write failed, file system is full cp: /earth/creatures: No space left on device
Re: spamdb expire value gets default value instead of spamd_flag value (-G)
Who put the 36 hour date in there? spamd or spamlogd?
spamlogd may have done that for you. look at your syslogs
-Bob
* Claes Str?m [EMAIL PROTECTED] [2007-10-24 10:13]:
Hi,
When testing greylisting with synchronizing we noticed the following
strange behavior:
Machine A (10.100.64.234) is the machine we receive mail through.
Machine B (10.100.64.233) is synced through spamd
Check out the expire value on machine A after the state have gone from
Grey to White!
It has taken the default 36 days ahead instead of our 2 hour (testvalue)
from spamd_flags!!
But Machine B (the passive brother which gets synced through
spamd-sync) behaves as it should!?
spamdb (A):
WHITE|10.100.64.199|||1193231843|1193232057|1196342528|3|1
spamdb (B):
WHITE|10.100.64.199|||1193231843|1193232057|1193239279|3|1
pf.conf:
no rdr inet proto tcp from spamd-white to any port smtp
rdr pass inet proto tcp from !own_ips to $ext_if:0 port smtp -
127.0.0.1 port spamd
pass in quick log on $ext_if proto tcp from any to ($ext_if) port
$public_tcp
pass in log on $int_if proto tcp from own_ips to ($int_if) port $sec_tcp
/etc/rc.conf.local (B)
pf=YES
syslogd_flags=-a /var/spool/postfix/dev/log
spamd_flags=-y fxp0 -Y 10.100.64.234 -G 3:1:2
spamlogd_flags=-i fxp0 -Y 10.100.64.234
/etc/rc.conf.local (A)
pf=YES
syslogd_flags=-a /var/spool/postfix/dev/log
spamd_flags=-y fxp0 -Y 10.100.64.233 -G 3:1:2
spamlogd_flags=-i fxp0 -Y 10.100.64.233
---
Probably some small feature to fix
Regards
Claes Strvm
--
#!/usr/bin/perl
if ((not 0 not 1) != (! 0 ! 1)) {
print Larry and Tom must smoke some really primo stuff...\n;
}
spamdb expire value gets default value instead of spamd_flag value (-G)
Hi, When testing greylisting with synchronizing we noticed the following strange behavior: Machine A (10.100.64.234) is the machine we receive mail through. Machine B (10.100.64.233) is synced through spamd Check out the expire value on machine A after the state have gone from Grey to White! It has taken the default 36 days ahead instead of our 2 hour (testvalue) from spamd_flags!! But Machine B (the passive brother which gets synced through spamd-sync) behaves as it should!? spamdb (A): WHITE|10.100.64.199|||1193231843|1193232057|1196342528|3|1 spamdb (B): WHITE|10.100.64.199|||1193231843|1193232057|1193239279|3|1 pf.conf: no rdr inet proto tcp from spamd-white to any port smtp rdr pass inet proto tcp from !own_ips to $ext_if:0 port smtp - 127.0.0.1 port spamd pass in quick log on $ext_if proto tcp from any to ($ext_if) port $public_tcp pass in log on $int_if proto tcp from own_ips to ($int_if) port $sec_tcp /etc/rc.conf.local (B) pf=YES syslogd_flags=-a /var/spool/postfix/dev/log spamd_flags=-y fxp0 -Y 10.100.64.234 -G 3:1:2 spamlogd_flags=-i fxp0 -Y 10.100.64.234 /etc/rc.conf.local (A) pf=YES syslogd_flags=-a /var/spool/postfix/dev/log spamd_flags=-y fxp0 -Y 10.100.64.233 -G 3:1:2 spamlogd_flags=-i fxp0 -Y 10.100.64.233 --- Probably some small feature to fix Regards Claes Strvm
Re: SMTP flood + spamdb
On Wed, 26 Sep 2007 17:02:50 +0300 Liviu Daia [EMAIL PROTECTED] wrote: Why should it? The second copy is sent in a separate run, that's the whole point. The only thing the bot has to figure out is how long to wait until the second run. A smart one would send a second copy after 10 minutes, and a third one after, say, 35 minutes. They would also need to use the same from address. If they randomly choose from addresses, it wouldn't make any difference how often they send the spam. I've seen numerous attempts to deliver the same message (presumably) to the same recipient but with a different from address for each attempt. Eric Johnson
Re: SMTP flood + spamdb
--- Bob Beck [EMAIL PROTECTED] wrote: [snip] greylisting does what it does. It delays the initial email for 30 minutes or more. what you do with that 30 minutes will decide on how effective it is for you. In that 30 minutes) [snip] 4) optionally, if you check the greylist against valid local mail addresses, you could trap them if they're mailing to bogus local addresses (we do that here) Is there a standard way to achieve that or does one just hack a shell script together? // juan Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail at http://mrd.mail.yahoo.com/try_beta?.intl=ca
Re: SMTP flood + spamdb
* Juan Miscaro [EMAIL PROTECTED] [2007-09-27 11:36]: --- Bob Beck [EMAIL PROTECTED] wrote: [snip] greylisting does what it does. It delays the initial email for 30 minutes or more. what you do with that 30 minutes will decide on how effective it is for you. In that 30 minutes) [snip] 4) optionally, if you check the greylist against valid local mail addresses, you could trap them if they're mailing to bogus local addresses (we do that here) Is there a standard way to achieve that or does one just hack a shell script together? Yes, there are some standard ways as documented in spamd(8)- they are relatively new, so if your spamd is old you don't have them. see the /etc/mail/spamd/alloweddomains, etc. etc. There is a quasi standard perl script which I have posted and is available frequently referenced in the archives of this list, and has already been mentioned twice in this thread. it is not standard with OpenBSD because pieces of it must be customized to be site specific, so it's not really a generic solution, but it can do some things the generic stuff can't. -Bob
Re: SMTP flood + spamdb
Bob Beck wrote: There is a quasi standard perl script which I have posted and is available frequently referenced in the archives of this list, and has already been mentioned twice in this thread. it is not standard with OpenBSD because pieces of it must be customized to be site specific, so it's not really a generic solution, but it can do some things the generic stuff can't. And that script works quite well, I can report. Heck, even not using the user validation parts it cuts a lot of crud out. (And by a lot, I mean a lot of what just spamd doesn't grab...). --Kurt
Re: SMTP flood + spamdb
Chris Smith wrote: On Tuesday 25 September 2007, Craig Skinner wrote: If you are using postfix: /etc/postfix/main.cf: .. .. smtpd_recipient_restrictions = reject_non_fqdn_hostname reject_invalid_hostname reject_non_fqdn_sender reject_non_fqdn_recipient reject_unlisted_recipient -- this one Isn't this actually a postfix default? As smtpd_reject_unlisted_recipient defaults to yes. Absolutely correct, but by poking it in smtpd_recipient_restrictions you can choose when to invoke it. Probably makes little difference either way in the real world.
Re: SMTP flood + spamdb
RW wrote: What I was getting looked like backscatter and smelled like backscatter it is just that some of the IPs sending it didn't check out as MTAs. i.e. they were not listed MXs for the domain they came from AND the domain was not likely someone with separate outbound senders. They all retried too and when I had them as TRAPPED entries the logged data included typical failed-to-deliver messages. 'bots getting smart eh? Bugger! If that is the trend, greylisting starts to lose its value as spammers adapt to the RFCs. Set up a pf queue of dialup speed for windows boxes connecting to port 25? Should slow them down a bit, but still let the odd legit extrange sent mail in.
Re: SMTP flood + spamdb
Craig Skinner [EMAIL PROTECTED] writes: 'bots getting smart eh? Bugger! If that is the trend, greylisting starts to lose its value as spammers adapt to the RFCs. If they adapt to greylisting and start following relevant RFCs, we've succeeded in making spamming more expensive. I don't see that happening much, though. The spam that reaches content filtering here has managed to get itself into the queue on real mail servers which for some reason allows them. On the 'spamd looks like an open relay' issue, it would make sense to use a relay checker which actually checks for mail received, not just the status codes. On the other hand I actually like that part of spamd the way it is. Spammers who apparently think every IP address in our range is an open relay occasionally swell our greylists quite a bit. None of it ever gets delivered, of course, but we see the attempts quite often these days. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.datadok.no/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: SMTP flood + spamdb
On 26 September 2007, Craig Skinner [EMAIL PROTECTED] wrote: RW wrote: What I was getting looked like backscatter and smelled like backscatter it is just that some of the IPs sending it didn't check out as MTAs. i.e. they were not listed MXs for the domain they came from AND the domain was not likely someone with separate outbound senders. They all retried too and when I had them as TRAPPED entries the logged data included typical failed-to-deliver messages. 'bots getting smart eh? Bugger! If that is the trend, greylisting starts to lose its value as spammers adapt to the RFCs. [...] Greylisting is trivial to bypass, with or without a queue: just send the same messages twice. Some spammers have figured that out long ago. Ever wondered why sometimes you receive 2 or 3 copies of the same spam, from the same IP, with the same Message-Id etc., a few minutes apart? Regards, Liviu Daia -- Dr. Liviu Daia http://www.imar.ro/~daia
Re: SMTP flood + spamdb
On Wed, 26 Sep 2007, Liviu Daia wrote: Greylisting is trivial to bypass, with or without a queue: just send the same messages twice. Some spammers have figured that out long ago. Ever wondered why sometimes you receive 2 or 3 copies of the same spam, from the same IP, with the same Message-Id etc., a few minutes apart? That doesn't work, at least not against spamd.
Re: SMTP flood + spamdb
On 26 September 2007, Damien Miller [EMAIL PROTECTED] wrote: On Wed, 26 Sep 2007, Liviu Daia wrote: Greylisting is trivial to bypass, with or without a queue: just send the same messages twice. Some spammers have figured that out long ago. Ever wondered why sometimes you receive 2 or 3 copies of the same spam, from the same IP, with the same Message-Id etc., a few minutes apart? That doesn't work, at least not against spamd. How does spamd distinguish between a legitimate retry and a re-injection of the same message with the same Message-Id, sender etc.? Regards, Liviu Daia -- Dr. Liviu Daia http://www.imar.ro/~daia
Re: SMTP flood + spamdb
Liviu Daia wrote: How does spamd distinguish between a legitimate retry and a re-injection of the same message with the same Message-Id, sender etc.? It doesn't. Just what you described would probably be within the default 25 mins grey period. Another delivery attempt would be needed after this time to pass spamd.
Re: SMTP flood + spamdb
On Wed, 26 Sep 2007, Liviu Daia wrote: On 26 September 2007, Damien Miller [EMAIL PROTECTED] wrote: On Wed, 26 Sep 2007, Liviu Daia wrote: Greylisting is trivial to bypass, with or without a queue: just send the same messages twice. Some spammers have figured that out long ago. Ever wondered why sometimes you receive 2 or 3 copies of the same spam, from the same IP, with the same Message-Id etc., a few minutes apart? That doesn't work, at least not against spamd. How does spamd distinguish between a legitimate retry and a re-injection of the same message with the same Message-Id, sender etc.? It can't. But spamd's default of 25 minute passtime should help. (Well it does help someone -- since it limits the spammer's resources.) (spamd doesn't know about Message-Id, it uses connecting IP address, HELO/EHLO, envelope-from, and envelope-to.)
Re: SMTP flood + spamdb
On 26 September 2007, Craig Skinner [EMAIL PROTECTED] wrote: Liviu Daia wrote: How does spamd distinguish between a legitimate retry and a re-injection of the same message with the same Message-Id, sender etc.? It doesn't. Just what you described would probably be within the default 25 mins grey period. Why should it? The second copy is sent in a separate run, that's the whole point. The only thing the bot has to figure out is how long to wait until the second run. A smart one would send a second copy after 10 minutes, and a third one after, say, 35 minutes. Another delivery attempt would be needed after this time to pass spamd. Moral: randomize the greylisting time... Regards, Liviu Daia -- Dr. Liviu Daia http://www.imar.ro/~daia
Re: SMTP flood + spamdb
On Wed, 2007-09-26 at 17:02 +0300, Liviu Daia wrote: Another delivery attempt would be needed after this time to pass spamd. Moral: randomize the greylisting time... Between which min/max valuse? Keep in mind that this corresponds to the (minimum) delay introduced in delivering a good messages to the mailbox. ciao Luca
Re: SMTP flood + spamdb
Liviu Daia wrote: Why should it? The second copy is sent in a separate run, that's the whole point. The only thing the bot has to figure out is how long to wait until the second run. A smart one would send a second copy after 10 minutes, and a third one after, say, 35 minutes. OK, but you did say a few minutes apart. In English, few is normally a single digit.
Re: SMTP flood + spamdb
On 26 September 2007, Luca Corti [EMAIL PROTECTED] wrote: On Wed, 2007-09-26 at 17:02 +0300, Liviu Daia wrote: Another delivery attempt would be needed after this time to pass spamd. Moral: randomize the greylisting time... Between which min/max valuse? Keep in mind that this corresponds to the (minimum) delay introduced in delivering a good messages to the mailbox. That's up to you. The minimum should be large enough to keep away naive bots, as it does now. The maximum should be as large as you can afford without being too anti-social. :) Some crap will still pass through anyway. Regards, Liviu Daia -- Dr. Liviu Daia http://www.imar.ro/~daia
Re: SMTP flood + spamdb
On 26 September 2007, Liviu Daia [EMAIL PROTECTED] wrote: On 26 September 2007, Luca Corti [EMAIL PROTECTED] wrote: On Wed, 2007-09-26 at 17:02 +0300, Liviu Daia wrote: Another delivery attempt would be needed after this time to pass spamd. Moral: randomize the greylisting time... Between which min/max valuse? Keep in mind that this corresponds to the (minimum) delay introduced in delivering a good messages to the mailbox. That's up to you. The minimum should be large enough to keep away naive bots, as it does now. The maximum should be as large as you can afford without being too anti-social. :) Some crap will still pass through anyway. The maximum should also leave plenty of time before expiry. Some mailers use queue backoff algorithms, which means some legitimate messages might never get a chance to pass if the window is too small... Regards, Liviu Daia -- Dr. Liviu Daia http://www.imar.ro/~daia
Re: SMTP flood + spamdb
Liviu Daia [EMAIL PROTECTED] writes: Why should it? The second copy is sent in a separate run, that's the whole point. The only thing the bot has to figure out is how long to wait until the second run. A smart one would send a second copy after 10 minutes, and a third one after, say, 35 minutes. *BZZT!* Assuming facts not in evidence: a *smart* spambot /and/ a spammer who actually *cares* about the delivery of individual messages. Another delivery attempt would be needed after this time to pass spamd. Actually, the way it works is more like this: 1st try: 451 try again later * At this point, anywhere between 80%-97% of spammers just move on, there's millions more messages to spew out there, and other hosts which are way more receptive. 2nd try, after passtime: 451 try again later (spamd to self: oh, this one retried, better whitelist) * This is where we decide it has a chance to be non-trash, but we don't let on just yet 3rd try: now you talk to the real smtp daemon (if there is one) * They've passed the test. They may still be bastids, but at least they know some basic rules of conduct. Moral: randomize the greylisting time... Random numbers can be fun, but I'd like to see real world data which support your theory. I'm beginning to think that this is another one of those 'I refuse to believe greylisting works because I refuse to understand it' episodes. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.datadok.no/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: SMTP flood + spamdb
Liviu Daia wrote: That's up to you. The minimum should be large enough to keep away naive bots, as it does now. The maximum should be as large as you can afford without being too anti-social. :) Some crap will still pass through anyway. The maximum should also leave plenty of time before expiry. Some mailers use queue backoff algorithms, which means some legitimate messages might never get a chance to pass if the window is too small... The defaults work very well: See: http://www.ualberta.ca/~beck/nycbug06/spamd/mgp1.html Hear: http://www.fetissov.org/public/nycbsdcon06/2.4.mp3
Re: SMTP flood + spamdb
On Wed, 26 Sep 2007, Liviu Daia wrote: On 26 September 2007, Craig Skinner [EMAIL PROTECTED] wrote: Liviu Daia wrote: How does spamd distinguish between a legitimate retry and a re-injection of the same message with the same Message-Id, sender etc.? It doesn't. Just what you described would probably be within the default 25 mins grey period. Why should it? The second copy is sent in a separate run, that's the whole point. The only thing the bot has to figure out is how long to wait until the second run. A smart one would send a second copy after 10 minutes, and a third one after, say, 35 minutes. Another delivery attempt would be needed after this time to pass spamd. Moral: randomize the greylisting time... Or take advantage of the (by default) 25 minute window to use other means to detect that this address is sending spam. Perhaps spamd should be extended to look for excessive attempts to send messages from an address during that period? (How often do spammers' lists contain only one or two addresses from a domain?) Dave -- Dave Anderson [EMAIL PROTECTED]
Re: SMTP flood + spamdb
On 2007/09/26 11:03, Dave Anderson wrote: Or take advantage of the (by default) 25 minute window to use other means to detect that this address is sending spam. Perhaps spamd should be extended to look for excessive attempts to send messages from an address during that period? google: greyscanner
Re: SMTP flood + spamdb
Dave Anderson [EMAIL PROTECTED] writes: Or take advantage of the (by default) 25 minute window to use other means to detect that this address is sending spam. Perhaps spamd should be extended to look for excessive attempts to send messages from an address during that period? (How often do spammers' lists contain only one or two addresses from a domain?) You could probably use straight rdr instead of rdr pass to feed spamd, then in the relevant pass rule apply your source tracking options and overload and some table magic for that -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.datadok.no/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: SMTP flood + spamdb
On Wed, 2007-09-26 at 17:38 +0300, Liviu Daia wrote: That's up to you. The minimum should be large enough to keep away naive bots, as it does now. The maximum should be as large as you can afford without being too anti-social. :) Some crap will still pass through anyway. Sometimes is not about being social, but about receiving important messages in time. Some greylisting implementations have a bit more relaxed policies on whitelisting (e.g. whitelisting /24 subnets on which the sender resides instead of single IP address) to speed up whitelisting efficiency. Not sure about how much spam this allows to slip through. ciao Luca
Re: SMTP flood + spamdb
On Wed, 2007-09-26 at 16:01 +0100, Craig Skinner wrote: The defaults work very well: See: http://www.ualberta.ca/~beck/nycbug06/spamd/mgp1.html Hear: http://www.fetissov.org/public/nycbsdcon06/2.4.mp3 Maybe this also has to do with amount and type of traffic you get. Small shops are probably more likely to experience delays, while hosters of a large number of high traffic domains should have whitelists converge fast. ciao Luca
Re: SMTP flood + spamdb
On 26 September 2007, Peter N. M. Hansteen [EMAIL PROTECTED] wrote: Liviu Daia [EMAIL PROTECTED] writes: Why should it? The second copy is sent in a separate run, that's the whole point. The only thing the bot has to figure out is how long to wait until the second run. A smart one would send a second copy after 10 minutes, and a third one after, say, 35 minutes. *BZZT!* Assuming facts not in evidence: a *smart* spambot /and/ a spammer who actually *cares* about the delivery of individual messages. My point is it doesn't have to. The third copy passes regardless of what happens with the first two. [...] Moral: randomize the greylisting time... Random numbers can be fun, but I'd like to see real world data which support your theory. Ok, since you ask, here's a recent one. The message passed all my filters, so it was received three times. Please note the identical message-id. First run: Sep 25 18:06:16 ns1 postfix-localhost/smtpd[27143]: 9FAE1142A7: client=unknown[212.239.40.101] Sep 25 18:06:17 ns1 postfix/cleanup[3734]: 9FAE1142A7: message-id=[EMAIL PROTECTED] Sep 25 18:06:18 ns1 postfix/qmgr[1554]: 9FAE1142A7: from=[EMAIL PROTECTED], size=2545, nrcpt=2 (queue active) Sep 25 18:06:18 ns1 postfix/pipe[25075]: 9FAE1142A7: to=[EMAIL PROTECTED], relay=uucpz, delay=1.8, delays=1.7/0/0/0.06, dsn=2.0.0, status=sent (delivered via uucpz service) Sep 25 18:06:18 ns1 postfix/local[7260]: 9FAE1142A7: to=[EMAIL PROTECTED], relay=local, delay=1.9, delays=1.7/0/0/0.24, dsn=2.0.0, status=sent (delivered to command: /usr/local/sbin/gather_stats.pl /usr/local/share/Mail_stats) Sep 25 18:06:18 ns1 postfix/qmgr[1554]: 9FAE1142A7: removed The same message, sent 8 minutes later: Sep 25 18:14:14 ns1 postfix-localhost/smtpd[8404]: 1649714331: client=unknown[212.239.40.101] Sep 25 18:14:15 ns1 postfix/cleanup[21622]: 1649714331: message-id=[EMAIL PROTECTED] Sep 25 18:14:15 ns1 postfix/qmgr[1554]: 1649714331: from=[EMAIL PROTECTED], size=2547, nrcpt=2 (queue active) Sep 25 18:14:15 ns1 postfix/pipe[25075]: 1649714331: to=[EMAIL PROTECTED], relay=uucpz, delay=1.4, delays=1.4/0/0/0.05, dsn=2.0.0, status=sent (delivered via uucpz service) Sep 25 18:14:15 ns1 postfix/local[7260]: 1649714331: to=[EMAIL PROTECTED], relay=local, delay=1.6, delays=1.4/0/0/0.25, dsn=2.0.0, status=sent (delivered to command: /usr/local/sbin/gather_stats.pl /usr/local/share/Mail_stats) Sep 25 18:14:15 ns1 postfix/qmgr[1554]: 1649714331: removed Same, 28 minutes later: Sep 25 18:42:52 ns1 postfix-localhost/smtpd[13055]: 72BCD142A7: client=unknown[212.239.40.101] Sep 25 18:42:53 ns1 postfix/cleanup[21622]: 72BCD142A7: message-id=[EMAIL PROTECTED] Sep 25 18:42:53 ns1 postfix/qmgr[1554]: 72BCD142A7: from=[EMAIL PROTECTED], size=3724, nrcpt=2 (queue active) Sep 25 18:42:53 ns1 postfix/pipe[25075]: 72BCD142A7: to=[EMAIL PROTECTED], relay=uucpz, delay=0.81, delays=0.75/0.01/0/0.05, dsn=2.0.0, status=sent (delivered via uucpz service) Sep 25 18:42:53 ns1 postfix/local[7260]: 72BCD142A7: to=[EMAIL PROTECTED], relay=local, delay=1, delays=0.75/0.01/0/0.24, dsn=2.0.0, status=sent (delivered to command: /usr/local/sbin/gather_stats.pl /usr/local/share/Mail_stats) Sep 25 18:42:53 ns1 postfix/qmgr[1554]: 72BCD142A7: removed Should I have used spamd, the first two copies would have been discarded, but the third would have passed. That said, randomizing the greylisting time probably is probably a lot of trouble, for little added value (it still doesn't solve the problem). I'm beginning to think that this is another one of those 'I refuse to believe greylisting works because I refuse to understand it' episodes. Oh, I'm not saying it doesn't work. What I'm saying is, greylisting is trivial to bypass, and some spammers have figured that out. Amazingly, most of them still haven't, which is why it still works in a significant number of cases. Regards, Liviu Daia -- Dr. Liviu Daia http://www.imar.ro/~daia
Re: SMTP flood + spamdb
Oh, I'm not saying it doesn't work. What I'm saying is, greylisting is trivial to bypass, and some spammers have figured that out. Amazingly, most of them still haven't, which is why it still works in a significant number of cases. greylisting does what it does. It delays the initial email for 30 minutes or more. what you do with that 30 minutes will decide on how effective it is for you. In that 30 minutes) 1) you can look at their traffic profile and decide you don't want to talk to them. 2) they can hit a greytrap locally and you can decide you don't want to talk to them. 3) they can hit a bogus domain mx'ed to you locally and you can decide you don't want to talk to them. 4) optionally, if you check the greylist against valid local mail addresses, you could trap them if they're mailing to bogus local addresses (we do that here) 5) 1-4 could happen above at someone elses site (like nixspam or uatraps) that you are using as a blacklist. And in the end some of it gets through. That's why you run other stuff in addition to spamd if you really can't stand that. spamd is designed to get the low hanging fruit. It is *NOT* designed to stop all possible spam, forever. attempting to do so there is wrong. Spamd is a *tool* - it's good for what it's good for - stopping stuff that is easily identifiable in the smtp dialogue. It is not intended for other things. -Bob
Re: SMTP flood + spamdb
On Wed, 26 Sep 2007, Liviu Daia wrote: Same, 28 minutes later: Sep 25 18:42:52 ns1 postfix-localhost/smtpd[13055]: 72BCD142A7: client=unknown[212.239.40.101] Sep 25 18:42:53 ns1 postfix/cleanup[21622]: 72BCD142A7: message-id=[EMAIL PROTECTED] Sep 25 18:42:53 ns1 postfix/qmgr[1554]: 72BCD142A7: from=[EMAIL PROTECTED], size=3724, nrcpt=2 (queue active) Sep 25 18:42:53 ns1 postfix/pipe[25075]: 72BCD142A7: to=[EMAIL PROTECTED], relay=uucpz, delay=0.81, delays=0.75/0.01/0/0.05, dsn=2.0.0, status=sent (delivered via uucpz service) Sep 25 18:42:53 ns1 postfix/local[7260]: 72BCD142A7: to=[EMAIL PROTECTED], relay=local, delay=1, delays=0.75/0.01/0/0.24, dsn=2.0.0, status=sent (delivered to command: /usr/local/sbin/gather_stats.pl /usr/local/share/Mail_stats) Sep 25 18:42:53 ns1 postfix/qmgr[1554]: 72BCD142A7: removed Should I have used spamd, the first two copies would have been discarded, but the third would have passed. Not good example. As that would still hit spamd (default 25 minutes and your earlier one was too fast). Now it is whitelisted. Do you have a fourth email sent? (Which will have passed.)
Re: SMTP flood + spamdb
On 26 September 2007, Jeremy C. Reed [EMAIL PROTECTED] wrote: On Wed, 26 Sep 2007, Liviu Daia wrote: Same, 28 minutes later: Sep 25 18:42:52 ns1 postfix-localhost/smtpd[13055]: 72BCD142A7: client=unknown[212.239.40.101] Sep 25 18:42:53 ns1 postfix/cleanup[21622]: 72BCD142A7: message-id=[EMAIL PROTECTED] Sep 25 18:42:53 ns1 postfix/qmgr[1554]: 72BCD142A7: from=[EMAIL PROTECTED], size=3724, nrcpt=2 (queue active) Sep 25 18:42:53 ns1 postfix/pipe[25075]: 72BCD142A7: to=[EMAIL PROTECTED], relay=uucpz, delay=0.81, delays=0.75/0.01/0/0.05, dsn=2.0.0, status=sent (delivered via uucpz service) Sep 25 18:42:53 ns1 postfix/local[7260]: 72BCD142A7: to=[EMAIL PROTECTED], relay=local, delay=1, delays=0.75/0.01/0/0.24, dsn=2.0.0, status=sent (delivered to command: /usr/local/sbin/gather_stats.pl /usr/local/share/Mail_stats) Sep 25 18:42:53 ns1 postfix/qmgr[1554]: 72BCD142A7: removed Should I have used spamd, the first two copies would have been discarded, but the third would have passed. Not good example. As that would still hit spamd (default 25 minutes and your earlier one was too fast). Now it is whitelisted. Do you have a fourth email sent? (Which will have passed.) Not at hand, but I haven't been looking for one either. Does spamd really behave like that? That is, ignore retries during the greylisting period, and whitelist messages only on subsequent attempts? Regards, Liviu Daia -- Dr. Liviu Daia http://www.imar.ro/~daia
Re: SMTP flood + spamdb
On 26 September 2007, Bob Beck [EMAIL PROTECTED] wrote: Oh, I'm not saying it doesn't work. What I'm saying is, greylisting is trivial to bypass, and some spammers have figured that out. Amazingly, most of them still haven't, which is why it still works in a significant number of cases. greylisting does what it does. It delays the initial email for 30 minutes or more. what you do with that 30 minutes will decide on how effective it is for you. In that 30 minutes) [...] Ok, brain dump: That's an interesting idea, a lot of slow operations could be offloaded to those 30 minutes. Your greyscanner script does DNS checks on the envelope. A lot more could be done, should the script have access to the body. I think it's legal to reply with 4xx (that is, simulate a queue error) to the final . . That could be used to gather and inspect the data; basically do at greylisting time what Postfix does with the after-queue filters. I suppose gathering everything would be prohibitive though, and against the entire philosophy of greylisting. Which brings me to a different approach: use a pre-queue filter instead of spamd (which is what I'm doing now). Now, the problem with pre-queue filters is they can take too long to scan a message. Thus, the better mouse trap: a pre-queue filter, which can send feedback to smapd, and use spamd's database to keep some state across messages. I need to ponder on that some more. spamd is designed to get the low hanging fruit. It is *NOT* designed to stop all possible spam, forever. attempting to do so there is wrong. Spamd is a *tool* - it's good for what it's good for - stopping stuff that is easily identifiable in the smtp dialogue. It is not intended for other things. We are in violent agreement here... Regards, Liviu Daia -- Dr. Liviu Daia http://www.imar.ro/~daia
Re: SMTP flood + spamdb
Oh, I'm not saying it doesn't work. What I'm saying is, greylisting is trivial to bypass, and some spammers have figured that out. Amazingly, most of them still haven't, which is why it still works in a significant number of cases. Just to give an additional data point here: I work for an ISP that receives upwards of a million inbound SMTP connections per day. While watching the connection logs, I've noticed that a large majority of spammers get the first spamd response (250 Hello, spam sender. Pleased to be wasting your time.) and immediately disconnect. This suggests to me that rather than spend time trying to get whitelisted by spamd servers, they've mostly decided to skip them entirely and move on to servers that aren't running spamd. Spamd, by itself, filters out almost 90% of our inbound email. So far, I've had just two false positives from mail servers that weren't behaving correctly, that I had to whitelist manually. We're running spamd with its defaults, for now. spamd doesn't catch everything, but it works is a bit of an understatement. We've also been hit by backscatter, and I haven't had the time to figure out how to stop that one yet. - R.
Re: SMTP flood + spamdb
Hi! On Wed, Sep 26, 2007 at 02:03:03PM -0700, Rob wrote: [...] While watching the connection logs, I've noticed that a large majority of spammers get the first spamd response (250 Hello, spam sender. Pleased to be wasting your time.) and immediately disconnect. This suggests to me that rather than spend time trying to get whitelisted by spamd servers, they've mostly decided to skip them entirely and move on to servers that aren't running spamd. Interesting. Do you think they pattern match on the response, or do you think they disconnect if the initial greeting takes too long (spamd stutters for the first 10 seconds, in its default settings)? I'd guess the latter. [...] We've also been hit by backscatter, and I haven't had the time to figure out how to stop that one yet. For some, signed envelope senders or variations thereof work. That depends on a few circumstances. The basic idea is this: My email address is [EMAIL PROTECTED] Normal mail installations would send mails out with both the From header *and* the envelope sender set to [EMAIL PROTECTED] SES and similar schemes instead create a modified sender address like [EMAIL PROTECTED] That is used in the envelope. The header From address is left unmodified. TAG is a tag saying this is a address created using the envelope signing scheme, hannah is the original local part, timestamp can be made short by making it have only day granularity, and perhaps even only days modulo 2^ sig is a MAC, created from the local part, the timestamp and a host specific key. When a legitimate bounce (empty envelope from) is received, it must be in response to a mail recently sent out from our domain. If all mails sent out from our domain use the envelope signing scheme, bounces need only be accepted if they are to *signed* addresses that are recent enough and have a valid MAC. Bounces that don't fulfill that can be rejected (I'd reject after DATA or later so address verification will not lead to false positive rejects in other situations). In addition, bounces should be only addressed to exactly *one* recipient... Some also use SRS (sender rewriting scheme, from the SPF people), signing their own envelope as if the mail were forwarded, and accept bounce traffic only to SRS'ed addresses. - R. Kind regards, Hannah.
Re: SMTP flood + spamdb
Hannah, On 9/26/07, Hannah Schroeter [EMAIL PROTECTED] wrote: Hi! On Wed, Sep 26, 2007 at 02:03:03PM -0700, Rob wrote: [...] While watching the connection logs, I've noticed that a large majority of spammers get the first spamd response (250 Hello, spam sender. Pleased to be wasting your time.) and immediately disconnect. This suggests to me that rather than spend time trying to get whitelisted by spamd servers, they've mostly decided to skip them entirely and move on to servers that aren't running spamd. Interesting. Do you think they pattern match on the response, or do you think they disconnect if the initial greeting takes too long (spamd stutters for the first 10 seconds, in its default settings)? I'd guess the latter. I would guess the latter too, except that they tend to wait the full default 10 seconds until the first 250 response. I'm looking forward to increasing the stutter time to something on the order of 60 seconds and watching to see what happens then. We've also been hit by backscatter, and I haven't had the time to figure out how to stop that one yet. For some, signed envelope senders or variations thereof work. That depends on a few circumstances. The basic idea is this: [...snip...] That would be nifty, but I don't think it would work in our case. We have a number of customers that send mail through their own mail server (or another provider's mail server) and receive mail through ours (old email addresses, hosted domains, etc.). So far we've seen the backscatter come through in a quick burst from a handful of mail servers. For one example, one of our unlucky users received 800+ bounce messages from about four mail servers in Italy. I think I can use max-src-conn and max-src-conn-rate, plus a few whitelist entries for Google, Yahoo, etc., to stop that, but it requires careful monitoring. - R.
Re: SMTP flood + spamdb
Rob [EMAIL PROTECTED] writes: I would guess the latter too, except that they tend to wait the full default 10 seconds until the first 250 response. I'm looking forward to increasing the stutter time to something on the order of 60 seconds and watching to see what happens then. I have reports that increasing the -s value to 4 seconds serves to keep the stupid ones around for (you guessed it) four times as long, at least for the tarpitted ones. So the host which would hang on for 404 seconds earlier now beavers on for 1616, and so on. - P -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.datadok.no/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: SMTP flood + spamdb
On Wed, 26 Sep 2007 17:26:22 +0200, Peter N. M. Hansteen wrote: Or take advantage of the (by default) 25 minute window to use other means to detect that this address is sending spam. Perhaps spamd should be extended to look for excessive attempts to send messages from an address during that period? (How often do spammers' lists contain only one or two addresses from a domain?) You could probably use straight rdr instead of rdr pass to feed spamd, then in the relevant pass rule apply your source tracking options and overload and some table magic for that Have you been looking at my ruleset? ;-) I took out the pass on the rdr ages ago because unless I did my personal blacklist could not be used to block things like stormers and some tedious twits like a movie-house chain which keeps on sending to a long gone client of mine even though the address returns a 554 every time. I blacklist those permanently to stop log clutter. Rod/ _ Depressed? Me? Don't make me laugh! :Spike Milligan:1918-2002:
Re: SMTP flood + spamdb
On 9/23/07, Peter N. M. Hansteen [EMAIL PROTECTED] wrote: patrick keshishian [EMAIL PROTECTED] writes: I'm running spamdb in greylist mode, but these servers were getting white-listed very quickly. Then it sounds almost like you were running with a too short passtime, but then that's easy to adjust. The default (which I believe is 25 minutes). At around 1:40 PM (PDT) my SMTP server started getting flooded by enormous amount of connections. The connections were for seemingly random users @my-domain-name. We've been seeing a lot of that here, too. Mostly it's a few (maybe 20) a day to the most widely known domain here, then occasionally somebody pushes the generate button for too long and one domain almost nobody actually uses gets the bouces for 700+ fake addresses[1]. Bob Beck's greyscanner is rather effective, as is the more manual methods I've blogged about the observations quite a bit, starting with [2]. I have just re-opened my SMTP port which I had shut since 1440 Sunday. Not 1 hour has passed yet and my GREY list is almost at 300. I've added about 250 (count at the time) bogus emails to the greytrap list but since they are unique I don't think it will help the situation much. I'm very certain right now, this flood is due to a spammer using these fake addresses @my-domain-name to spam these mail server (all around the world -- Japan, South America, US, Germany, Ireland, etc...) and I'm getting the brunt of it in the form of these bounced messages. At this point I think I have no other choice but to wait out the storm. Short summary for those who are not too interested in blog posts: I started seeing more than the usual amount of bounce activity in my mail server log summaries, close enough to what you describe. So after a bit of thinking and log browsing I decided this was generated mainly by misconfigured mail servers bouncing spam. Then I decided I wanted to do an experiment, to see if I could poison the well and at the same time get a feel for the data I was collecting. When you speak of misconfigured mail servers bouncing spam, what exactly is a proper configured mail server supposed to do with spam directed at non-existing user @their-host-name? Just curious. FYI, as of now my: - GREY list count is 342 (and growing) - unique bogus email count is 341 - ESTABLISHED spamd connection count is 63 (and growing) This is not fun :-\ I started publishing the fake addresses on a web page[3] as well as entering them into the list of trap addresses. I've been seeing evidence that the addresses are actually being harvested and used as to-be-spammed addresses too: addresses which are all uppercase on the web page turning up in the spamd logs and greylist dumps in all lowercase, addresses which have been on my flypaper list for months turn up all the time, and we see a steadily growing number of hosts in TRAPPED state. My users here are not getting any more spam than they used to (as close as does not matter to none), false positives are pretty much an unknown, and it looks like we're succeeding in making the spammers work harder. [1] http://bsdly.blogspot.com/2007/08/lady-in-distress-or-then-again-maybe.html [2] http://bsdly.blogspot.com/2007/07/hey-spammer-heres-list-for-you.html [3] http://www.bsdly.net/~peter/traplist.html
Re: SMTP flood + spamdb
patrick keshishian [EMAIL PROTECTED] writes: When you speak of misconfigured mail servers bouncing spam, what exactly is a proper configured mail server supposed to do with spam directed at non-existing user @their-host-name? The real question in there is, what does a properly configured mail server do with spam? My answer is, if it gets as far as content filtering, drop it as soon as it's classified as spam, don't bounce it. Bouncing spam is never useful, the purported return address is extremely unlikely to be deliverable. A bounce is only useful for valid messages (which happen to be sent to a mistyped address), which in our context means that the message has passed greylisting and most likely some content filtering or other. In all likelihood you will still bounce to a few bogus ones, but taking this approach makes you a lot less noisy. The noise you are seeing is from sites which either don't bother much with filtering, or if they do, belong to that little cult of bouncing spam is good believers. - GREY list count is 342 (and growing) - unique bogus email count is 341 - ESTABLISHED spamd connection count is 63 (and growing) Unless your spamd box is extremely skinny, none of these figures are particularly worrying. spamd allocates IIRC about 12 kilobytes of buffers per tarpitted host, for greylist entries just another tuple in the database. My list of trap addresses, all harvested from stuff from out there, is just over 2700. Right now there are 273 hosts in the greylist at the gateway closest to where I'm sitting (my home net, actually), with 533 in TRAPPED state. This is not fun :-\ Well, it should not be a huge problem. IMO people who fake addresses in other people's domains should be prosecuted for some variety of fraud, but with the current level of digital competence in law enforcement that is just not going to happen. In the meantime we have reasonable countermeasures. See what greyscanner can do for you. - P -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.datadok.no/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: SMTP flood + spamdb
patrick keshishian wrote: I'm very certain right now, this flood is due to a spammer using these fake addresses @my-domain-name to spam these mail server (all around the world -- Japan, South America, US, Germany, Ireland, etc...) and I'm getting the brunt of it in the form of these bounced messages. At this point I think I have no other choice but to wait out the storm. Read up on backscatter spam. This is a deliberate attack on your domain. How it works: A spammer uses infected home user boxes to send random mail to various domains, with fake random addresses in your domain as the from or reply-to address. When the target domain of the initial domain does not do recipient validation at the smtp connection stage (as it should do), but spools and then rejects the mail - to you, hence you are the real target. Greylisting is of no use whatsoever because the servers sending the bounces to you are actual smtp boxes (sendmail, extrange, ), not malware, so they will quickly bypass spamd. Spamd greytraps will help a great deal, but you say that the addresses are random. How to cope with it: All you can do is make sure that you reject mail for unknown users at the smtp connection stage. You can rate limit most mail daemons so they don't overwhelm your box. Don't worry about it, I sometimes have up to 1300 messages a minute hitting my PII 350 box on a 500M ADSL and can not tell the difference when surfing about. How to run a mailserver: Reject mail for unknown users at the initial smtp connection stage. For valid users; either reject spam at the smtp connection stage, or spool it, process it later, tag it as spam and deliver it to the user's spam box - do not bounce it later as you will then be generating backscatter for some other poor soul. Note: some versions of exchange can not do recipient validation at the smtp connection stage, so this will always be a problem, and is yet another reason never to have exchange as an internet facing mail server.
