OBSOLETE WHEN UA.CA IMPLEMENTS NEW SPAMLOGD Because traplist.gz sometimes expires hosts that are still sending spam to the world, I'd like to keep these addresses tarpitted for a while after they are removed, say 24 hours. This logic doesn't apply to the nixspam list because it contains addresses of legitimate hosts that temporarily send spam. (I've found that keeping these addresses tarpitted longer is counterproductive.)
This blacklist specifies single addresses (not blocks), so I could add/update all these addresses as TRAPPED entries in /var/db/spamd, but that would make the database quite unwieldy and also makes it impossible to see in the log files which blacklist it was. So I modified spamdb(8) to add an -f option for specifying an alternate db file and an -e option for removing all expired entries. Then I created a script that is called by cron every half hour (:15 and :45) and does the following: - sleep randomly 0..5 minutes to spread the peak load - fetch traplist.gz using wget/curl (because ftp(1) doesn't do HTTP timestamping) - add/update the addresses from this list in a separate db file - remove expired db entries - dump the db into a new blacklist file - run spamd-setup(8), aggregating this new file (and a few others) I've been running this set-up for a few months now. The DB manipulation places a significant load on the server, but I believe that further optimizations are possible. Does anyone feel the need to comment on this approach? -- Boudewijn Dijkstra Indes-IDS B.V. +31 345 545 535
