News from the front lines: In a world that has mostly benign spam where spammers with real return addresses send messages to valid recipients, qmail-scanner has its place. You can easily tag spam qith qmail-scanner so that the POP/IMAP clients can deal with the messsages appropriately.
For a small site with a few users ("few" < 1000), using .qmail-(USER|default) or user-based implementation rules is fine. For an ISP with thousands of users, it's not good enough anymore. Even qmail-scanner-queue doesn't help protect servers from the constant deluge of malignant messages. I've been finding that at a small ISP (20k users), the final delivery is far too late in the process to deal with spam. Address harvesters (sending to 99% invalid addresses to find the 1% that don't bounce) and spam blasters (sending spam to >3 invalid recipients per message) tax the server processing hard enough to cause problems, particularly from bounce addresses to forged senders. As the spammers get more persistent or desperate, they've been less gracious about how they spam. In one case recently, I had a DDoS from 40 sites sending similar spam all at once to/through our server to thousands of bad addresses. Our servers spent a whole weekend trying to deliver the bounce messages until I could clean/drain the queues of 85000 bounce messages. There's not much that qmail-scanner can do itself to protect the server. I am using two tools for the benefit of my users: SpamAssassin (www.spamassassin.org) Vexira virus scanner (www.centralcommand.com) If the spammers weren't too peristent, I'd be able to just use qmail-scanner-queue.pl and be mostly done. This worked for a couple months before our ISP became a heavily hit target (60% spam, 25% malignant spam). My implementation now includes: a qmail-smtpd that rejects mail based on environment variables set from tcp.smtp. a qmail.c hacked to provde better SMTP error codes based on qq exit codes. a rewitten qmail-scanner-queue that is highly optimized at letting SA/spamc and Vexira do their job with minimal system resources a qmail-send that injects bounce messages to the sender only when it's a non-malignant message (one-to-one communication to a valid recipient) a qmail-send that puts messages into a holding queue rather than fully processing them right away. An asynchronous program comes by and processes each message in the holding queue linearly to prevent load swings from simultaneous qmail-send/vdelivermail instances. a procmail-like perl program responsible for final delivery that queries a mysql database for a user's spam preference and uses those preferences to tag/delete/pass messages based on SA scores and user-defined keywords. A coworker made a web user interface. added functionality to auto-add and auto-remove statistically defined address harvesters and spam blasters to my tcp.smtp block lists (with appropriate 400 or 500 messages based on severity) a program to create a cdb database of valid users to help the filtering programs detect how many valid vs invalid users an IP address or netblock is attempting to send to. This my the third round of an on-and-off 6-month long fight. It's not about filtering spam anymore, it's about protecting our mail servers. As I leave, I have a big "I told you so" about how our CEO should have just subscribed to BigFish/Frontbridge and paying the extra money instead of going it alone. It would have saved money and reduced downtime if SPAM processing weren't our problem. The system is complex (some new perl/SspeedyCGI programs plus several patches to qmail and one to vdelivermail), but it actively provides negative feedback to spammers and harvesters with (hopefully) little to no administration from a mail administrator. The good news is that I'm about to finish up, and I don't have any IP restricions with the ISP, so I believe I'll be able to share most of my work. I hope to be posting some patches and programs soon. Another approach could have been to just integrate everything into SpamAssassin, but it's getting too huge already. Each of thousands of 4K messages doen't need to go through a program that sucks 16MB RSS memory. A large program isn't the most efficient place to block/route messages. -- Eric Ziegast internet!vix.com!ziegast Winning another battle in the losing war against spam.