Re: OT: Trigger words in email addresses?
GMail just... sucks. I have an email server in EC2 that also passes all tests, but they insist on dumping our emails into users' spam folders. Good luck trying to get anyone at GMail to actually do their jobs and change whatever is causing them to mark your emails as spam. In my case, they are not coming from donotreply@, so I don't think that address does anything towards marking your mail as spam. Thomas On 4/7/24 20:40, Jerry Malcolm wrote: Slightly off-topic from SpamAssassin specifically. But I have a question about certain email addresses triggering spam filter scores. I know anybody can create any rule they want to. I just want to understand best practices and recommendations. I work for a medium size but growing company that needs to have user accounts verified. Same process a billion other sites use. I send an email with a link. The user clicks the link, and voila...validated. The problem is that gmail, in particular continues to insist on putting these in spam folders and (theoretically) discarding some of them completely. Some of users swear they never get them and then go on social media, etc disparaging our company. You know the drill. Some end up with a typo in their email address, and some finally figure out they have a spam folder. But this is big problem that it's not showing up in everyone's inbox. I have validated my outbound emails with mail-tester.com and get a 10/10 perfect score. So SPF, DKIM, DMARC, everything is correct. Now here's my question (at least one of them)... I send the validation email from donotre...@xyz.com. We have a ticket reporting system and seriously want to discourage users from sending in problem reports by email. DoNotReply is actually a legit inbox, and I monitor it to catch users that haven't yet mastered the art of reading. I want to keep that DoNotReply email address to tell the user "don't send an email to this address" But I have a co-worker that is convinced that "donotre...@xyz.com" is a trigger for gmail's spam filters and all spam filters will score the email higher as spam due simply to that word in the email address. I'm not convinced. I do not want to change it to something else that will encourage users to start inundating us with questions/problems by email instead of using our established ticket system.. But I also don't want to be shooting myself in the foot with spam filters by using that name if it's indeed a trigger word. So... recommendations, please... should I change donotre...@.com to something else, and if so, what is the accepted (non-spam-trigger) email address to use to still get the point across to not send anything to that account? Secondly... more generally, any suggestions on how to crack the gmail code and make them know we aren't spammers? BTW we are generating these emails from an AWS EC2 server and using AWS's SES SMTP server for outbound. The emails are html and have a little bit of border, font, and embedded logo. Content is a Click here to validate your account and an https link, followed by a thank you. I can remove the letterhead and footer, but then I'm worried about get a "not enough content with a link" rule triggered. Help! Thanks, Jerry
Re: OT: Microsoft Breech
On 3/19/24 09:52, Michael Storz wrote: Am 2024-03-19 14:51, schrieb Thomas Cameron: Does anyone else just block all traffic from *.onmicrosoft.com? I have literally NEVER gotten anything from that domain which is not obvious junk. We block and have a whitelist with 49 entries at the moment. Michael Thanks, sir. I will whitelist anyone who complains, but like I said... I've literally never gotten email from that domain which was not spam. -- Thomas
Re: OT: Microsoft Breech
I am using this setup in my postfix main.cf. [obfuscated] is my actual key for spamhaus. smtpd_recipient_restrictions = check_sender_access regexp:/etc/postfix/sender_access permit_mynetworks permit_auth_destination permit_sasl_authenticated reject_rbl_client [obfuscated].zen.dq.spamhaus.net=127.0.0.[2..11] reject_rhsbl_sender [obfuscated].dbl.dq.spamhaus.net=127.0.1.[2..99] reject_rhsbl_helo [obfuscated].dbl.dq.spamhaus.net=127.0.1.[2..99] reject_rhsbl_reverse_client [obfuscated].dbl.dq.spamhaus.net=127.0.1.[2..99] reject_rhsbl_sender [obfuscated].zrd.dq.spamhaus.net=127.0.2.[2..24] reject_rhsbl_helo [obfuscated].zrd.dq.spamhaus.net=127.0.2.[2..24] reject_rhsbl_reverse_client [obfuscated].zrd.dq.spamhaus.net=127.0.2.[2..24] reject I was still getting a TON of junk from onmicrosoft.com. I blocked the domain many months ago... Do you recommend I let that back open? I definitely don't want to miss emails from folks who use outlook.com (although, not gonna lie, it feels nice to raise a middle finger to Microsoft for their terrible email practices). -- Thomas On 3/19/24 09:02, Marc wrote: I am using spamcop and spamhaus to block. There are indeed outlook.com ip addresses that bounce. Does anyone else just block all traffic from *.onmicrosoft.com? I have literally NEVER gotten anything from that domain which is not obvious junk. I set up postfix to just flat out refuse anything from that domain.[1] If I get any complaints, I may ease it up, but I was getting TONS of spam messages from that domain and I figured it was easiest to just block it.
Re: OT: Microsoft Breech
Does anyone else just block all traffic from *.onmicrosoft.com? I have literally NEVER gotten anything from that domain which is not obvious junk. I set up postfix to just flat out refuse anything from that domain.[1] If I get any complaints, I may ease it up, but I was getting TONS of spam messages from that domain and I figured it was easiest to just block it. -- Thomas [1] [root@east ~]# grep onmicrosoft /etc/postfix/sender_access /@*.onmicrosoft\.com/ REJECT [root@east ~]# grep sender_access /etc/postfix/main.cf check_sender_access regexp:/etc/postfix/sender_access On 3/18/24 21:13, Jimmy wrote: It's possible that certain email accounts utilizing email services with easily guessable passwords were compromised, leading to abuse of the .onmicrosoft.com subdomain for sending spam via email. I've observed an increase in the blocking of IPs belonging to Microsoft Corporation by the SpamCop blacklist since November 2023, with a notable spike in activity during February and March 2024. Jimmy On Tue, Mar 19, 2024 at 12:10 AM Jared Hall via users mailto:users@spamassassin.apache.org>> wrote: I've several customers whose accounts were used to send spam as a result of Microsoft's infrastructure breech. Curiously, NOBODY has received any breach notifications from Microsoft, despite personal information being compromised. What has anyone else experienced? Thanks, -- Jared Hall
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/19/24 16:32, Byung-Hee HWANG wrote: There is a filtering rule in Gmail: *Never send it to Spam* I apply that rule to extremely important emails such as debian-bugs- dist and debian-devel-announce. You know that. I know that. But trying to explain to the board members I'm helping out is... painful. Thomas
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/19/24 14:33, Matija Nalis wrote: You would need to encourage at least several of the recepients (the more the better) to click on "Not spam" button on GMail on such mails. Then it will (eventually) start accepting them normally. Yup, that's basically what I've been doing. see e.g. https://serverfault.com/questions/953486/repairing-e-mail-domain-reputation-on-google I suspect that Google might even doing it on purpose, in order to "encourage" even more users to be locked in their e-mail walled-garden ecosystem. Google being anti-competitive? I'm shocked! SHOCKED, I say! -- Thomas
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/7/24 05:40, Matus UHLAR - fantomas wrote: I built email servers for a non-profit I volunteer for. If email comes into the server for presid...@myassociation.org, I would normally just create an alias in /etc/aliases so that emails to president@ get forwarded to the president's "real" email address, say presidents_real_em...@gmail.com. postfix supports expand_owner_alias, which, when you are sending to al...@example.com, will set sender to owner-al...@example.com. That way SPF should pass. The problem is, when I send email to presid...@myassociation.org, gmail rejects the forwarded email because it appears to come from my personal domain, not the mythical myassociation.org domain. DKIM, DMARC, and SPF all fail, which I totally understand. How can I make this work? DKIM should not fail, unless you modify the message. Do you modify the message? On 07.01.24 19:07, Byung-Hee HWANG wrote: See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1043539#88 Cite: If your dkim signature is OK, then Gmail does accept all mails. So never use SRS. DKIM is enough. This is not a good advice. Whoever filters SPF at SMTP time will reject that message. Gmail is not the only mail service available. Initially, I was seeing errors where GMail didn't list SPF as "passed." But after about an hour, it started passing. I think it was an old DNS record that finally expired. The forwarded email is being *accepted* by GMail. My issue now is that GMail drops it into the recipient's spam folder. I suspect it's a reputation thing. Once the server is up and running for a while, I'm hoping that GMail will stop flagging the emails from the server as spam. Thomas
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/7/24 04:07, Byung-Hee HWANG wrote: Hellow Thomas, See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1043539#88 Sincerely, Byung-Hee The issue is not so much that GMail doesn't accept the email. It does, since I have DKIM, DMARC, and SPF set up. But it drops it into the spam folder every time. So when I'm sending emails to someone's alias, they have to check their spam folder. Even when they mark it as "not spam," GMail still drops it into the spam folder. It's very frustrating. Thomas
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/4/24 06:35, Matus UHLAR - fantomas wrote: On 03.01.24 20:36, Thomas Cameron wrote: Fair point. But I'm guessing that because it has two DKIM signatures, it's not passing the DKIM check. only one of those DKIM dignatures needs to pass, with the domain in From: Yup, and it seems to be working now. After about an hour, it suddenly started working as expected. GMail doesn't flag it as "passed" for DKIM. I am looking to see if PostSRSd has any sort configuration option to delete the DKIM of the original sending server so that it will "pass" DKIM checks. Not sure why pass is in quotes. But again if you don't change headers the original signature should be valid. Well, it's not marked as failed, and it's not marked as passed, but I am looking at the OpenDKIM headers. It's in a weird limbo where I can see the email got marked but GMail is not marking it either way. can we see headers From: and Authentication-Results as they were seen on your server? I absolutely can send them, but since it's working now, I'm going to blame this on Google and run. :-D -- Thanks! Thomas
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/4/24 06:31, Matus UHLAR - fantomas wrote: On 03.01.24 19:30, Thomas Cameron wrote: Thanks for the advice on SRS - I have set it up and it's mostly working. At least GMail accepts the emails, although it seems to be failing DKIM and DMARC tests. I'm digging into what, if anything, can be done to make PostSRSd fix this issue. DKIM fails if the message is modified in your server (or, if DKIM failed already when it came to it) DMARC fails if neither DKIM nor SPF succeed, where DKIM signature or the SPF record must be from the domain in From: When you forward e-mail, SRS makes sure SPF record is from your domain, but the DKIM signature must be made by sending server, so forwarded messages without valid DKIM signature will not pass. The weird thing is, after a little while, everything seems to be working just fine. When I send an email to one of the aliases on the server, it sends it to the "real" email address at GMail. It now passes SPF, DMARC, and DKIM tests. Looking in the headers on GMail, I see both DKIM signatures, from the server which sent the original email, and the one on our mail server. I have no idea why GMail was saying it didn't pass checks earlier. I saw the same DKIM signatures in the headers before. Anyway, SRS is very cool, and I appreciate all the folks who pointed me to it. -- Thanks for the advice, Matus! Thomas
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/3/24 19:45, Greg Troxel wrote: Thomas Cameron writes: Yeah, the weird thing is, when I check the forwarded email on GMail, I see in the headers that both the original sending email server (call it mail.somedomain.com) and the relay server (call it mail.myassociation.org) put DKIM signatures in the message. That's more or less broken in my opinion. I think an MTA should only DKIM-sign messages that it is responsible for in the sense of origination, because it is from an authenticated sender. Fair point. But I'm guessing that because it has two DKIM signatures, it's not passing the DKIM check. GMail doesn't flag it as "passed" for DKIM. I am looking to see if PostSRSd has any sort configuration option to delete the DKIM of the original sending server so that it will "pass" DKIM checks. Not sure why pass is in quotes. But again if you don't change headers the original signature should be valid. Well, it's not marked as failed, and it's not marked as passed, but I am looking at the OpenDKIM headers. It's in a weird limbo where I can see the email got marked but GMail is not marking it either way. Thomas
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/3/24 17:41, Greg Troxel wrote: You are overlooking that DKIM from the original From: is the responsibility of that domain and that if you do not modify the message then it should still pass. Domains sending without DKIM are going to be a mess. Yeah, the weird thing is, when I check the forwarded email on GMail, I see in the headers that both the original sending email server (call it mail.somedomain.com) and the relay server (call it mail.myassociation.org) put DKIM signatures in the message. GMail doesn't flag it as "passed" for DKIM. I am looking to see if PostSRSd has any sort configuration option to delete the DKIM of the original sending server so that it will "pass" DKIM checks. Thomas
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/3/24 15:44, Bill Cole wrote: Indeed: your solution is known as "SRS" (Sender Rewriting Scheme) and it has multiple implementations. If you forward mail, you will break SPF unless you fix the envelope sender so that it uses a domain that permits the example.org server to send for it. OR, you could instead deliver to a POP mailbox locally and have users fetch from there instead of simply forwarding mail to them. This also avoids a completely distinct problem of places like GMail deciding that your org's mail server is a spamming service because it is forwarding spam. If users POP their mail instead of having it forwarded via SMTP, that does not happen. Thanks for the advice on SRS - I have set it up and it's mostly working. At least GMail accepts the emails, although it seems to be failing DKIM and DMARC tests. I'm digging into what, if anything, can be done to make PostSRSd fix this issue. Many thanks for your help, it's genuinely appreciated! Thomas
[SOLVED] Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/3/24 18:16, Michael Grant wrote: Here's what I have done in the past from my server to get around this situation you are having: 1. In my .procmailrc file :0c: !exam...@gmail.com This sends a copy (the c flag in first line) of the message to the gmail account and leaves a copy in your inbox. 2. From your exam...@gmail.com acct, go to Settings -> Accounts and Import. Under the section 'Check email from other accounts', Add an email account. Then add your server's account and use POP to suck over emails as they arrive. Have it delete the emails once they are sucked over. What this does is it causes messages to be forwarded to gmail, but some small number of them bounce because of whatever decision gmail makes. But those messages are popped in later, so there's no lost mail. Gmail de-duplicates the messages so you don't get messages twice, and it never refuses to pop the messages in. Popping in messages is slow, so when the forward works (which seems to be most of the time), mail comes in quick, unless it bounces, in which case, it's popped in a few minutes, sometimes 10s of minutes, later. If you are concerned about the bounce messages going back into your mailbox (gmail doesn't loop here fortunately), you can write a procmail rule to siphon those off into another folder or into /dev/null. (Left as exercise for the reader...) 3. You *may* need to do one further thing, you may need to go back into gmail's Account and Import settings and set up 'Send mail as' and set up to send mail as your email address on your server. I can't remember if gmail does this automatically for you in step 2 above or not. 4. You probably want to then click the radio button "Reply from the same address to which the message was sent". Otherwise, when you reply, it'll come from your gmail address and not your server's email address. These radio buttons only appear once you have at least one Send As address set up. Michael Grant This is super helpful, thank you very much! I was not aware you could configure GMail to pull from another account, that's incredibly helpful! I wound up installing PostSRSd (https://github.com/roehling/postsrsd/tree/main). Now, when I send email to one of the officers in the non-profit, I have their actual email address set up in /etc/aliases, and SRSd rewrites the headers so that GMail at least accepts them now. Before, it was just flat out rejecting them. The annoying thing is that when I send email from the mail server I set up, even though it *passes* SPF, DKIM, and DMARC (https://imgur.com/a/FuA6HiK), GMail is still dumping into the Spam folder. It's incredibly irritating. After I marked a handful of them "not spam," it stopped doing it, but we're going to be sending emails to the members of the association (and I know several use GMail). I really don't know what the heck I am supposed to do to get GMail to stop dropping the messages into the spam folder. I thought you could set up some sort of DNS TXT record for Google to show that you're a legit sender, but I can't find documentation for it except for Google Workplaces. Anyway, thanks everyone for the great suggestions! I learned a lot doing this, and I was unaware of SRS... That's fantastic info! -- Thomas
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/2/24 17:51, Andy Smith wrote: Hi Thomas, On Tue, Jan 02, 2024 at 04:24:37PM -0600, Thomas Cameron via users wrote: I built email servers for a non-profit I volunteer for. If email comes into the server for presid...@myassociation.org, I would normally just create an alias in /etc/aliases so that emails to president@ get forwarded to the president's "real" email address, say presidents_real_em...@gmail.com. This causes your server to pass on email without changing envelope sender, so your server is purporting to be whoever the email is originally from. Any email authentication measure working on the envelope sender, such as SPF, will then fail, as your server is indistinguishable from a random host forging the original sender's domain. Yup, that's exactly what's happening. Email from an association member may come in from u...@otherdomain.com and when it gets forwarded to GMail, they reject it because the mail server isn't otherdomain.com's email server. I get *why* it's failing, I was just hoping someone had a better idea. How can I make this work? Is there a good way to use something like /etc/aliases to forward emails to the domain I manage to another recipient? Or is there something better I can do? You need to give up on /etc/aliases for external routing of email unless you control all the original sender domains and can for example add your server IPs to its authentication mechanisms (e.g. SPF). Since you probably can't do that for any recipient domain that expects to receive Internet email, you need to either: - Implement Sender Rewriting Scheme (SRS) so that your server takes responsibility for forwarded emails with its own envelope sender. https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme This is excellent, I was not aware of it. I'm digging into it now. I was playing around with using a procmail recipe to munch the "from" address, but SRS looks like a MUCH better plan. Thank you so much! Or: - Have your users collect their your-org email by some means other than SMTP, such as running an IMAP server and having them view both their gmail mailbox and their your-org inbox in one place (I have no idea if that is feasible with gmail). This is what *I* would do, for sure. But the members of the association are incredibly non-technical, and trying to walk them through setting up an email client like Thunderbird or Outlook is a recipe for disaster. I really like the SRS idea, I'm digging into that now. Thanks, Andy Thanks a bunch! Thomas
Re: Question about forwarding email (not specifically SA, pointers greatly appreciated)
On 1/3/24 01:21, Jared Hall wrote: On 1/2/2024 5:24 PM, Thomas Cameron via users wrote: The problem is, when I send email to presid...@myassociation.org, gmail rejects the forwarded email because it appears to come from my personal domain, not the mythical myassociation.org domain. DKIM, DMARC, and SPF all fail, which I totally understand. How can I make this work? Is there a good way to use something like /etc/aliases to forward emails to the domain I manage to another recipient? Or is there something better I can do? You will probably find that forwarding Emails to most systems, including MSN/Live/Hotmail/Outlook and Yahoo/AOL works OK (for now). But if you want Vacation/Out-Of-Office/Autoresponders to work to Gmail addresses, you MUST run DKIM on your managed domain. Even valid SPF alone will NOT do. I actually set up SPF, DMARC, and DKIM on the non-profit's email server. It works fine if I send email from the server. The rub is, I want all emails to presid...@example.org to be forwarded to presidents_real_addr...@gmail.com. Since the forward happens at mail.example.org, the "from" is from some other domain from example.org, so it fails all the tests. Implementing DKIM w/ DMARC is a good, if not the best, practice. Considering present trends, SPF/DKIM/DMARC Auth-neutral will become the new "bad". Oh, I firmly agree with you. I have all three services configured, and I wouldn't deploy a mail server without them. This is just an odd corner case where the easiest thing to do is just redirect emails to the non-profit's president's real email address. Instead of using /etc/aliases, I'm playing around with a procmail recipe to munge the "from." We'll see if it works. I apologize this isn't strictly SA related, I am just hoping someone can give me advice or provide I link to follow on how to make this work. package: opendkim + access to your managed domain's DNS records. I agree, and that's already done. Thanks, sir! Thomas
Question about forwarding email (not specifically SA, pointers greatly appreciated)
Howdy, all - This is not strictly SpamAssassin related, but y'all probably know where to point me to make this work. I built email servers for a non-profit I volunteer for. If email comes into the server for presid...@myassociation.org, I would normally just create an alias in /etc/aliases so that emails to president@ get forwarded to the president's "real" email address, say presidents_real_em...@gmail.com. The problem is, when I send email to presid...@myassociation.org, gmail rejects the forwarded email because it appears to come from my personal domain, not the mythical myassociation.org domain. DKIM, DMARC, and SPF all fail, which I totally understand. How can I make this work? Is there a good way to use something like /etc/aliases to forward emails to the domain I manage to another recipient? Or is there something better I can do? I apologize this isn't strictly SA related, I am just hoping someone can give me advice or provide I link to follow on how to make this work. Thanks, Thomas
Re: Really hard-to-filter spam
On 8/4/23 02:15, Sean Greenslade wrote: On Wed, Aug 02, 2023 at 04:17:22PM -0500, Thomas Cameron via users wrote: On 8/2/23 15:52, David B Funk wrote: I have the users move spam to an imap folder, and then run (via the user's cron job): sa-learn --mbox --spam /home/[username]/mail/spam If something is flagged as spam and it's not supposed to be, I have them copy it to the ham folder and I run (also via cron job): sa-learn --mbox --ham /home/[username]/mail/spam Hopefully this is just a typo in your email, but the above line trains your spam folder as if it's ham. That could easily cause your screwed-up bayes scores. --Sean It was a typo, sorry. I have a cron job that uses --spam against the spam folder, and --ham against the ham folder. I just copied and pasted poorly. This is the actual script for my account: [thomas.cameron@mail-east ~]$ cat bin/spamcheck #!/bin/bash sa-learn --progress --spam --mbox /home/thomas.cameron/mail/INBOX/spam sa-learn --progress --ham --mbox /home/thomas.cameron/mail/INBOX/ham Bayes tests for other messages, like the one you sent me, looks like this: -- Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mail-east.camerontech.com X-Spam-Level: X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 -- But messages flagged as spam look like this: -- Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mail-east.camerontech.com X-Spam-Flag: YES X-Spam-Level: X-Spam-Status: Yes, score=36.8 required=5.0 tests=BAYES_99,BAYES_999, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FROM_FMBLA_NEWDOM, FROM_SUSPICIOUS_NTLD,FROM_SUSPICIOUS_NTLD_FP,HTML_IMAGE_ONLY_32, HTML_MESSAGE,PDS_OTHER_BAD_TLD,RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK, RCVD_IN_DNSWL_HI,RDNS_NONE,SH_HELO_DBL,SH_HELO_ZRD_FRESH, SH_ZRD_HEADERS_FRESH,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_ABUSE_SURBL,URIBL_BLACK,URIBL_ZRD shortcircuit=no autolearn=spam autolearn_force=no version=3.4.6 -- The previous email I copied headers from as an example was just a bad example. Usually Bayes is /pretty/ accurate on my system. I only used that one because it was a message which made it through SpamAssassin. I was trying to demonstrate that the checks were not failing, as suggested in an earlier comment. Thanks for catching that, though. I have made silly mistakes like that so I appreciate you checking me. -- Thomas
Re: Really hard-to-filter spam
On 8/2/23 15:52, David B Funk wrote: Regardless, if a message has never been seen before and has little correlation to earlier messages its Bayes should hit someplace in the 40% to 60% range. The fact that it hit 00% indicates a strong correlation to lots of ham (or something is screwy with your Bayes). OK, here's what I got just now: [thomas.cameron@mail-east ~]$ sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 41449 0 non-token data: nspam 0.000 0 49720 0 non-token data: nham 0.000 0 162741 0 non-token data: ntokens 0.000 0 1689089541 0 non-token data: oldest atime 0.000 0 1691009577 0 non-token data: newest atime 0.000 0 1691007146 0 non-token data: last journal sync atime 0.000 0 1690991018 0 non-token data: last expiry atime 0.000 01382400 0 non-token data: last expire atime delta 0.000 0 13879 0 non-token data: last expire reduction count I can absolutely re-train Bayes. I am kind of an email pack-rat, so I have over a gig of saved known good emails in various folders. I have SA set up so that emails are scanned individually on a per user basis via procmail rule: [thomas.cameron@mail-east ~]$ head .procmailrc MAILDIR=$HOME/mail LOGFILE=$MAILDIR/procmail.log :0fw: spamassassin.lock * < 512000 | spamassassin I have the users move spam to an imap folder, and then run (via the user's cron job): sa-learn --mbox --spam /home/[username]/mail/spam If something is flagged as spam and it's not supposed to be, I have them copy it to the ham folder and I run (also via cron job): sa-learn --mbox --ham /home/[username]/mail/spam For my email account, I've used my inbox and various other folders to train Bayes in the past (although it's definitely been a while since I did Bayes maintenance), but I have zero issue nuking my personal Bayes data and starting over. Thoughts? -- Thomas
Re: Really hard-to-filter spam
On 8/2/23 14:32, Dave Funk wrote:
On Wed, 2 Aug 2023, Thomas Cameron via users wrote:
Wow! What a charming response! You must be a LOT of fun at parties,
and have lots of friends!
Please don't feed the troll. There's a reason that Reindl is blocked
from this list.
I was not aware, and I apologize.
No, I did not get that response. I don't have any of those specific
spam to sample, as I have not gotten one today. But the last spam I
got that
slipped through SA had this score:
X-Spam-Status: No, score=-5.1 required=5.0
tests=BAYES_00,DEAR_SOMETHING,
DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,
HTML_MESSAGE,RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H2,RCVD_IN_PBL,
SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE shortcircuit=no
So nothing about any tests not working, or queries being rejected.
Nothing that looks like misconfiguration on my end. I am not saying
there are
no misconfigurations on my end, but if there are, it's not super
obvious to me.
The fact that you're getting BAYES_00 on that message indicates that
Bayes -really- thinks it's ham.
Given that you've trained multiple instances of this kind of message
to Bayes as spam but it still gets BAYES_00 score means one of two
things:
1) Either you've got thousands of instances of similar messages that
were learned as 'ham'
2) or the database that Bayes in your running SA instance is using is
not the same one that you were doing your training to.
This could be configuration issues or pilot error (using the wrong
identity when doing the training, training on the wrong machine, etc).
On your SA machine what does the output of "sa-learn --dump magic"
show you?
(IE how many nspam & nham tokens, what is the newest "atime", etc).
If careful config & log inspection doesn't give clues, try this
brute-force test.
Shut down your SA, move the directory containing your Bayes database
out of the way and create a new empty one.
("sa-learn --dump magic" should now show 0 tokens).
Then train a few ham & spam messages (only a dozen or so), recheck the
--dump magic to see that there are now some tokens in the database but
not too many.
Restart your SA and watch the log results. If there are fewer than 200
messages (both ham & spam) in your Bayes database then SA won't use
it, so make sure that's the case, your new database should be too
empty for SA to be willing to use it.
So if you -are- getting Bayes scores then that indicates that SA is
using some database other than what you think it has.
Now start manually training more messages (spam & ham). When you hit
the 200 count threashold Bayes scores should start showing up in your
logs.
Good luck.
Thank you very much. The message that slipped through today was NOT one
of the ones being discussed in this thread, it was a different format
and totally different message. I only included it to demonstrate that my
server was not being rejected for queries as the blocked user intimated.
I will dig deeper into the --magic and make sure I'm feeding Bayes with
spam and ham.
Thanks for your response, and again, I apologize for leaking that user's
garbage to the list. I was not aware that he was blocked.
--
Thomas
My apologies
I was notified privately that Reindl Harald is blocked on this list. I replied to him and accidentally polluted the list with more of his toxicity. I apologize, and I've blocked him on my mail server, as well. I'm sorry for posting that. -- Thomas
Re: Really hard-to-filter spam
On 8/2/23 13:28, Reindl Harald wrote: then i bet you have the same "RCVD_IN_ZEN_BLOCKED_OPENDNS" as the OP which means you are not capable to operate a mailserver https://www.spamhaus.org/returnc/pub/ throwen against our spamfilter it would be blocked without any question - above 8.0 points the spamass-milter rejects Content analysis details: (32.3 points, 5.5 required) pts rule name description -- -- 1.0 CUST_DNSBL_26_UCE2 RBL: dnsbl-uce-2.thelounge.net (dnsbl-2.uceprotect.net) [60.176.201.72 listed in dnsbl-uce-2.thelounge.net] 6.5 CUST_DNSBL_4_ZEN_PBL RBL: zen.spamhaus.org (pbl.spamhaus.org) [60.176.201.72 listed in zen.spamhaus.org] 5.5 CUST_DNSBL_6_ZEN_XBL RBL: zen.spamhaus.org (xbl.spamhaus.org) 1.0 CUST_DNSBL_25_NSZONES RBL: bl.nszones.com [60.176.201.72 listed in bl.nszones.com] 5.5 BAYES_80 BODY: Bayes spam probability is 80 to 95% [score: 0.9084] 0.1 HK_RANDOM_ENVFROM Envelope sender username looks random 0.1 HK_RANDOM_FROM From username looks random 6.5 CUST_DNSBL_2_SORBS_DUL RBL: dnsbl.sorbs.net (dul.dnsbl.sorbs.net) [60.176.201.72 listed in dnsbl.sorbs.net] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.1 SPF_NONE SPF: sender does not publish an SPF Record 0.0 HTML_MESSAGE BODY: HTML included in message 0.1 TVD_SPACE_RATIO No description available. 2.5 RDNS_NONE Delivered to internal network by a host with no rDNS -0.0 T_SCC_BODY_TEXT_LINE No description available. 0.5 INVALID_MSGID Message-Id is not valid, according to RFC 2822 2.5 TVD_SPACE_RATIO_MINFP Space ratio (vertical text obfuscation?) 0.5 BOGOFILTER_PROB_SPAM BOGOFILTER: No description available. Wow! What a charming response! You must be a LOT of fun at parties, and have lots of friends! No, I did not get that response. I don't have any of those specific spam to sample, as I have not gotten one today. But the last spam I got that slipped through SA had this score: X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00,DEAR_SOMETHING, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, HTML_MESSAGE,RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H2,RCVD_IN_PBL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE shortcircuit=no So nothing about any tests not working, or queries being rejected. Nothing that looks like misconfiguration on my end. I am not saying there are no misconfigurations on my end, but if there are, it's not super obvious to me. Cheers! -- Thomas
Re: Really hard-to-filter spam
On 7/28/23 00:23, Bill Cole wrote: 1. There are milters/content-filters that decode Base64 message parts (amavisd-new, mimedefang, etc) for processing by SA. 2. There are still sufficiently unique items: First-Name-Only, Mixed-Case word in the Subject (NLP modeling), and a Base-64 encoded HTML attachment (w/ UTF-8 encoding no less). Combined in a Meta rule, these innocuous items will likely hit with good accuracy even without Base64 decoding. Umm, unless I'm really missing something here the usual SA processing decodes such body stuff (QP, Base64, etc) and feeds the "cleaned" text to the rule processing engine. Correct. It has nothing to do with the calling glue. You have to work hard to get matches done on the raw stuff if you want to do special rule matching on the un-decoded body. Correct. That should only be needed in rare cases where you're looking for a pattern in a non-text part. I'm not sure why the OP's rule didn't match the target message, but it is NOT because of the Base64 encoding of parts with the 'text' primary MIME type. If I had to guess, I'd look for invisible characters hidden in the text (e.g. Unicode "zero width non-joiner" marks and the like) that break the pattern and for lookalike non-ASCII characters (often Cyrillic or Greek) in the target string. I am seeing the same issue. I get those same emails, with that 132.1532.1334 string or similar. SA is definitely not catching them, even though I dump them into my spam folder and run sa-learn --spam against them day after day. How can I check to see if it's actually decoding the base64? Or is that just a fact? It seems incredibly weird that I get these things every day, I mark them as spam every day, and they never hit more than a couple of points on the spam scale. Thomas
Re: Sudden surge in spam appearing to come from my email address
On 7/17/23 11:03, Reindl Harald wrote: - for rejecting spoofed envelopes nothing easier than that you need to have a full list of addresses you receive mail anyways, so any message with one of those addresses without authentication can be safely rejected main.cf smtpd_recipient_restrictions: check_sender_access proxy:hash:/etc/postfix/spoofing_protection.cf [root@mail-gw:/etc/postfix]$ head spoofing_protection.cf yourlocaladdress1 REJECT Sender Spoofed yourlocaladdress2 REJECT Sender Spoofed .. Many thanks. I'll figure out how to do this with sendmail, since that's what I use (yeah, I'm old). Thomas
Re: Sudden surge in spam appearing to come from my email address
On 7/16/23 17:57, Benny Pedersen wrote: back to basic: why accept local envelope SENDER domains on port 25 ? its safe to reject them its not a question on spf or stupid srs rewrites That's actually a great point. So you're saying to tell sendmail to reject emails purporting to come from me if they come from another mail server? Got a pointer to documentation on how to do that? I'm all ears. Thomas
Re: Sudden surge in spam appearing to come from my email address
On 7/16/23 00:41, Matija Nalis wrote: On Sat, Jul 15, 2023 at 10:04:18PM -0500, Thomas Cameron wrote: pass fail So, it fails SPF, but DKIM passes. Meaning, your mail would pass normally modern servers which check both. If you do not want to receive such status messages, you should update your DMARC records (currently _dmarc.camerontech.com indicates you want to receive BOTH aggregate "rua=" and forensic "ruf=" reports; and that you want to receive status updates when the message would've passed normally via "fo=1") Thanks. I set it up to send me everything it could, to see if I had done anything wrong. I will amend my DNS records as you suggested. So it seems like my emails are being quarantined when I send them to mailing lists, even this one. What? No. At least not in this report you shared. You seem to be confusing "" section (which is just a dump of DNS which that server sees) with actual ""s leading to final "" of "none" (which is good, as opposed to "reject" or "quarantine" which would not be). Ah, cool, thanks for the clarification! I saw "quarantined" and thought my emails were not getting through. You probably might want to use some nice frontend to visualizing DMARC results, if reading XML and SPF/DKIM/DMARC protocol internals is not second nature for you. e.g. https://github.com/topics/dmarc-reports I will definitely check that out, thanks! +1 for encouraging mailing list operators to get with the times. You can also do as Robert suggests and use a separate (sub)domain for mailing lists with different SPF settings thereon. It's not so much mailing list operators I'm worried about. It's that, when my email goes through a listserv mailing list, if I define hard failures, I am worried that my email isn't going to get to list members. That's not the mailing list admin, it's the admins of the list members' mail servers. If I'm not understanding something, please feel free to clarify. If mailing list is employing SRS, mail reaching final recipients would not be failing SPF checkes, as envelope sender (i.e. SMTP's "MAIL FROM: ") would be rewritten as the mail is coming from mailing list domain and their servers (as it would), not yours. See https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme Only if the mailing list remailing server leaves original (your) envelope sender (which it shouldn't be doing, yet often does), would you get such SPF problems. So, SPF problem is solvable from mailing list server side, if its admins are willing. Also, if your mails are signed by DKIM, and mailing list software is not rewriting signed headers nor body (as it shouldn't, but some mailing lists try to add annoying text to the bottom of messages like "to unsubscribe, do xyz", thus breaking both DKIM, S/MIME and PGP signatures), then your mail should pass DKIM checks too. So that problem is avoidable on mailing list server side too. Thank you so much, I am reading these articles now! I really appreciate your not busting my chops for not knowing this. -- Thomas
Re: Sudden surge in spam appearing to come from my email address
On 7/15/23 23:40, Loren Wilton wrote: > I assume this just needs to go in /etc/mail/spamassassin/local.cf, right? Or do I need to do separate stanzas for each domain? If you want this to work for all users, yes. If you have per-user rules enabled, then it could go in user_prefs for that user. The rules I posted assumed one sender u...@org.xxx, whth a known first and last name. If you have multiple personalities, then you have multiple "me's": us...@org.xxx, us...@org.xxx, and so on, then you needs to probably duplicate the rule set for each user. Probably all of the users have different first and last names. I'd probably change the meta rule name from NOT_FROM_ME to NOT_FROM_USER1, NOT_FROM_USER2, etc. If you have one "me" but multiple accounts for that person, then probably all of the accounts have the same first and last name. In that case things could be simplified a bit. Does that help or just add to the confusion? Thanks, Loren. It helps, I think - but I'm pretty new to using custom rules, so my understanding may be wrong. Since I use only one email address, I should probably set this up in local.cf like this: # # Ok, catch 'from me' when it isn't header __FROM_THOMAS_1 From =~ //i header __FROM_THOMAS_2 From =~ /\"Thomas Cameron\" / header __FROM_THOMAS_3 From =~ /Thomas Cameron / meta NOT_FROM_THOMAS __FROM_THOMAS_1 && !(__FROM_THOMAS_2 || __FROM_THOMAS_3) score NOT_FROM_THOMAS 10 describe NOT_FROM_THOMAS Spammer faking the mail from me! # End of custom rule for Thomas Then, for my wife and kids, the same thing but with their email addresses and domains. Am I correct? Sorry if I'm being dense. I'm just a sysadmin, not a developer, so I'm not super clear on how macros and expansions work in perl. -- Thomas
Re: Sudden surge in spam appearing to come from my email address
On 7/16/23 00:29, Grant Taylor via users wrote: Does that help clarify (my opinion)? It does clarify, but unfortunately, it doesn't alleviate my concerns. I totally understand why SPF et al. are good ideas. But I swear, I feel like they introduce darned near as many problems as they "solve." But that's another rant. Thanks for your explanations. -- Thomas
Re: Sudden surge in spam appearing to come from my email address
On 7/14/23 23:59, Loren Wilton wrote: I am suddenly getting hammered by a BUNCH of spam that appears to be from me. It scores low, and even though I keep feeding it to Bayes, it's still not hitting the threshold to be marked as spam. When I check the headers, it's coming from multiple random email servers, but many appear to originate from hotmail/outlook.com. So from outlook.com, through some unsecured email server, then to my server. SA can't block this trash by itself, but if something post the SA invocation can look at the headers you might be able to block it. You can certainly mark it as spam. For instance: # # Ok, catch 'from me' when it isn't header __FROM_ME_1 From =~ //i header __FROM_ME_2 From =~ /\"First Last\" / header __FROM_ME_3 From =~ /First Last / meta NOT_FROM_ME __FROM_ME_1 && !(__FROM_ME_2 || __FROM_ME_3) score NOT_FROM_ME 10 describe NOT_FROM_ME Spammer faking the mail from me! Mind the backslash on the quotes and at sign. Depending on versions of things these are necessary, and don't hurt if they are not necessary. Forgive my ignorance, I haven't really played with custom rules before. Are the entries like //i meant to edited for my actual email address and domain, or does "me" and "@myhost" get expanded somehow? I actually use sendmail for bunch of domains on my mail servers, and I want to make sure this will work for all those domains. I assume this just needs to go in /etc/mail/spamassassin/local.cf, right? Or do I need to do separate stanzas for each domain? Thomas
Re: Sudden surge in spam appearing to come from my email address
On 7/14/23 20:30, Grant Taylor via users wrote: On 7/14/23 6:06 PM, Thomas Cameron wrote: I'm trying to figure out how to block this stuff. Something like "if it appears to come from me, but it's not actually coming from my email server," block it. SPF with hard fail in your own domain /and/ filtering that respects SPF hard fail will almost certainly stop this like a switch. I'd love to do this, but see below. I get TONS of warnings every time I send email to lists (even this list) that make me hesitant to do hard fails. On 7/14/23 7:28 PM, Thomas Cameron wrote: But because I use several mailing lists, I do not have a hard fail set up. I get SO many notices when I send email to lists that I'm really worried about defining hard failures/rejections. I consider that to be a failure on the mailing list's part. Mailing lists can't successfully operate like they did 25+ years ago. I do, as well, but mailing lists outside of my sphere of influence. I can't very well dictate to mailing list admins that they change the way they do things. Even the earlier email I sent to this list generated a bunch of warning messages. One of many: nimitz.pl postmas...@nimitz.pl camerontech.com-1689379200-1689465...@nimitz.pl 1689379200 1689465599 camerontech.com r r quarantine quarantine 100 95.216.194.37 1 none pass fail camerontech.com spamassassin.apache.org pass camerontech.com pass So it seems like my emails are being quarantined when I send them to mailing lists, even this one. But I'll play around with what you suggested. +10 for SPF. +1 for encouraging mailing list operators to get with the times. You can also do as Robert suggests and use a separate (sub)domain for mailing lists with different SPF settings thereon. It's not so much mailing list operators I'm worried about. It's that, when my email goes through a listserv mailing list, if I define hard failures, I am worried that my email isn't going to get to list members. That's not the mailing list admin, it's the admins of the list members' mail servers. If I'm not understanding something, please feel free to clarify. Thomas
Re: Sudden surge in spam appearing to come from my email address
This kinda raises an important issue. I already have SPF/DMARC/DKIM set up. But because I use several mailing lists, I do not have a hard fail set up. I get SO many notices when I send email to lists that I'm really worried about defining hard failures/rejections. But I'll play around with what you suggested. Thomas On 7/14/23 18:58, David B Funk wrote: Assuming you own/manage your infrastructure it should be straight-forward. Create SFP records for your domain & SMTP server, set them to either soft or hard fail mode. If you can, also set up DKIM signing of your outgoing mail. Then create rules that looks for your from address in a message and a meta which says "if from me & DKIM-fail/SPF-fail hit it hard" If you can work with the SPF hard fail you will also help to improve your net reputation as spammers will have a harder time trying to "Joe Job" you. On Fri, 14 Jul 2023, Thomas Cameron wrote: All - I am suddenly getting hammered by a BUNCH of spam that appears to be from me. It scores low, and even though I keep feeding it to Bayes, it's still not hitting the threshold to be marked as spam. When I check the headers, it's coming from multiple random email servers, but many appear to originate from hotmail/outlook.com. So from outlook.com, through some unsecured email server, then to my server. I'm trying to figure out how to block this stuff. Something like "if it appears to come from me, but it's not actually coming from my email server," block it. I don't necessarily think this is a job for SA, but if there's a rule I can tweak or a setting I can change, I'm all ears. Thanks, Thomas
Sudden surge in spam appearing to come from my email address
All - I am suddenly getting hammered by a BUNCH of spam that appears to be from me. It scores low, and even though I keep feeding it to Bayes, it's still not hitting the threshold to be marked as spam. When I check the headers, it's coming from multiple random email servers, but many appear to originate from hotmail/outlook.com. So from outlook.com, through some unsecured email server, then to my server. I'm trying to figure out how to block this stuff. Something like "if it appears to come from me, but it's not actually coming from my email server," block it. I don't necessarily think this is a job for SA, but if there's a rule I can tweak or a setting I can change, I'm all ears. Thanks, Thomas
Re: 0 score not voiding rule
On 5/27/23 17:21, Noel Butler wrote: apparently does not disable the rule (like 0 disables all the others), is that a way of forcing your world view upon the rest of the world Kevin? > I thought this welcome crap wasnt being applied until next release... I guess Kevin that changed quickly, I might have missed the change as I admit to having little time for most lists these days, family life too hectic :) Pretty bold to be a jerk to a guy you're asking for help from. Be nice, Noel. It's not that hard. I don't know why you've got a burr under your saddle, but it's definitely not making a good impression to be shitty on a public mailing list while you're asking for help. -- Thomas
Re: Rule syntax in local.cf?
On 5/6/22 11:31, Bill Cole wrote: On 2022-05-06 at 10:58:15 UTC-0400 (Fri, 6 May 2022 09:58:15 -0500) Thomas Cameron is rumored to have said: Howdy, all - As I mentioned in a previous email, I'm trying to bump up the score for BAYES_999. I have not messed with SA in years, but I'm trying to get back into it. Sorry if this is a silly question. I tried to add the following line to /etc/mail/spamassassin/local.cf, but it's not firing: [root@mail-east ~]# cat /etc/mail/spamassassin/local.cf # These values can be overridden by editing ~/.spamassassin/user_prefs.cf # (see spamassassin(1) for details) # These should be safe assumptions and allow for simple visual sifting # without risking lost emails. score SPAM_999 3 Where are you getting that rule name??? If I'm reading it correctly, it is NOT bumping up the score for BAYES_999, it's only adding the default 0.2 to it. SA is not clairvoyant or telepathic. It has no idea that you want to change the score on BAYES_999 by using the name of a non-existent rule SPAM_999. I'm running this on Red Hat Enterprise Linux 8.5. The SA package is spamassassin-3.4.4-4.el8.x86_64. What am I doing wrong? Changing the score for a non-existent rule. Ugh. I have no idea how I got it in my head that it was SPAM and not BAYES. Sorry for the noise. Thomas
Rule syntax in local.cf?
Howdy, all - As I mentioned in a previous email, I'm trying to bump up the score for BAYES_999. I have not messed with SA in years, but I'm trying to get back into it. Sorry if this is a silly question. I tried to add the following line to /etc/mail/spamassassin/local.cf, but it's not firing: [root@mail-east ~]# cat /etc/mail/spamassassin/local.cf # These values can be overridden by editing ~/.spamassassin/user_prefs.cf # (see spamassassin(1) for details) # These should be safe assumptions and allow for simple visual sifting # without risking lost emails. score SPAM_999 3 required_hits 5 report_safe 0 rewrite_header Subject [SPAM _SCORE_] What I am seeing when I run spamassassin -D < mail/INBOX/spam looks like this: From powerpl...@sqribblemoney.cam Fri May 6 14:28:32 2022 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mail.redacted.foo X-Spam-Flag: YES X-Spam-Level: * X-Spam-Status: Yes, score=9.3 required=5.0 tests=BAYES_99,BAYES_999, HTML_IMAGE_ONLY_20,HTML_MESSAGE,HTML_SHORT_LINK_IMG_3,KHOP_HELO_FCRDNS, MAY_BE_FORGED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE, URIBL_ABUSE_SURBL,URIBL_BLACK autolearn=disabled version=3.4.4 X-Spam-Report: * 1.2 URIBL_ABUSE_SURBL Contains an URL listed in the ABUSE SURBL * blocklist * [URIs: sqribblemoney.cam] * 3.5 BAYES_99 BODY: Bayes May 6 14:46:39.902 [8259] dbg: check: tagrun - tag DKIMDOMAIN is still blocking action 1 May 6 14:46:39.905 [8259] dbg: plugin: Mail::SpamAssassin::Plugin::MIMEHeader=HASH(0x5567f8e09e90) implements 'finish_tests', priority 0 May 6 14:46:39.905 [8259] dbg: plugin: Mail::SpamAssassin::Plugin::Check=HASH(0x5567f8e0a430) implements 'finish_tests', priority 0 May 6 14:46:39.922 [8259] dbg: netset: cache trusted_networks hits/attempts: 11/12, 91.7 % spam probability is 99 to 100% * [score: 1.] * 0.2 BAYES_999 BODY: Bayes spam probability is 99.9 to 100% * [score: 1.] * 1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist * [URIs: sqribblemoney.cam] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * 0.0 SPF_NONE SPF: sender does not publish an SPF Record * 1.5 HTML_IMAGE_ONLY_20 BODY: HTML: images with 1600-2000 bytes of * words * 0.0 HTML_MESSAGE BODY: HTML included in message * -0.0 T_SCC_BODY_TEXT_LINE No description available. * 0.1 HTML_SHORT_LINK_IMG_3 HTML is very short with a linked image * 1.0 MAY_BE_FORGED Relay IP's reverse DNS does not resolve to IP * 0.0 KHOP_HELO_FCRDNS Relay HELO differs from its IP's reverse DNS If I'm reading it correctly, it is NOT bumping up the score for BAYES_999, it's only adding the default 0.2 to it. I'm running this on Red Hat Enterprise Linux 8.5. The SA package is spamassassin-3.4.4-4.el8.x86_64. What am I doing wrong? Thomas
Re: Why shouldn't I set the score for SPAM_99 and SPAM_999 higher?
On 5/5/22 14:28, Dave Wreski wrote: No, that's how you train your corpora. If you manually look through the headers of mail that's already been processed by your mail system, the ham should be as close to BAYES_00 as possible, and spam should be at BAYES_99. If that's not the case, then it's been trained incorrectly. /etc/mail/spamassassin/local.cf: bayes_auto_learn 0 bayes_auto_expire 0 I'd also recommend disabling auto-learn, if you have that enabled. If you've gone through your corpus manually, and are certain the ham is all good mail and the spam emails are all bad mail, then it might be worth it to dump the existing bayes database and just retrain it with the corresponding mboxes. I also typically add --progress to sa-learn. Best, Dave Thanks, I appreciate it. I'll tune it a bit. Thomas
Re: Why shouldn't I set the score for SPAM_99 and SPAM_999 higher?
On 5/5/22 11:59, Dave Wreski wrote: You should probably check that none of your ham (i.e. non-spam) messages contains SPAM_99 or SPAM_999. It can happen when spammers poison your bayes database, and increased score in that case might lead to legitimate mail being misclassified as a spam. That's a great call, thanks. I grepped my mail files and didn't find any SPAM_99 headers in any of them. You should be looking for BAYES_99 and BAYES_999 in your corpus. Thanks, Dave. I use my various mailboxes (sa-learn --ham --mbox /home/thomas.cameron/mail/INBOX/[mailbox file] and then sa-learn --spam --mbox /home/thomas.cameron/mail/INBOX/spam) to train SA, doesn't that mean that I've already checked my corpora? Thomas
Re: Why shouldn't I set the score for SPAM_99 and SPAM_999 higher?
On 5/5/22 11:47, Matija Nalis wrote: On Thu, May 05, 2022 at 10:37:40AM -0500, Thomas Cameron wrote: I understand that turning knobs without understanding the consequences can do bad thing, but almost all of the spam that gets through SA on my server has SPAM_99 or SPAM_999 set in the headers. It is obviously spam, so I don't really get how it wasn't flagged, but it wasn't. What are the risks of giving more weight to SPAM_99 and/or SPAM_999? Explain it like I'm five, sorry, it's probably something simple that I just don't understand. Thomas You should probably check that none of your ham (i.e. non-spam) messages contains SPAM_99 or SPAM_999. It can happen when spammers poison your bayes database, and increased score in that case might lead to legitimate mail being misclassified as a spam. That's a great call, thanks. I grepped my mail files and didn't find any SPAM_99 headers in any of them. Thomas
Re: Why shouldn't I set the score for SPAM_99 and SPAM_999 higher?
On 5/5/22 10:46, Reindl Harald wrote: Am 05.05.22 um 17:37 schrieb Thomas Cameron: I understand that turning knobs without understanding the consequences can do bad thing, but almost all of the spam that gets through SA on my server has SPAM_99 or SPAM_999 set in the headers. It is obviously spam, so I don't really get how it wasn't flagged, but it wasn't. What are the risks of giving more weight to SPAM_99 and/or SPAM_999? Explain it like I'm five, sorry, it's probably something simple that I just don't understand when your bayes is well trained just raise it the risk is simple: when you bayes isn't trained well or poisend (autolearning is the root of all evil) you risk FPs we milter-reject at 8.0 points and BAYES_99 + BAYES_999 are 7.5 points since 2014, the most junk collects the remaining 0.5 points with other rules and the few FP typically hit some DNSWL/SPF rules with negative score well, our bayes has 160k messages Many thanks! I appreciate the response! Thomas
Why shouldn't I set the score for SPAM_99 and SPAM_999 higher?
I understand that turning knobs without understanding the consequences can do bad thing, but almost all of the spam that gets through SA on my server has SPAM_99 or SPAM_999 set in the headers. It is obviously spam, so I don't really get how it wasn't flagged, but it wasn't. What are the risks of giving more weight to SPAM_99 and/or SPAM_999? Explain it like I'm five, sorry, it's probably something simple that I just don't understand. Thomas
Re: IMPORTANT NOTICE FOR PEOPLE RUNNING TRUNK re: [Bug 7826] Improve language around whitelist/blacklist and master/slave
On 7/14/20 5:55 AM, jdow wrote:
I gotta ask here, "Can't we all skip the ad hominem insults and stick
to technical merits and goals involved in this change?" Please.
{o.o}
LOL - coming from the woman who has been outright insulting,
condescending, and dismissive both on- and off-list, this is a
*hysterical* request. Pot, meet kettle.
Re: IMPORTANT NOTICE FOR PEOPLE RUNNING TRUNK re: [Bug 7826] Improve language around whitelist/blacklist and master/slave
On 7/10/20 12:07 PM, Eric Broch wrote: Amen! This is not about racism this is about a Marxist (Socialist) takeover. They don't care if you use the terms whitelist or blacklist, this is a revolution. Soon, it will be as in Dr. Zhivago. You'll come home being dispossessed of your house and belongings under the supervision of the state, already going on as BLM freely loots and pillages. The "Useful Idiots" (not trying to be offensive, Kevin, but get a grip) don't know that after the reorganization is done, their heads will be on the chopping block as well...all planned in advance. These are sad days, woe is me if I don't speak out. Man, your tinfoil hat is on WAY too tight. Inclusivity is not Marxism, Eric. It's being a decent human being. You should try it some time. Thomas
Re: IMPORTANT NOTICE FOR PEOPLE RUNNING TRUNK re: [Bug 7826] Improve language around whitelist/blacklist and master/slave
On 7/10/20 4:33 AM, jdow wrote:
Are we now going to be afraid of the unwelcome rather than the dark?
Are we going to shine a welcome on problems rather than light?
You guys are MAKING problems where they do not exist. Shame on you,
children.
{^_^}
Nah, you're clinging to old, exclusionary language and behavior when
being inclusive is so damned easy.
Shame on you, old-timer. Be better than this.
Thomas
Re: Thanks to Guardian Digital & LinuxSecurity for the nice post about SpamAssassin's upcoming change
On 7/15/20 9:12 PM, Eric Broch wrote: So, This is the heading of the article: Apache SpamAssassin Leads A Growing List of Open-Source Projects Taking Steps to Correct Instances of Racism and White Privilege Using the word "blacklist" is racism. Does everyone get this! By definition you ARE a "RACIST" and ARE "White Privilege[d]." This is a political movement to blacklist (oohhh, I said it) anyone who does not comply. We're no longer angry, we're "not excited," how generous. The spamassassin leadership team are political hacks. Don't let the door hit you on the way out, then. Thomas
Re: Tuning recommendations?
On 09/12/2016 02:32 PM, John Hardin wrote:
> On Mon, 12 Sep 2016, thomas cameron wrote:
>
>> On 09/12/2016 01:06 PM, John Hardin wrote:
>>> On Mon, 12 Sep 2016, thomas cameron wrote:
>>>
>>>
>>> Make sure you have a local recursing (**NOT** forwarding) DNS server
>>> that your MTA and SA are configured to use. Reason: if you're forwarding
>>> your MTA DNS requests to your ISP's DNS server, the aggregated traffic
>>> of you plus all the other ISP clients can exceed the various DNSBL and
>>> URIBL free-usage limits, rendering those tools useless.
>>
>> [root@mail-west ~]# grep recurs /etc/named.conf
>> allow-recursion { 127.0.0.1; };
>>
>>> A clear indicator this is happening: URIBL_BLOCKED hits.
>>
>> I see "URIBL_BLACK Contains an URL listed in the URIBL blacklist" in the
>> headers of many of the messages that got through. Is that what you mean?
>
> No. URIBL_BLACK indicates your URIBL queries are succeeding, that's a
> hit. URIBL_BLOCKED means "request blocked", probably due to exceeding
> the limits.
OK, thanks.
>>> Train up your Bayes using hand-vetted spam *and* ham, at least 200 of
>>> each. Using autolearn initially can be problematic, so disable that
>>> until SA is doing a fairly good job using hand-trained Bayes. Then you
>>> can let autolearn keep it up-to-date if you like, and continue to
>>> capture and manually train any persistent misses or near-misses.
>>> Generally the more you feed Bayes the better it performs, but it must be
>>> accurately classified. If you feeed garbage to Bayes, you'll get garbage
>>> results.
>>
>> Good to know, thanks. I am running sa-learn --ham --mbox $MAIL now. I've
>> been running sa-learn --spam against the spam messages I've moved to my
>> spam folder, but forgot to teach it about ham.
>
> It's a really bad idea to train your inbox as ham. There may be stuff
> (specifically, FNs) in there you haven't seen yet or haven't removed.
> Keep a separate train-as-ham folder that you manually populate after
> actually looking at the messages, just like you're keeping a
> train-as-spam folder.
>
> You might want to wipe and retrain from scratch after setting that up,
> especially if you're seeing low BAYES score hits on spams and FPs.
I can certainly do that.
> Are you seeing any BAYES rule hits at all yet?
Yes, including a fair number of BAYES_999 and BAYES_99, which I would
have thought would have more weight than it apparently does. I know I
can custom score in local.cf, but I've always read that I should avoid
changing default scores unless I *really* know what I'm doing. Clearly,
I'm not there yet.
>>> Keep hand-classified Bayes corpora around in case you ever need to wipe
>>> and retrain from scratch.
>>
>> OK.
>>
>>> Ensure you're training Bayes as the user that SA is running under.
>>> Training the wrong Bayes database is a common cause of problems.
>>
>> It's a small server, so I'm doing this via procmail and spamc.
>> Everything runs in the context of the individual users. I need to run
>> sa-learn --ham as each user against their inboxes, I guess. I can add
>> cron jobs for each user to do that.
>
> You might also consider running a shared/global Bayes, if all your
> users' mail streams are fairly similar w/r/t "what is ham?" There should
> be instructions in the SA wiki for setting up shared/global Bayes.
I used to run SA via spamass-milter, and use a single Bayes DB under
user spam, but when I downsized my server, the hassle of feeding that
shared DB became bigger than the benefit. I will revisit that conclusion.
>>> Consider doing some MTA-level DNSBL checks. The Zen DNSBL is
>>> well-regarded. If you're using Postfix then there are some emails from
>>> Reindl Harald on this list regarding weighted DNSBL scoring that you may
>>> find useful. You'll have to search the archives to find those.
>>
>> I'm using sendmail, and I have these checks on:
>>
>> FEATURE(`dnsbl',`in.dnsbl.org ')dnl
>> FEATURE(`dnsbl',`sbl-xbl.spamhaus.org')dnl
>> FEATURE(`dnsbl',`cbl.abuseat.org')dnl
>>
>> I will add FEATURE(`dnsbl',`zen.spamhaus.org')dnl to it.
>
> Zen incorporates a couple of the ones you're already using, don't double
> up.
OK, good to know.
>>> There are some other MTA-level checks you can perform, like greet pause
>>> and HELO validation (e.g. reject if the HELO has no dots).
>>
>> Like this? http://www.harker.com/sendmail/checkhelo.html
>
> Here's greet pause:
>
> FEATURE(`greet_pause',3000)dnl
This is very helpf
Re: Tuning recommendations?
On 09/12/2016 01:40 PM, li...@rhsoft.net wrote:
>
>
> Am 12.09.2016 um 20:34 schrieb thomas cameron:
>> On 09/12/2016 01:06 PM, John Hardin wrote:
>>> On Mon, 12 Sep 2016, thomas cameron wrote:
>>>
>>> Make sure you have a local recursing (**NOT** forwarding) DNS server
>>> that your MTA and SA are configured to use. Reason: if you're forwarding
>>> your MTA DNS requests to your ISP's DNS server, the aggregated traffic
>>> of you plus all the other ISP clients can exceed the various DNSBL and
>>> URIBL free-usage limits, rendering those tools useless.
>>
>> [root@mail-west ~]# grep recurs /etc/named.conf
>> allow-recursion { 127.0.0.1; };
>>
>>> A clear
>>> indicator this is happening: URIBL_BLOCKED hits.
>>
>> I see "URIBL_BLACK Contains an URL listed in the URIBL blacklist" in the
>> headers of many of the messages that got through. Is that what you mean?
>
> no that means the message had a hit and so it seems your are using only
> 127.0.0.1 as nameserver and that nameserver does *not* forwarding
Ah, OK. I actually just changed my resolv.conf to do DNS lookups from
127.0.0.1. Before, it was using public DNS servers.
> it would be really helpful if you just post the full report-header of
> such a message, otherwise you are at your own
Sure, I didn't want to bomb the list with crud, sorry. Here's the header
of the latest spam to slip through.
Return-Path: <paula.fie...@westbegalssc.com>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
mail-west.camerontech.com
X-Spam-Level: ***
X-Spam-Status: No, score=4.0 required=5.0 tests=BAYES_99,DIET_1,
HTML_FONT_LOW_CONTRAST,HTML_IMAGE_RATIO_02,HTML_MESSAGE,SPF_PASS,
T_HTML_TAG_BALANCE_CENTER,T_REMOTE_IMAGE autolearn=no autolearn_force=no
version=3.4.0
Received: from substantiate.westbegalssc.com
(91-239-125-145.thinkdedicated.com [91.239.125.145] (may be forged))
by mail-west.camerontech.com (8.14.7/8.14.7) with ESMTP id
u8CIX42I002741
for <thomas.came...@camerontech.com>; Mon, 12 Sep 2016 18:33:29 GMT
To: <thomas.came...@camerontech.com>
Date: Mon, 12 Sep 2016 13:33:01 -0500
From: "Paula Fields" <paula.fie...@westbegalssc.com>
Reply-To: <ps.fie...@westbegalssc.com>
Message-ID: <VA1C.G5KTp'jmi5891...@substantiate.westbegalssc.com>
Subject: Great Science: Shred lbs while you sit in your cublicle.
Mime-Version: 1
Content-Type: multipart/alternative;
boundary="897htfNCA6B4duDQ193OAjlzRH078d7wF"
X-Greylist: Sender passed SPF test, not delayed by
milter-greylist-4.5.16 (mail-west.camerontech.com [104.131.155.84]);
Mon, 12 Sep 2016 18:33:36 + (UTC)
Re: Tuning recommendations?
On 09/12/2016 01:06 PM, John Hardin wrote:
> On Mon, 12 Sep 2016, thomas cameron wrote:
>
>
> Make sure you have a local recursing (**NOT** forwarding) DNS server
> that your MTA and SA are configured to use. Reason: if you're forwarding
> your MTA DNS requests to your ISP's DNS server, the aggregated traffic
> of you plus all the other ISP clients can exceed the various DNSBL and
> URIBL free-usage limits, rendering those tools useless.
[root@mail-west ~]# grep recurs /etc/named.conf
allow-recursion { 127.0.0.1; };
> A clear
> indicator this is happening: URIBL_BLOCKED hits.
I see "URIBL_BLACK Contains an URL listed in the URIBL blacklist" in the
headers of many of the messages that got through. Is that what you mean?
> Train up your Bayes using hand-vetted spam *and* ham, at least 200 of
> each. Using autolearn initially can be problematic, so disable that
> until SA is doing a fairly good job using hand-trained Bayes. Then you
> can let autolearn keep it up-to-date if you like, and continue to
> capture and manually train any persistent misses or near-misses.
> Generally the more you feed Bayes the better it performs, but it must be
> accurately classified. If you feeed garbage to Bayes, you'll get garbage
> results.
Good to know, thanks. I am running sa-learn --ham --mbox $MAIL now. I've
been running sa-learn --spam against the spam messages I've moved to my
spam folder, but forgot to teach it about ham.
> Keep hand-classified Bayes corpora around in case you ever need to wipe
> and retrain from scratch.
OK.
> Ensure you're training Bayes as the user that SA is running under.
> Training the wrong Bayes database is a common cause of problems.
It's a small server, so I'm doing this via procmail and spamc.
Everything runs in the context of the individual users. I need to run
sa-learn --ham as each user against their inboxes, I guess. I can add
cron jobs for each user to do that.
> Consider doing some MTA-level DNSBL checks. The Zen DNSBL is
> well-regarded. If you're using Postfix then there are some emails from
> Reindl Harald on this list regarding weighted DNSBL scoring that you may
> find useful. You'll have to search the archives to find those.
I'm using sendmail, and I have these checks on:
FEATURE(`dnsbl',`in.dnsbl.org ')dnl
FEATURE(`dnsbl',`sbl-xbl.spamhaus.org')dnl
FEATURE(`dnsbl',`cbl.abuseat.org')dnl
I will add FEATURE(`dnsbl',`zen.spamhaus.org')dnl to it.
> There are some other MTA-level checks you can perform, like greet pause
> and HELO validation (e.g. reject if the HELO has no dots).
Like this? http://www.harker.com/sendmail/checkhelo.html
> Consider greylisting.
I am using milter-greylist, and it is very helpful. A lot of these
messages are actually skipping greylisting, though!
X-Greylist: Sender passed SPF test, not delayed by
milter-greylist-4.5.16 (XXX [XXX.XXX.XXX.XXX]); Mon, 12 Sep 2016
18:11:18 + (UTC)
Keep the tips coming, I appreciate learning from you!
Thomas
Re: Tuning recommendations?
On 09/12/2016 11:50 AM, Jesse Norell wrote: > On Mon, 2016-09-12 at 11:40 -0500, thomas cameron wrote: >> Any other tips welcome! > > You didn't mention any details of your setup, but some very basic tips > are to run a current version of spamassassin and run sa-update > regularly. You might verify (or confirm) the user you train bayes with > is the same user that the scanner runs as. > Sorry, forgot to mention that I'm running Sendmail and not postfix. It's a small server, only serving a couple of users, so I am just using ~/.procmailrc: [root@mail-west ~]# cat /home/thomas.cameron/.procmailrc MAILDIR=$HOME/mail LOGFILE=$MAILDIR/procmail.log :0fw: spamassassin.lock * < 1024000 | spamc :0 * ^X-Spam-Flag:.*YES spam
Re: Tuning recommendations?
On 09/12/2016 11:50 AM, Jesse Norell wrote: > On Mon, 2016-09-12 at 11:40 -0500, thomas cameron wrote: >> Any other tips welcome! > > You didn't mention any details of your setup, but some very basic tips > are to run a current version of spamassassin and run sa-update > regularly. You might verify (or confirm) the user you train bayes with > is the same user that the scanner runs as. Fair point, sorry. I'm running RHEL 7, using spamassassin-3.4.0-2.el7.x86_64 The only real changes I've made are in local.cf: [root@mail-west ~]# cat /etc/mail/spamassassin/local.cf # These values can be overridden by editing ~/.spamassassin/user_prefs.cf # (see spamassassin(1) for details) # These should be safe assumptions and allow for simple visual sifting # without risking lost emails. required_hits 5 report_safe 0 rewrite_header Subject ***SPAM(_SCORE_)*** use_bayes 1 bayes_auto_learn 1 bayes_ignore_header X-Bogosity bayes_ignore_header X-Spam-Flag bayes_ignore_header X-Spam-Status # per http://www.spamtips.org/2011/02/smfbracketsto-rule.html header SMF_BRACKETS_TO To:raw =~ /<<[^<>]+>>/ describe SMF_BRACKETS_TO Double-brackets around To header address score SMF_BRACKETS_TO 1.5 # per http://www.spamtips.org/2011/01/disable-dnsfromahblrhsbl.html score DNS_FROM_AHBL_RHSBL 0 # per http://www.spamtips.org/2011/01/disable-rfc-ignorantorg-rules.html # Add these lines to your local.cf then restart your spamd score __RFC_IGNORANT_ENVFROM0 score DNS_FROM_RFC_DSN 0 score DNS_FROM_RFC_BOGUSMX 0 score __DNS_FROM_RFC_POST 0 score __DNS_FROM_RFC_ABUSE 0 score __DNS_FROM_RFC_WHOIS 0 Other than that, it's bone stock.
Re: Tuning recommendations?
On 09/12/2016 10:53 AM, li...@rhsoft.net wrote: > > > Am 12.09.2016 um 17:51 schrieb thomas cameron: >> I rolled a new mail server out for my small business, and I've got a >> pretty vanilla SA setup. It's just not doing a very good job of catching >> spam. I'm getting a TON of "Amazon gift card" and "female hair loss" and >> "work from home" spam in my inbox. I feel like if I see one more e-mail >> about Blake Shelton, I'm gonna scream > > train your bayes proper with enough ham *and* spam and do it with the > user spamassassin runs Yeah, I have a cron job that does that. 0 2 * * * sa-learn --mbox --spam $HOME/mail/spam 1 2 * * * sa-learn --mbox --spam $HOME/mail/super-spam 2 2 * * * sa-learn --mbox --ham $HOME/mail/ham I hesitate to run --ham against my inbox because so much spam is getting through, so I copy a bunch of stuff over to the ham folder and train from there. Any other tips welcome! Thomas
Tuning recommendations?
Howdy, all - I rolled a new mail server out for my small business, and I've got a pretty vanilla SA setup. It's just not doing a very good job of catching spam. I'm getting a TON of "Amazon gift card" and "female hair loss" and "work from home" spam in my inbox. I feel like if I see one more e-mail about Blake Shelton, I'm gonna scream. Is there a good tuning/config page anywhere? Last time I messed with SA, I used www.spamtips.org. It's pretty old, though, so I imagine there are better ways. I also used to use rules du jour, but I read that that's old and not maintained any more. What do you guys recommend for tuning? It's been so long since I really dove deep into SA, you can just assume I'm starting from scratch. Many thanks! Thomas
Re: Anyone else just blocking the ".top" TLD?
On 03/28/2016 05:23 AM, Reindl Harald wrote: > > > Am 28.03.2016 um 05:24 schrieb Bill Cole: >> On 27 Mar 2016, at 21:58, Thomas Cameron wrote: >> >>> Has anyone actually gotten a single legit message from that domain? >> >> IMHO we're close to the point where it will make sense to make email >> default-deny and to build standard protocols for senders to be returned >> to the traditional trust model on a domainwise basis for each receiving >> system or domain. The authentication methods already exist, there just >> isn't enough adoption (for some good reasons) and we don't have usable >> authorization models > > what we do is: > > * reject every non-existent tld > * download http://data.iana.org/TLD/tlds-alpha-by-domain.txt daily > * if new domains arrived allow them as sender/helo in theory > * BUT blacklist_tld.cf comes after the spf-policyd > * old gTLD and ccTLD are excluded here > * some speical friends like .top and *.xyz* are in a own sender-access > and even in a unconditional helo-reject > > Weitergeleitete Nachricht > Betreff: Cron <root@mail-gw> update-spamfilter.sh > Datum: Sat, 26 Mar 2016 02:40:03 +0100 (CET) > Von: (Cron Daemon) > UPDATED: /etc/postfix/blacklist_generic_ptr.cf > 1145a1146 >> /.*\.ally$/ DUNNO > 1189a1191 >> /.*\.barefoot$/ DUNNO > - > UPDATED: /etc/postfix/blacklist_helo.cf > 44a45 >> /.*\.ally$/ DUNNO > 88a90 >> /.*\.barefoot$/ DUNNO > - > UPDATED: /etc/postfix/blacklist_tld.cf > 22a23 >> /.*\.ally$/ REJECT Spam-TLD (SPF Required: .ally - see > http://en.wikipedia.org/wiki/Sender_Policy_Framework) > 51a53 >> /.*\.barefoot$/ REJECT Spam-TLD (SPF Required: .barefoot - see > http://en.wikipedia.org/wiki/Sender_Policy_Framework) > - > > OK: /usr/bin/systemctl reload postfix.service > Wow! I almost didn't post this, I figured I'd get yelled at for such a heavy-handed approach. Thanks for letting me know I'm not completely nuts. Well, at least not as regards to this particular subject! :-) Thomas signature.asc Description: OpenPGP digital signature
Anyone else just blocking the ".top" TLD?
Has anyone actually gotten a single legit message from that domain? Thomas
Somewhat OT - how do I whitelist a host which is in a DNSBL in sendmail?
Howdy - I have two VMs at Digital Ocean, one on the east coast, one on the west. I'm running Sendmail-8.14.8-2.fc20.x86_64. I have several DNSBLs listed: FEATURE(`dnsbl',`in.dnsbl.org ')dnl FEATURE(`dnsbl',`sbl-xbl.spamhaus.org')dnl FEATURE(`dnsbl',`cbl.abuseat.org')dnl FEATURE(`dnsbl',`dul.dnsbl.sorbs.net')dnl Unfortunately, my home network is attached to a cable provider which shows up in dul.dnsbl.sorbs.net. Can I whitelist my IP address so that I can send mail through my mail servers? Right now, it gets rejected. Yeah, I know, I can always use my ISP's smtp server, I guess. But that kind of sucks. I would rather use mine. Purely a pride thing, I know. Thomas
Re: Somewhat OT - how do I whitelist a host which is in a DNSBL in sendmail?
On 07/24/2014 09:58 AM, Thomas Cameron wrote: Howdy - I have two VMs at Digital Ocean, one on the east coast, one on the west. I'm running Sendmail-8.14.8-2.fc20.x86_64. I have several DNSBLs listed: FEATURE(`dnsbl',`in.dnsbl.org ')dnl FEATURE(`dnsbl',`sbl-xbl.spamhaus.org')dnl FEATURE(`dnsbl',`cbl.abuseat.org')dnl FEATURE(`dnsbl',`dul.dnsbl.sorbs.net')dnl Unfortunately, my home network is attached to a cable provider which shows up in dul.dnsbl.sorbs.net. Can I whitelist my IP address so that I can send mail through my mail servers? Right now, it gets rejected. Yeah, I know, I can always use my ISP's smtp server, I guess. But that kind of sucks. I would rather use mine. Purely a pride thing, I know. Thomas Disregard. I was way over thinking it. A quick line in /etc/mail/access fixed it. Sorry for the noise. TC
Re: Somewhat OT - how do I whitelist a host which is in a DNSBL in sendmail?
On 07/24/2014 10:37 AM, Dave Funk wrote: Thomas. Do you have 'MSA' port enabled for your sendmail? (IE port 567) and SMTP-AUTH? Then just skip the dnsbl checks for auth'ed mail submissions. You could whitelist your client IP address in your 'access' file but what happens when that address changes? (I assume your ISP gives you a DHCP address). Hi, Dave - I actually have SMTP AUTH enabled, and it was working fine (albeit on port 25 with STARTTLS) until I added the DNSBL. Even connecting from my MUA (Thunderbird on Linux) to port 587 on my server, I get this (identifying info changed) in the log file if I enable the DNSBL: Jul 24 11:57:36 YYY dovecot: imap-login: Login: user=thomas.cameron, method=PLAIN, rip=1.2.3.4, lip=4.5.6.7 mpid=469, TLS, session=GG70g/L+xwBGw8l/ Jul 24 11:57:59 YYY sendmail[472]: ruleset=check_relay, arg1=cpe-.austin.res.rr.com, arg2=127.0.0.10, relay=cpe-.austin.res.rr.com [1.2.3.4], reject=550 5.7.1 Rejected: 68.203.17.142 listed at dul.dnsbl.sorbs.net TC
tips and tricks?
Howdy - Last time I set up SA I used http://www.spamtips.org/p/ultimate-setup-guide.html, but it appears to be somewhat dated (2011). Is it still a good guide? Is there a better simple collection of tips and tricks? Thanks! Thomas
pyzor: check failed: internal error, python traceback seen in response
Howdy, I'm running SA on a RHEL 6.5 machine.
Using spamassassin-3.3.1-3.el6.x86_64, pyzor-0.5.0-3.el6.noarch,
spamass-milter-0.3.2-3.el6.x86_64 and milter-greylist-4.5.7-1.el6.x86_64
(if that matters).
The relevant parts of my sendmail.mc are:
INPUT_MAIL_FILTER(`spamassassin',
`S=unix:/var/run/spamass-milter/spamass-milter.sock, F=,
T=C:15m;S:4m;R:4m;E:10m')dnl
define(`confMILTER_MACROS_CONNECT',`t, b, j, _, {daemon_name},
{if_name},{if_addr}')dnl
INPUT_MAIL_FILTER(`greylist',`S=local:/var/run/milter-greylist/milter-greylist.sock')dnl
define(`confMILTER_MACROS_HELO', `{verify}, {cert_subject}')dnl
define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}')dnl
define(`confMILTER_MACROS_ENVRCPT', `b, r, v, Z, {greylist}')dnl
define(`confINPUT_MAIL_FILTERS', `spamassassin, greylist')
I set spamassassin to run as user spam:
[root@ns2 ~]# cat /etc/sysconfig/spamassassin
# Options to spamd
SPAMDOPTIONS=-u spam -d -c -m5 -H
I also set spamass-milter to run as spam:
[root@ns2 ]# grep RUN_AS_USER /etc/rc.d/init.d/spamass-milter
RUN_AS_USER=spam
...
I am seeing this in /var/log/maillog every time I start up SpamAssassin:
Mar 5 23:26:34 ns2 spamd[9065]: pyzor: check failed: internal error,
python traceback seen in response
I've done pyzor -discover as the spam user, and pyzor ping reports
everything is OK.
What am I doing wrong? Everything is, as far as I can tell, running as
spam.
Why am I getting an error in pyzor when SA starts up? Anyone know?
Thomas
Re: Dev-nulling is a bad idea [Was: Verifying .procmailrc settings to delete high scoring spam messages]
On 04/08/2013 03:52 AM, Andrzej A. Filip wrote: On 04/08/2013 05:12 AM, Thomas Cameron wrote: [...] I want to delete any spam that scores over 10, though. I believe that I should insert a new rule between the first and second, and I want to use the X-Spam-Level header. But since it uses asterisks, which are interpreted as regex wildcards, I want to make sure I've got the right syntax. I think I would need to escape out the asterisks, right? Would it look like this? :0: * ^X-Spam-Level:.*\*\*\*\*\*\*\*\*\*\* /dev/null I believe that would match 10 asterisks or more, and redirect the e-mail to /dev/null. Am I right? I would suggest redirecting such messages to another folder/maildir. The folder should auto-purge old messages (e.g. older than 30 days). Shit does happen. I remember at least one case in which mailing list (ham) thread about spammer scored 10. Such very false positives are very unlikely/rare *but* nobody responsible is going to guarantee it will not happen to you. So, I've set up two IMAP folders, spam for messages which are in the 5-10 range and super-spam which are over 10. I've been watching them since the 7th, when I updated SA and configured it based on Warren Togami's most excellent guide at http://www.spamtips.org/p/ultimate-setup-guide.html. So far the super-spam folder is getting messages at about 10:1 over spam. I have not seen a single FP in super-spam in that time. In fact, I have not seen ANY FPs in either folder. At this point, I'm pretty comfortable just nuking that e-mail instead of wasting space with it. Currently I'm using procmail recipes for individual users, but I'm leaning heavily towards going back to spamass-milter, and rejecting everything that scores 10 or more. I'm definitely open to suggestions, though. The only argument I have seen so far is you might get a FP. While that is absolutely valid, it has not happened so far. If I use spamass-milter, the sender will get a rejection notice, so important senders which trigger FPs will be able to call me and let me know. Otherwise, I don't think the message is that important. ;-) Thoughts? Thomas
Re: Dev-nulling is a bad idea [Was: Verifying .procmailrc settings to delete high scoring spam messages]
On 04/22/2013 09:03 AM, Matus UHLAR - fantomas wrote: On 22.04.13 08:27, Thomas Cameron wrote: Currently I'm using procmail recipes for individual users, but I'm leaning heavily towards going back to spamass-milter, and rejecting everything that scores 10 or more. with thing like spamass-milter I found REFUSING mail (not devnulling!) sa safe. I also use score 10 as rejecting threshold. Yeah, exactly what I had in mind.
Re: Dev-nulling is a bad idea [Was: Verifying .procmailrc settings to delete high scoring spam messages]
On 04/22/2013 09:29 AM, Andrzej A. Filip wrote: False positives in super-spam (10 SA score) should be very rare. Exactly my point. Are you ready/willing to report spam you receive to spamcop.net, razor, pyzor, ...? That's an interesting question... Each user has their own spam folders, so I guess I should create a cron job per user to do so, maybe? Does anyone do that? Is it smart? TC
Verifying .procmailrc settings to delete high scoring spam messages
All - I have a pretty simple .procmailrc setup for my home mail server. Right now it looks like: :0fw: spamassassin.lock * 256000 | spamc :0: * ^X-Spam-Flag:.*YES spam That dumps everything that is flagged as spam into my spam folder. I want to delete any spam that scores over 10, though. I believe that I should insert a new rule between the first and second, and I want to use the X-Spam-Level header. But since it uses asterisks, which are interpreted as regex wildcards, I want to make sure I've got the right syntax. I think I would need to escape out the asterisks, right? Would it look like this? :0: * ^X-Spam-Level:.*\*\*\*\*\*\*\*\*\*\* /dev/null I believe that would match 10 asterisks or more, and redirect the e-mail to /dev/null. Am I right? Thanks! Thomas
Re: Verifying .procmailrc settings to delete high scoring spam messages
On 04/07/2013 10:44 PM, Bob Proulx wrote:
Thomas Cameron wrote:
:0:
* ^X-Spam-Level:.*\*\*\*\*\*\*\*\*\*\*
/dev/null
I believe that would match 10 asterisks or more, and redirect the
e-mail to /dev/null. Am I right?
Mostly all okay. However I don't like the .* in the front of
it. That isn't likely to cause trouble but it is possible that it
could on a crafted email message with a lot of garbage cause trouble.
And it isn't needed. We know there will always be one space there.
So no need for the .* there.
Noted, thank you!
With /dev/null you don't need the trailing : in the :0:
designating a lockfile. I think procmail special cases /dev/null to
avoid the lock file in that case anyway. But just the same I wouldn't
put the trailing colon lockfile for /dev/null.
Thanks, I realized that after I hit send. I think that was a bad
copy-n-paste, it's been taken out.
Also it is safer to store to a mail folder at least long enough to
test your recipe. So just as a general paranoia instead of /dev/null
I would at least start with a mail folder and then only after I have
convinced myself that it is good to go only then convert it to a real
/dev/null. I like maildir folders so will normally use folder/ to
have procmail create a maildir folder format. And maildir folders
never need a lockfile. But use what you like.
:0
* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*
devnull/
Good call, done.
Since procmail uses Extended Regular Expressions there is one more
optimization I would make. I wouldn't list out every star. It gets
hard to count. Is there ten there? Or nine? Or eleven? Quick,
without counting, how many? See that is hard. But you can use the
normal extended regular expression syntax to simply list the number.
:0
* ^X-Spam-Level: \*{10}
devnull/
That makes the counting quick and easy.
That is very cool, thank you for the regex advice!
For me I don't tend to /dev/null things immediately. I tend to always
keep at least a queue of them around so that I can look at them. With
maildir format each message is an individual file. Meaning that it is
easy to delete them by age from the devnull/* directories. I would
keep something like this around for whatever you feel is reasonable.
I would probably say ten days. That way if I need to go looking for a
potentially very spammy message I could still find it within the time
window. I would run this daily from cron.
find $HOME/Mail/devnull -type f -mtime +10 -delete
HTH,
Bob
Great advice, Bob, thank you very much! I've been watching the cruft in
my spam mail folder, and I've never seen anything over 10 that was a
false positive. I'm very confident that 10+ needs to just be nuked, but
I see your point. I'll let it get filtered into a temporary mail folder
for a few days to make sure I'm right, though.
Thank you very much for the excellent advice, I really appreciate it!
TC
Could not retrieve sendmail macro _!. Please add it to confMILTER_MACROS_CONNECT for better spamassassin results
I am getting $SUBJECT on my RHEL6 box running sendmail, spamassassin,
spamass-milter, clamav-milter, and milter-greylist. My sendmail.mc looks
like this:
[root@spamcatcher ~]# grep -v ^dnl /etc/mail/sendmail.mc
divert(-1)dnl
include(`/usr/share/sendmail-cf/m4/cf.m4')dnl
VERSIONID(`setup for linux')dnl
OSTYPE(`linux')dnl
define(`confDEF_USER_ID', ``8:12'')dnl
define(`confTO_CONNECT', `1m')dnl
define(`confTRY_NULL_MX_LIST', `True')dnl
define(`confDONT_PROBE_INTERFACES', `True')dnl
define(`PROCMAIL_MAILER_PATH', `/usr/bin/procmail')dnl
define(`ALIAS_FILE', `/etc/aliases')dnl
define(`STATUS_FILE', `/var/log/mail/statistics')dnl
define(`UUCP_MAILER_MAX', `200')dnl
define(`confUSERDB_SPEC', `/etc/mail/userdb.db')dnl
define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn,restrictqrun')dnl
define(`confAUTH_OPTIONS', `A')dnl
define(`confTO_IDENT', `0')dnl
FEATURE(`no_default_msa', `dnl')dnl
FEATURE(`smrsh', `/usr/sbin/smrsh')dnl
FEATURE(`mailertable', `hash -o /etc/mail/mailertable.db')dnl
FEATURE(`virtusertable', `hash -o /etc/mail/virtusertable.db')dnl
FEATURE(redirect)dnl
FEATURE(always_add_domain)dnl
FEATURE(use_cw_file)dnl
FEATURE(use_ct_file)dnl
FEATURE(local_procmail, `', `procmail -t -Y -a $h -d $u')dnl
FEATURE(`access_db', `hash -TTMPF -o /etc/mail/access.db')dnl
FEATURE(`blacklist_recipients')dnl
EXPOSED_USER(`root')dnl
DAEMON_OPTIONS(`Port=smtp, Name=MTA')dnl
FEATURE(`accept_unresolvable_domains')dnl
LOCAL_DOMAIN(`localhost.localdomain')dnl
MAILER(smtp)dnl
MAILER(procmail)dnl
INPUT_MAIL_FILTER(`spamassassin',
`S=unix:/var/run/spamass-milter/spamass-milter.sock, F=,
T=C:15m;S:4m;R:4m;E:10m')dnl
define(`confMILTER_MACROS_CONNECT',`t, b, j, _, {daemon_name},
{if_name},{if_addr}')dnl
define(`confMILTER_MACROS_ENVRCPT',confMILTER_MACROS_ENVRCPT`, b, r, v,
Z')dnl
INPUT_MAIL_FILTER(`clmilter',`S=local:/var/run/clamav/clamav-milter.sock, F=,
T=S:4m;R:4m')dnl
INPUT_MAIL_FILTER(`greylist',`S=local:/var/milter-greylist/milter-greylist.sock')dnl
define(`confMILTER_MACROS_CONNECT', `j, {if_addr}')dnl
define(`confMILTER_MACROS_HELO', `{verify}, {cert_subject}')dnl
define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}')dnl
define(`confMILTER_MACROS_ENVRCPT', `{greylist}')dnl
define(`confINPUT_MAIL_FILTERS', `spamassassin, clmilter, greylist')
What have I done wrong? I confess I am not an m4 maven, I just copied
recommended settings from various web sites.
TC
Re: SpamAssassin wins 2007 InfoWorld Best of Open Source Software award
Justin Mason wrote: I'm happy to announce that we have won an InfoWorld Best Of Open Source Software BOSSIE Award, as the winner in the anti-spam category for 2007! more info here: http://www.infoworld.com/archives/t.jsp?N=sV=91650 --j. Well deserved, all. Outstanding product, you do not know how much SA has helped me out. TC
dcc HOWTO?
All -
I'm using Sendmail on RHEL 4 with SA and spamass-milter, clamav-milter
and milter-greylist. What is the best way for SpamAssassin to use DCC?
So far I've created an RPM with these configure options:
./configure \
--homedir=/var/dcc \
--bindir=/usr/bin \
--libexecdir=/usr/libexec \
--mandir=/usr/share/man \
--with-sendmail \
--with-cgibin=/var/www/cgi-bin \
--with-rundir=/var/run \
--disable-sys-inst \
--with-installroot=/var/tmp/%{name}-root
Once I created that RPM I set DCCUID=spam in /var/dcc/dcc_conf.
I also set DCCD_ENABLE=off since I am using a remote server.
I set GREY_ENABLE=off since I am using milter-greylist.
I set DCCM_ENABLE=off as I am not using a milter for DCC.
I set DCCIFD_ENABLE=yes as the DCC docs say If you are using
SpamAssassin, then you almost certainly should be using dccifd. Do I
need to do anything besides set use_dcc 1 in local.cf?
I copied /usr/libexec/rcDCC to /etc/rc.d/init.d and chkconfig'd it on.
Missing anything?
Thanks!
Thomas
Re: User getting spammed to death
On Tue, 2006-02-14 at 07:45 +, [EMAIL PROTECTED] wrote: It seems my email appears on one of those millions of emails cdroms Egads, are those things still out there? I used to get spammed with offers for them. Of course, I don't get spam any more (thanks, SA team)! Thomas
Re: hey john spam
On Fri, 2006-01-27 at 17:13 -0800, Kelson wrote: John Fleming wrote: This is a new one for me. Today I've received some mail with hey john in the subject, and the mail otherwise appears blank. It didn't contain a virus, or it would've been discarded by ClamAV. Are these familiar to you guys? What's the point of them? Headers of one below: Thanks! - John I've been seeing a lot of these over the last two days. In each case it's hey LHS-of-address So I've seen a lot of hey kelson and hey webmaster. I thought hey postmaster was funny, but then I saw hey mailer-daemon Most of them have been blank, like the one you saw. What's interesting is that they aren't actually empty -- they're multipart/alternative messages containing both HTML and plaintext parts -- it's just that there's no content in either of them. I did see one that had some text and an attached image, but I didn't pay much attention to it and discarded it after training Bayes reporting to Razor. Nothing really stood out about it, so I don't remember the topic, and I'm not 100% certain it was one of these and not another piece of spam that showed up in the search for Subject: hey My guess is that it's just a broken or misconfigured mailer. It's sending incorrectly, or the spammer forgot to paste in the body of the message, or something. I wonder if perhaps it's just some sort of probe. Maybe they send out a bunch of them and then make a note of the ones which don't bounce. Those are then used for the real spam. Thoughts? TC
Error building 64-bit on FC2
All - I am running Fedora Core 2 on an AMD Athlon(tm) 64 Processor 3000+. It is up to date with the latest packages available. I grabbed the latest SA tarball from a mirror, and ran rpmbuild -ta against it. It fails with this: Manifying blib/man3/Mail::SpamAssassin::DnsResolver.3pm Manifying blib/man3/Mail::SpamAssassin::SubProcBackChannel.3pm Manifying blib/man3/Mail::SpamAssassin::Plugin::RelayCountry.3pm + /usr/bin/make spamc/libspamc.so /usr/bin/make -f spamc/Makefile spamc/libspamc.so make[1]: Entering directory `/home/thomas.cameron/redhat/BUILD/Mail- SpamAssassin-3.1.0' gcc -rdynamic -Wl,-rpath,/usr/lib64/perl5/5.8.3/x86_64-linux-thread- multi/CORE spamc/libspamc.c spamc/utils.c \ -o spamc/libspamc.so -shared -ldl /usr/bin/ld: /tmp/cc2TpgXy.o: relocation R_X86_64_32S can not be used when making a shared object; recompile with -fPIC /tmp/cc2TpgXy.o: could not read symbols: Bad value collect2: ld returned 1 exit status make[1]: *** [spamc/libspamc.so] Error 1 make[1]: Leaving directory `/home/thomas.cameron/redhat/BUILD/Mail- SpamAssassin-3.1.0' make: *** [spamc/libspamc.so] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.97589 (%build) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.97589 (%build) Odd thing is, if I just grab the tarball and run perl Makefile.PL /dev/null make make install DESTDIR=/var/tmp/sa it installs just fine to /var/tmp/sa Thoughts? Thomas
RE: [OTAnn] Feedback
On Tue, 2005-11-08 at 10:32 -0800, List Mail User wrote: No, this *is* spam. They're hosted by Hurricane Electric, who clearly wouldn't care; But they are registered by easyDNS and get name service from them - who probably does care (it looks like a violation of easyDNS's TOS/AUP). Someone who has seen multiple copies of this should send an email to easyDNS. Easy enough to see how many groups these morons are spamming: http://www.google.com/search?hl=enlr=q=roomity+%22I+was+interested+in +getting+feedback%22btnG=Search Thomas
Using spam tools for viruses
Howdy - I recently responded to a thread on a local LUG mailing list where a guy wanted to report a virus as spam. I have always thought that using a spam tool to fight viruses was wrong, and I said so. He asked why, and basically my response was use the right tool for the job, as in use a virus tool for viruses, and use a spam tool for spam. What is the conventional wisdom on this list? Should viruses be reported as spam? If so, why? If not, why not? Thanks! Thomas
Re: [OT] Looking for a cartoon for a proposal cover
On Sun, 2005-09-18 at 04:31 -0700, Loren Wilton wrote: This is almost completely off topic, but someone here might know where I can find something like what I'm looking for. I'm doing a proposal on flattening out an incredibly hierarchical architecture to make it more efficient. I'm looking for a cartoon I can put on the front page that has some Donald-Duck like character with a HUGE mallet SMASHING it down onto something that is now completely flat. Maybe with steams of 1s and 0s coming out from under the mallet. Or maybe just smash type lines coming out from the mallet, I can add my own binary streams. I'm absolutely positive I've seen any number of cartoons of this general sort over the years, but I'm not having a lot of luck finding something like that at the moment. Suggestions appreciated. Loren Is this what you are looking for? http://simpler-solutions.net/jansdiary/images/pressanykey.jpg Thomas
Re: More unintentional spam humor/irony
At 03:21 PM 9/11/2005, Justin Mason wrote: The choice of anti-bayes-filler below is unfortunate on so many levels nasty. but unsurprising -- I've always thought that news/current events would make the best bayes poison -- certainly beats 19th century prose J, I think the unfortunate part that Barton was referring to (the part that creates humor) is the joining of e-colli with a weight loss spam. Getting e. coli is a quick way to loose weight, but a VERY unpleasant and rather grotesque way to do it. (slightly gross, as this page describes the symtpoms of e. coli, but nothing too graphic:) http://www.cdc.gov/ncidod/dbmd/diseaseinfo/escherichiacoli_g.htm So, how would you like to try my new weight loss program, recognized by the CDC itself! I dunno, I thought the mention of the Army Corps of Engineers and pumping in the same message as a lose weight message was pretty funny as well... Thomas
Re: phish/bayes
On Thu, 2005-08-25 at 15:49 -0700, satalk (sent by Nabble.com) wrote: I could not find any email in this forum addressing this issue - it does not mean there is not one - I just could'nt find it :) MY question is as follows: Given that so many valid tokens from ebay/paypal sites exist in phish emails, am I correct in saying that it is imperative to avoid phish emails entering the bayes database? It has been my experience that the more of them I teach Bayes, the less get through. None of my legit eBay/PayPal e-mail has been tagged. Thomas
Re: When is Bulk Bulk
On Tue, 2005-08-09 at 13:37 -0400, Rob McEwen wrote: When is Bulk Bulk? The reason I ask is because I have a client who sends unsolicited e-mails to prospective clients. But he does this manually by visiting relevant web sites and then one-at-a-time, he personally e-mails these prospective clients. I don't consider this spam because it is not bulk and my client can actually tell you who he e-mailed that day and why. Still, this is a very slippery slope... what happens if he e-mails 50 such addresses that he manually spotted using a generic form letter? Would that be spam? I'm thinking yes. ...However, if these e-mails are sent one at a time and individualized to the recipient in a way that could NOT possibly be computer generated (not another I visited your web site and I think its great statements... but meaningful content that only a person with knowledge of the recipient could write)... in that case, I think he is ok, even if most of each letter came from a generic template. Maybe there are no hard simple rules... but I'd sure love some additional advice? Spam is often called UCE - unsolicited commercial e-mail. If it's commercial, and it is unsolicited, and it's e-mail, it's spam. If you are off-loading your advertising costs onto *my* e-mail system, it's a sure-fire way to make sure I never use your product or service. Thomas
RE: When is Bulk Bulk
On Tue, 2005-08-09 at 23:06 -0400, Rob McEwen wrote: I applaud both of your tenacity in your fight against spam... but do you really think that the average user is going to be soo offended by the particular message that I originally described on this thread if received only once? Goddamn right I will. If you send me UCE and through some miracle it somehow manages to get through all the spam blocking tools I have in place, your company or organization is permanently and irrevocably doomed to never get any business from me. You're spending *my* money to advertise to me, and that seriously pisses me off. If I want your product, I will do research. If your product is the best in its class, I will buy it. If your product is spamvertised, your screwed getting me as a client. Thomas
RE: When is Bulk Bulk
On Tue, 2005-08-09 at 15:59 -0400, Rob McEwen wrote: OBSERVATION: Could some of us be treating unsolicited Business-to-Consumer and unsolicited Business-To-Business the same? Should they be treated the same? If it walks like a duck and quacks like a duck, it's a duck. No matter if it's B2B or B2C. If not, the perhaps some people's irritation about getting called at dinner-time for the 10th time by the same phone company be influencing their opinions here? Nope - I own a small business and I dealt with B2B spam as often as I dealt with B2C. It's all spam, and it all ends with the same results - the spammer loses my biz forever. Thomas
RE: When is Bulk Bulk
On Tue, 2005-08-09 at 16:36 -0400, Rob McEwen wrote: But I do hate the idea of someone sending out 10 unsolicited but hand-typed e-mails being treated the same as a spammer sending out 10,000 unsolicited and impersonal e-mails per day... but somehow I think that this is already taken care of in spite of what some of the more aggressive mail administrators have said today. You miss the point - UCE is UCE is UCE, no matter how nice the guy is who sends it or whether it is hand typed or not. It pushes the cost of the sender's advertising onto the victim. In pretty much any other arena this would be called theft of service and prosecutable. The reason that is not the case with spam is because of people like you who have the attitude that a little spam is OK. No, it's not. UCE is not OK, no matter what. It should be treated as theft of service. I've set up dozens of SpamAssassin servers for clients to the tune of many many thousands of dollars, and I'm a pretty small operation. Do you think they have me set these up because they like me and they want to put money in my pocket? No! It's because it costs them more to deal with spam when it hits their users inboxes than it does to deal with it at the server. Spam has cost my clients TONS of money. It's wrong, no matter how well intentioned it is. If you support a spammer then you are part of the problem. Nothing against you personally Rob - I am sure you're a nice guy. You should not support people who spam. Thomas
RE: When is Bulk Bulk
On Tue, 2005-08-09 at 16:56 -0400, Rob McEwen wrote: There is no way you can prove in your message that it is not a spam run of 10,000. If it wasn't personalized or very personalized, then that would be true. Is it unsolicited? Is it commercial? Is it e-mail? Then it's spam. Don't make me pay for your advertising. Thomas
Re: Bogus MS 'critical update'
On Mon, 2005-07-25 at 10:33 +0100, Nigel kendrick wrote: I have just had a bogus Microsoft update slip through the net. Is there a rule to combat these? In any case, here's the info in case it's of use: snip IMHO that's a virus, not spam. You should prolly install ClamAV on your mail server. -- Thomas Cameron, RHCE, CNE, MCSE, MCT 512-241-0774 (office) 512-924-8592 (cell)
Re: Multiple messages on this list
On Fri, 2005-06-17 at 18:54 +0100, Duncan Hill wrote: On Friday 17 June 2005 12:33, Kai Schaetzl wrote: I've been getting multiples of messages from this list recently as if the list software is sending out a spool again and again. Is the list admin aware of the problem? Check the headers and see if there's anything about SMTPSVC and pickup. If there is, you might be victim of a wonderful bug in Small Business Server 2003(?) that causes wonderful mail storms. No, I am on a pure Linux e-mail environment and I am seeing it as well. No Microsoft products here. Thomas
Re: DNS lookup fails
On Sat, 2005-06-11 at 12:15 -0400, Rick Macdougall wrote: Hi, As was mentioned yesterday on the list, Net::DNS 0.50 seems to be broken. If you are running 0.50, upgrade to 0.51 or downgrade to 0.48 I found that I had to downgrade to 0.48_1, 0.48_3 was broken. YMMV. Thomas
Re: DNS lookup fails
On Sat, 2005-06-11 at 18:41 +0200, Stefan Ewert wrote: but as you can see im using dns version 0.51 and it doesnt work for me. so if anyone has another suggest id happy to hear about it ;) 0.51 didn't work for me, either on RHEL 2.1. I had to downgrade to http://www.net-dns.org/download/Net-DNS-0.48_01.tar.gz. 0.48_3 was b0rken for me, too. Thomas
Re: Advice for a weekend spam assassin?
On Fri, 2005-06-10 at 08:06 -0700, James Bucanek wrote: Greetings, I consider myself a weekend spam assassin. I run my own server (co-located), and have about a dozen users (mostly friends and family, but a few paying customers). But running a mail server isn't my day job. I don't run Razor or any of the cooperative spam filters simply because I didn't have the time to figure them out and set them up. I'm running Spamassassin 3.0.2 which I installed a few months ago. SA is still only catching about 50-75% of the spam. I've set up Bayes learn ham/spam mailboxes, and I regularly feed them 200 to 500 messages a day. Yet even after months of training, I still get messages like this: Subject: (6/10/05) Mortgage Rate Report X-Spam-Status: No, score=3.6 required=7.0 tests=BAYES_99,HTML_80_90, HTML_FONT_TINY,HTML_IMAGE_RATIO_04,HTML_MESSAGE,NORMAL_HTTP_TO_IP, OPTING_OUT autolearn=no version=3.0.2 As you can see, the Bayes filter has nailed it as spam, but it still only gets a score of 3.6. I currently have my threshold set to 7.0. I've been considering lowering it again (maybe to 5.0), but am paranoid about false positives. I can go through my mailbox and see ham that has scores of 3 or even 4. I was hoping that someone here could give me some quick advice as to what I might be doing wrong, or point me to a trouble-shooting site for SA. I was previously using a client-side Bayes filtering system and was getting 99.8+% spam identification rates. SA has been, so far, a bit of a disappointment and I'm sure it's my fault. :) I have SA (plus spamass-milter to reject, but that's not important for this discussion) on a bunch of servers at various client sites. All of them except one just flat stop spam. Period. Those clients are just tickled pink with the results. The one client who does not allow me to use Razor, Pyzor and DCC (they won't open their firewall) is very dissatisfied with the solution. It is incredibly frustrating. So my answer to you would be to install those three helpers and make sure that you have a recent Net::DNS installation. You will see accuracy go *way* up. Thomas
RDJ errors
Hey all - I am brand new to RDJ. I just set up my script and I am getting the no index errors below. Is this normal? ** Rules Du Jour Run Summary:RulesDuJour Run Summary on vidar: No index found for ruleset named SARE_REDIRECT_POST300. Check that this ruleset is still valid. Ruleset for html coding abuse has changed on vidar. Version line: # Version: 01.03.06 SARE Specific Ruleset has changed on vidar. Version line: # Version: 01.03.05 SARE BIZ/Marketing/Learning Ruleset (for SA ver. 2.5x and greater) has changed on vidar. Version line: # Version: 01.02.02 # The BML set has been renamed to match SARE's updated standards, the new name is 72_sare_bml_post25x.cf SARE Fraud Detection Ruleset (for SA ver. 2.5x and greater) has changed on vidar. Version line: # Version: 01.03.02 # NOTE: Please update your scripts to pull this file from it's new location http://www.rulesemporium.com/rules/99_sare_fraud_post25x.cf SARE Spoof Ruleset for SpamAssassin has changed on vidar. Version line: # Version: 1.06.12 SARE OEM Ruleset for SpamAssassin has changed on vidar. Version line: # Version: 1.05.07 No index found for ruleset named SARE_GENLSUBJ1. Check that this ruleset is still valid. No index found for ruleset named SARE_GENLSUBJ2. Check that this ruleset is still valid. No index found for ruleset named SARE_GENLSUBJ3. Check that this ruleset is still valid. No index found for ruleset named SARE_UNSUB. Check that this ruleset is still valid. No index found for ruleset named SARE_uri0. Check that this ruleset is still valid. No index found for ruleset named SARE_uri1. Check that this ruleset is still valid.
At wit's end - SA is *still* tagging list traffic!
All - I have added these to my local.cf: whitelist_from_rcvd [EMAIL PROTECTED] But I am still seeing list traffic with spam samples being tagged. Can someone please tell me what on Earth I need to do to tell SA to ignore anything on this list? Procmail rules are not an option - I use SA on a relay server which uses a milter. Thanks Thomas
RE: At wit's end - SA is *still* tagging list traffic!
On Thu, 2005-06-02 at 16:42 -0500, Kristopher Austin wrote: Thomas, You can do one of two things: whitelist_to users@spamassassin.apache.org or whitelist_from_rcvd [EMAIL PROTECTED] apache.org I prefer the latter. Notice the correct format as opposed to what you used. Make sure to restart SA after performing a --lint. Kris Not that I am arguing, but that's not what the man page says. The example for whitelist_from_rcvd there shows this: whitelist_from_rcvd [EMAIL PROTECTED] Why is your syntax better? Again, not arguing, just want to understand. Thomas
[SOLVED] Re: At wit's end - SA is *still* tagging list traffic!
On Thu, 2005-06-02 at 16:32 -0500, Thomas Cameron wrote: All - I have added these to my local.cf: whitelist_from_rcvd [EMAIL PROTECTED] But I am still seeing list traffic with spam samples being tagged. Can someone please tell me what on Earth I need to do to tell SA to ignore anything on this list? Procmail rules are not an option - I use SA on a relay server which uses a milter. Thanks Thomas I was whitelisting apache.org instead of spamassassin.apache.org. I assumed (bad, I know) that child domains would be covered by whitelisting the parent domain. Now my local.cf setting is: whitelist_from_rcvd [EMAIL PROTECTED] Thanks to all who helped. Thomas
[REALLY SOLVED THIS TIME] Re: At wit's end - SA is *still* tagging list traffic!
On Thu, 2005-06-02 at 16:32 -0500, Thomas Cameron wrote: All - I have added these to my local.cf: whitelist_from_rcvd [EMAIL PROTECTED] But I am still seeing list traffic with spam samples being tagged. Can someone please tell me what on Earth I need to do to tell SA to ignore anything on this list? Procmail rules are not an option - I use SA on a relay server which uses a milter. Thanks Thomas My last was a typo - the line in local.cf is whitelist_from_rcvd [EMAIL PROTECTED] apache.org That causes SA to score messages with -100. Thanks all! Thomas
Re: Is Bayes Really Necessary?
On Thu, 2005-05-26 at 10:08 -0400, Jake Colman wrote: Given the rather complete set of rules that ship with SA and which can expanded with SARE, does bayes learning really help? Won't the rules catch pretty much everything anyway? I have used SA with Bayes and it took quite a bit of administrative overhead. It worked amazingly well, though. I now run SA with DCC, Razor, Pyzor and network checks and without Bayes and it still Just Works(TM). Seriously - I have customers who slather their e-mail addresses all over Usenet, message boards, on their web pages, etc. They might as well put a big sign up that says SPAM ME PLEASE!!! But they don't get any spam - SA and spamass-milter rejects all of it. It is really amazing - I've got clients who went from hundreds of spams per day down to one or two that slip through per week. Of course, when one gets through, my phone rings! I guess my experience is that either way, SA Just Works(TM). Cheers, Thomas
Re: Help mp3 attachment
On Sun, 2005-05-15 at 22:34 -0500, John Fleming wrote: I run a very simple Postfix - Procmail - SpamAssassin - CLamAV setup that has been working great, but tonight I see something I don't understand. I suspect that your procmail recipe doesn't scan files over a certain size. What does your .procmailrc file look like? Thomas
Suddenly load average of 15-18???
All -
spamc is suddenly bringing my mail server to its knees.
Running RHEL 4 with the spamassassin-3.0.1-0.EL4 (supplied by Red Hat) and
spamass-milter-0.3.0-3 (I made that RPM) along with razor-agents-2.67-0,
dcc-1.3.0-0 and pyzor-0.4.0-0.
All of a sudden about two days ago spamc processes were chewing up the
machine - sendmail was actually rejecting messages because the load average
was so high! This is a machine that is only used for about 6 users... It
only handles around a thousand to two thousand messages a day. I am the
only admin on it and nothing has changed.
Here is my local.cf:
--- begin ---
required_score 5
report_safe 1
rewrite_header subject **SPAM** _SCORE_
ok_languages en
ok_locales en
use_dcc 1
use_pyzor 1
use_razor2 1
whitelist_from_rcvd [EMAIL PROTECTED]
whitelist_from_rcvd [EMAIL PROTECTED]
score ALL_TRUSTED 0 0 0 0
--- end ---
Here are the relevant lines from my sendmail.mc:
--- begin ---
INPUT_MAIL_FILTER(`greylist',`S=local:/var/milter-greylist/milter-greylist.sock')dnl
define(`confMILTER_MACROS_HELO', `{verify}, {cert_subject}')dnl
define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}')dnl
INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass.sock, F=,
T=C:15m;S:4m;R:4m;E:10m')dnl
define(`confMILTER_MACROS_CONNECT',`b, j, _, {daemon_name}, {if_name},
{if_addr}')dnl
INPUT_MAIL_FILTER(`clamav-milter',
`S=local:/var/run/clamav/clamav-milter.sock, F=T,T=S:4m;R:4m;E:10m')
--- end ---
I have no idea why it is doing this... It was working fine and then this
happened sort of out of the blue. Any pointers?
Thanks!
Thomas
Re: Suddenly load average of 15-18???
On Thu, 2005-05-12 at 11:19 -0400, Stephen M. Przepiora wrote: Take a look at the switches you have in /etc/init.d/spamassassin change them to only run 5 processess and to die off after 15 or twenty scans. -m5 --max-conn-per-child=5 Steve I just tried that and as soon as I restarted everything the load shot up to ~ 6. I had to kill everything and remove the SA milter. I'd like to figure out what the root cause is rather than band-aid the symptom. Anyone have any ideas why this would suddenly start? Thomas
Re: Suddenly load average of 15-18???
On Thu, 2005-05-12 at 08:31 -0700, Loren Wilton wrote: Usually a high load average means that a spamd child suddenly (or possibly slowly) got fat, and you are out of memory and thrashing to beat the band. The two most common causes of this seem to be Bayes expiry runs and Awl expiry runs. Sometimes though it can seemingly happen from some unknown sequence of mail messages. Is there something I should/could do about these expiry runs? It seems odd that it's been like this for a couple of days now... How could I know that this was the issue? How many children are you running? What is the max lifetime (messages processed) per child? Limiting to probably 5 children, or maybe even less in your case with so few users, and limiting to maybe 20-100 connections per child will probably work around your problems. My rc file has this: SPAMDOPTIONS=-d -c -m5 --max-conn-per-child=5 -H I just added the --max-conn-per-child=5 per Stephen Przepiora's suggestion but that didn't seem to help. Oh, I'm assuming you have at least 512M or so. If not, you might want to cut down to only a couple of children, and definitely go with the lower number of connections per child. Yes, I have 512M. As I said - this has been working flawlessly since the server was installed several weeks ago. It just suddenly went bonkers a couple of days ago. Thomas
Re: Suddenly load average of 15-18???
On Thu, 2005-05-12 at 18:10 +0200, Christoph Petersen wrote: Hi, Thomas Cameron schrieb: I just tried that and as soon as I restarted everything the load shot up to ~ 6. I had to kill everything and remove the SA milter. I'd like to figure out what the root cause is rather than band-aid the symptom. Anyone have any ideas why this would suddenly start? Do you use the sa-blacklist? I've recently had problems with it. My load was getting very high. I have done nothing past the initial installation and adding spamass- milter... This is about as vanilla an installation as you can get. Thomas
[SOLVED] Re: Suddenly load average of 15-18???
OK, this is a weird solution... I rebooted the server and all the problems went away. It's chuffing along happily now. Memory leak, maybe? Thomas
RE: [SOLVED] Re: Suddenly load average of 15-18???
On Thu, 2005-05-12 at 11:46 -0500, Jon Dossey wrote: From: Thomas Cameron [mailto:[EMAIL PROTECTED] Sent: Thursday, May 12, 2005 11:38 AM To: spamassassin-users; spamass-milt-list@nongnu.org Subject: [SOLVED] Re: Suddenly load average of 15-18??? OK, this is a weird solution... I rebooted the server and all the problems went away. It's chuffing along happily now. Memory leak, maybe? What kind of hardware? Are you scanning zips? I had to just start blocking zip attachments all together until these virii settle down a bit. .jon It's just a plain Jane P-III 800MHz with 512MB memory on a 7-disk RAID 5 Ultra 160 SCSI array. I have not disabled scanning of zip files. It is running just fine now. Very odd. Thomas
Re: Suddenly load average of 15-18???
On Thu, 2005-05-12 at 10:53 -0500, Dan Nelson wrote: In the last episode (May 12), Thomas Cameron said: spamc is suddenly bringing my mail server to its knees. Running RHEL 4 with the spamassassin-3.0.1-0.EL4 (supplied by Red Hat) and spamass-milter-0.3.0-3 (I made that RPM) along with razor-agents-2.67-0, dcc-1.3.0-0 and pyzor-0.4.0-0. All of a sudden about two days ago spamc processes were chewing up the machine - sendmail was actually rejecting messages because the load average was so high! This is a machine that is only used for about 6 users... It only handles around a thousand to two thousand messages a day. I am the only admin on it and nothing has changed. What's the average processing time for a message, and are you using any -i flags on your spamass-milter commandline? Grep your maillog for in .* seconds, to get the timings. If they're all under 10 seconds or so and you're not using -i, check for things like mail loops, or large outgoing mail bursts. It was up around 50-60 seconds per message. I rebooted the machine and it has cleared up. Thanks for the help! Thomas
RE: [SOLVED] Re: Suddenly load average of 15-18???
On Thu, 2005-05-12 at 12:20 -0500, Jon Dossey wrote: This may only be a temporary fix. Personally, rebooting a linux machine to solve a problem just isn't acceptable. Did you try restarting spamd before rebooting? Several times. I restarted the entire mail suite - sendmail, clam, SA, milter-greylist, etc. I'd go through your maillog, and check the spamassassin processing times, and see if you can pinpoint where the processing time shoots up. Then, go through your mqueue and take a look at the offending message. It wasn't just one message. It was every message. Thomas
Re: Suddenly load average of 15-18???
On Thu, 2005-05-12 at 09:31 -0700, Loren Wilton wrote: Is there something I should/could do about these expiry runs? It seems odd that it's been like this for a couple of days now... How could I know that this was the issue? Um, this isn't my area of expertise. I suspect Matt or Justin will be along with a workable suggestion fairly soon. I'm pretty sure that there is some logging to indicate when an expiry run happens, but I don't know precisely what to look for. OK, I'll look for that. At least with bayes there is a way you can turn off the auto-expire and then use a cron job to schedule a manual expiry once a day/week/whatever. I'm not sure if similar functionality exists for awl. I don't know either. Did you happen to notice if all of your spamd children get fat at once, or if just one of them got really huge? All of them gettiing big might indicate something changed with your rules files. A single fat child would be more indicitave of an expiry run. Loren It didn't really look like any of them were really fat... The machine's drives just started hammering and the load average shot up. It's all cleared up now after a reboot. Thomas
