spamassassin bug
At the suggestion of a local user I ran 'sa-update -D' to bring my Slackware-12.2 system running SA-3.2.5 up to date. Instead, I just dug myself a hole and fell in by running the above. Sigh. What I see as a result is: [6753] error: check: no loaded plugin implements 'check_main': cannot scan! at /usr/lib/perl5/vendor_perl/5.10.0/Mail/SpamAssassin/PerMsgStatus.pm line 164. check: no loaded plugin implements 'check_main': cannot scan! at /usr/lib/perl5/vendor_perl/5.10.0/Mail/SpamAssassin/PerMsgStatus.pm line 164. My Google search found one thread on this subject from 2007, but he was running amavisd, too, and it could not find the *.pre files. No help for me there. In /etc/mail/spamassassin/rules/ I have init.pre, v310.pre, and v312.pre files. SA has been running so I am totally clueless why this attempted update finds an error that prevents /usr/bin/spamd from starting again. Perhaps someone here can tell me how to resolve this problem so I can once again invoke spamd. Rich
Re: spamassassin bug
On Mon, 11 Jan 2010, Warren Togami wrote: Try wiping out /var/lib/spamassassin/*, does spamassassin work then? Warren, I don't have a spamassassin directory in /var/lib/. Rich
Re: spamassassin bug
On Mon, 11 Jan 2010, Kai Schaetzl wrote: In /etc/mail/spamassassin/rules/ I have init.pre, v310.pre, and v312.pre files. which would be a non-default location. Kai, Almost all SA-related files are in /etc/mail/spamassassin/; that's where the Slackware package installs everything. If this is a non-default location, where should they be? FWIW, that's were everything's been for years and I've had no problems until yesterday when running sa-learn kicked up that error. Thanks, Rich
Re: spamassassin bug
On Mon, 11 Jan 2010, David Michaels wrote: find / -name spamassassin -print /etc/mail/spamassassin /etc/mail/spamassassin/blib/script/spamassassin /etc/mail/spamassassin/spamassassin /opt/src/slackbuilds/spamassassin /usr/bin/spamassassin /usr/share/spamassassin Rich
Re: spamassassin bug
On Mon, 11 Jan 2010, Benny Pedersen wrote: rules dir error Benny, Should I copy the .pre files to /usr/share/spamassassin? spamassassin 21 -D --lint | less I can post the results, but the final error still is: check: no loaded plugin implements 'check_main': cannot scan! at /usr/lib/perl5/vendor_perl/5.10.0/Mail/SpamAssassin/PerMsgStatus.pm line 164. What plugin do I need to have loaded to resolve this error? Rich
Re: spamassassin bug
On Mon, 11 Jan 2010, Benny Pedersen wrote: no do not copy, edit the files in there place where thay got installed Benny, OK. where is your pre files located ? if thay are in /etc/mail/spamassassin/rules ? /etc/mail/spamassassin/rules output from --lint tells where thay should be I don't see this: [r...@salmo /etc/postfix]# spamassassin --lint [22649] warn: netset: cannot include 127.0.0.1/32 as it has already been included check: no loaded plugin implements 'check_main': cannot scan! at /usr/lib/perl5/vendor_perl/5.10.0/Mail/SpamAssassin/PerMsgStatus.pm line 164. the above error says me you dont have any pre files loaded at all What do I do to load them? Thanks, Rich
RE: spamassassin bug
On Mon, 11 Jan 2010, Rosenbaum, Larry M. wrote: It looks like you are missing the v320.pre file, which contains Larry, Yes, that's true. I have v310.pre and v312.pre. loadplugin Mail::SpamAssassin::Plugin::Check along with several other important loadplugin lines. So I found and downloaded v320.pre into /etc/mail/spamassassin/rules and still get the error. What have I missed in getting SA to see the new .pre file is now present and calls for the check_main to be run? Thanks, Rich
RE: spamassassin bug -- RESOLVED!
On Mon, 11 Jan 2010, Benny Pedersen wrote: mv /etc/mail/spamassassin/rules /etc/mail/spamassassin no ? Benny: You and Larry gave me the solution. I found v320.pre and put it in /etc/mail/spamassassin/rules. Then, based on your suggestion above, I copied it up one directory level to /etc/mail/spamassassin. Now everything's working. It was the missing v320.pre file, and the .pre files need to be in /etc/mail/spamassassin and not the rules subdirectory. Many thanks to everyone! Rich
Setting Up Additional User with SA
I've read the FAQ and Wiki without seeing an answer to my question. The answer may very well be in a document I've not examined; if it is, please point me to it. Here's the situation: SpamAssassin-3.2.5 is installed here and works well for me with our postfix MTA. We have two users here: me and my wife. I read mail on the server/workstation using alpine and she reads it on her laptop using seamonkey. When spam gets through to my inbox I save it in 'spam-uncaught' and once a week run 'sa-learn' with those messages as '--spam.' Works well for me. My question is what I need to do to set up the equivalent abilities on my wife's laptop (running xubuntu-9.10). Do I need to install SA on her machine, too, or is there a way to filter her mail through the server's installation? Pointers, guidance, and suggestions are needed. TIA, Rich
Re: Setting Up Additional User with SA
On Mon, 14 Dec 2009, Bowie Bailey wrote: That depends on how she is getting the mail. If she is using IMAP, then you can just set up a folder and it will work just like yours does since everything remains on the server with IMAP. If she is using POP3, then you'll have to get more creative. I would be tempted to create a folder for her to use and then add a script to cron to copy the emails over to the server on a regular basis so they can be learned. Bowie, We use POP3 here (qpopper, in fact). If I correctly understand your suggestion, I add a subdirectory to her ~/ called, like mine 'spam-uncaught'. She saves spam messages there, then cron will scp that file to the server where I can run sa-learn on them (or have cron do that, too). That'll work. Many thanks, Rich
Re: Next Rule Causing False Positives: BOTNET
On Sat, 6 Jun 2009, John Rudd wrote: Probably a good approach for your situation. Let me know how the lower score works out for you (when you said 80% in the other message, do you mean you're lowering it to a score of 1.0, or to a score of 4.0?) John, It seems to be working better with the SPAMBOT score lowered to 2.0, and scores of 0.4 for the individual components. Now that my mail has settled back to normal I'll again thank everyone and leave the list. Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Next Rule Causing False Positives: BOTNET
Now that the EMPTY_BODY and mis-identified spam issues have been resolved I've countered a new one creating false positives: the rule (in /etc/mail/spamassassin/Botnet.cf is: describeBOTNET Relay might be a spambot or virusbot header BOTNET eval:botnet() score BOTNET 5.0 I've read Botnet.txt but I've no clue what to do to reduce the number of false positives. I could include a specific example that came today from a client via his Crackberry, if that would help. Do I need to build a white list of all such senders? Is there a better way to tune this rule so it's not triggered so frequently? Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Next Rule Causing False Positives: BOTNET
On Sat, 6 Jun 2009, Karsten Br?ckelmann wrote: This is a third-party plugin, deliberately installed by you. Actually, it was most likely installed with the SA upgrade because I've not made any modifications or tuning to the system. I figure that those who set up defaults know much more than do I, so I leave things until there's a reason for change. With any custom rule-set, it's definitely the admins duty to score it appropriately, and to tune to their specific mail stream. After all, you already tuned SA by installing the rule-set in the first place. It must have been the Slackware build/installation script that included it because adding such a plug-in would not have occurred to me. Regardless, I will decrease the score by 80%. Thanks, Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Next Rule Causing False Positives: BOTNET
On Sat, 6 Jun 2009, John Rudd wrote: The thing thing to do to fix messages from given locations is lean, heavily, upon the sender to get their sending environment fixed. What botnet finds are sites with bad DNS (no full circle reverse DNS), or sending hosts that look like clients instead of looking like servers. If the exact cause was the former, then that site is poorly configured (violating best practices). If it's the latter, then that's a little more tricky. But there are entries you can put in the Botnet.cf to exempt sites that actually can't fix their own reverse DNS, or sites that you really need to communicate with, but that wont fix their reverse DNS. John, The false positives I'm seeing now are primarily from people who know virtually nothing about computers. Sure, they have some competence in their Microsoft applications, but to them anything else is a black box. Not only do they not run their own servers, but they couldn't clearly communicate the problem to someone who could fix it even if they wanted to do so. There are obviously many poorly or mis-configured servers and clients on the 'Net. That's why there's so much spam and malware out there. I'm lowering the score on that rule. Thanks, Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Identifying Source of False Positives -- RESOLVED
On Tue, 2 Jun 2009, Rich Shepard wrote: I started doing this today. Each of the false positive messages was exported from alpine to a file, and I ran sa-learn on that file telling it the text is ham. Today the mail and logwatch summary reports appeared in my inbox and there were no false positives in the holding cell. This may have resolved the issue of missing messages, but I'll continue to monitor and train SA on the ham that was mistakenly labeled as spam. The empty body problem is a more difficult problem. Have procmail save a copy of the raw message somewhere and take a look at it. Make sure there is a blank line between the headers and the body. Run 'spamassassin -D' on this saved message and look for anything unusual in the debug output. This seems to have been resolved by replacing the old /etc/mail/spamassassin/local.cf with the new version. Many fewer rules and other entries, but I no longer see the EMPTY_BODY test adding 2.5 to the scores. Thank you all very much, Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Identifying Source of False Positives -- RESOLVED
On Fri, 5 Jun 2009, Bowie Bailey wrote: In that case, you should be able to track down the issue by comparing the two files. Is the EMPTY_BODY rule defined in the old local.cf file? If so, what does it say? Bowie, Yes, it was in the old local.cf: # for empty message bodies: body EMPTY_BODY m'^[^\n]+\n\s*$' describe EMPTY_BODY Message has subject but no body score EMPTY_BODY 2.5 It apparently used to work, but isn't with the new SA to which I upgraded a few months ago. Thanks, Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: [SA] Identifying Source of False Positives -- RESOLVED
On Fri, 5 Jun 2009, Adam Katz wrote: Since that regex matches nothing, I assume you meant it to be m'^[^\n]+\n\s*$'s or m'^[^\n]+\n\s*$'ms Adam, I didn't write this. It apparently came with the local.cf file a few years ago. Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Identifying Source of False Positives
On Mon, 1 Jun 2009, Bowie Bailey wrote: The empty body problem is a more difficult problem. Have procmail save a copy of the raw message somewhere and take a look at it. Make sure there is a blank line between the headers and the body. Bowie, et al.: Progress is being made. I discovered that the local.cf was for sa-1.3 or so, and there was a local.cf.new in the same directory. I saved the old version and made the .new one the working copy. Many fewer rules. On a real spam that was saved for my examination I see that the EMPTY_BODY check was not triggered. I'll watch this a couple of days and see if that continues to hold true. In the meantime, I'm retraining SA on the false positives to teach it that they are ham rather than spam. When my log summary reports start appearing in my INBOX and the other false positives from the mail lists (such as this one), stop appearing in the spam hold mailbox, I'll relax. Thank you all for the very helpful suggestions. I'll update the status over the next days. Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Identifying Source of False Positives
On Tue, 2 Jun 2009, Charles Gregory wrote: This *really* suggests that one of two things MUST be occuring: 1) What you are seeing is NOT what spamassassin sees. Charles, Quite possible. 2) A character (null/ascii-zeros?) has been injected into the e-mail somewhere in the headers, causing Spamassassin to cease its scan at that point... Hmm-m-m-m. I cannot perceive a scenario where this is selective. For example, the log reports sent by local root to me on the local machine, some messages posted to this mail list (but not others in the same thread), some messages posted to other mail lists (again, not all in the same thread), and so on. There is no consistent pattern other than the locally generated log summary reports. Presuming upon the latter, try examining all the headers injected by other processes like clamav. Particularly where *some* messages receive this treatment, but not *all*, you should be able to find a 'header difference' between the passed and failed messages. No clamav or similar. We run only linux with incoming mail processed by postfix and procmail. Something to try: Setup a custom rule in local.cf to match a custom header X-Spam-Test: YES And then , just before you scan the e-mail with spamassasin, use 'formail' to add that header to the mail. I've not before used formail. SA is called from within ~/procmail/recipes.rc: ## Call SpamAssassin :0fw: spamassassin.lock * 256000 | spamassassin Where do I insert a call to formail and what is the appropriate format? Thanks, Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Identifying Source of False Positives
On Mon, 1 Jun 2009, Bowie Bailey wrote: Your biggest problems here are BAYES_99 and EMPTY_BODY. To fix the Bayes problem, sa-learn some of these messages as ham. Make sure you are learning as the right user... Bowie, I started doing this today. Each of the false positive messages was exported from alpine to a file, and I ran sa-learn on that file telling it the text is ham. The empty body problem is a more difficult problem. Have procmail save a copy of the raw message somewhere and take a look at it. Make sure there is a blank line between the headers and the body. Run 'spamassassin -D' on this saved message and look for anything unusual in the debug output. Part of the problem is that I cannot tell what's unusual in the debug output. When I tried this yesterday (properly), I saw where the score suddenly jumped from 1.2 to 5.21 with no visible (to me) explanation. Rich
Identifying Source of False Positives
I'm running SA-3.2.5 on Slackware-12.2 and encountering false positives on messages that have not before been seen as spam by SA. Specifically, the daily postfix mail log summary report and the daily logwatch report are marked at spam; they are sent by root to me as a user. Because /etc/procmailrc threw these messages away it took a long time to figure out that it was SA mis-labeling these messages that was the immediate problem. Over the past few months I've also had problems with messages from three specific domains that were never delivered to my inbox. However, when a procmail recipe directed all messages to me at my business domain to a different mail file, they were delivered. How can I determine what causes SA to mark the log summary reports as spam? This is the first issue I want to resolve. I saw nothing appropriate on the web site's FAQ or front page so if I missed the information please point me to the appropriate location. Rich
Re: Identifying Source of False Positives
On Mon, 1 Jun 2009, Charles Gregory wrote: Well, firstly, examine the mail full headers. There should be an X-Spam-Status header listing the tests that matched on the e-mail. Charles/Dan/John: I certainly managed to forget this. I just ran /etc/cron.daily/1pflogsumm and looked at the report. Here are the headers: From r...@salmo.appl-ecosys.com Mon Jun 1 11:25:44 2009 Return-Path: r...@salmo.appl-ecosys.com X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 3.2.5-ph20040310.0 (2008-06-10) on salmo.appl-ecosys.com X-Spam-Level: X-Spam-Status: Yes, score=4.9 required=4.0 tests=ALL_TRUSTED,AWL,BAYES_99, EMPTY_BODY,NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR,URI_HEX,URI_NOVOWEL autolearn=no version=3.2.5-ph20040310.0 X-Spam-Report: * -1.3 ALL_TRUSTED Passed through trusted hosts only via SMTP * 3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100% * [score: 1.] * 2.5 EMPTY_BODY BODY: Message has subject but no body * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * 0.4 URI_HEX URI: URI hostname has long hexadecimal sequence * 0.0 NUMERIC_HTTP_ADDR URI: Uses a numeric IP address in URL * 1.6 URI_NOVOWEL URI: URI hostname has long non-vowel sequence * -1.8 AWL AWL: From: address is in the auto white-list X-Original-To: rshep...@appl-ecosys.com I can send the entire report if that's necessary. There is certainly body content in the message; it's not empty so I don't understand the 2.5 on that third test. I also don't know where the 3.5 on the second test arises. For about a decade these log summary reports showed up every day with no problems. Earlier this spring they became sporatic, then ceased appearing at all. This correlates with a distribution and SpamAssassin upgrade, so it must be something different in SA that's triggering this response now. Suggestions on how to proceed greatly appreciated. Thanks, Rich
Re: [sa] Re: Identifying Source of False Positives
On Mon, 1 Jun 2009, Charles Gregory wrote: Just to be clear, are you looking at the body in the actual rejected message, Charles, Yes. The body consists of the mail log summary. First guess, look at the procmail code that 'chooses' to run spamassassin. Have you used an 'h' where you meant to use an 'H', thereby feeding *only* the header to spamassassin? ## Call SpamAssassin :0fw: spamassassin.lock * 256000 | spamassassin This is how it's been for years. Rich
Re: Identifying Source of False Positives
On Mon, 1 Jun 2009, John Hardin wrote: If these are system-generated messages, something is improperly training SA that they are spam. Do you use autolearn? John, No. Once a week or so I run sa-learn specifying spam on the spam-uncaught mbox file. Less frequently I run it on mail list files specifying them as ham. Primarily I'd suggest you exclude locally-generated emails from SA completely. If you'd post the Received: headers from such a message and the procmail stanza where you pass messages to SA for scoring I could suggest something. Here are all headers from the mail log summary: From r...@salmo.appl-ecosys.com Mon Jun 1 11:25:44 2009 Return-Path: r...@salmo.appl-ecosys.com X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 3.2.5-ph20040310.0 (2008-06-10) on salmo.appl-ecosys.com X-Spam-Level: X-Spam-Status: Yes, score=4.9 required=4.0 tests=ALL_TRUSTED,AWL,BAYES_99, EMPTY_BODY,NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR,URI_HEX,URI_NOVOWEL autolearn=no version=3.2.5-ph20040310.0 X-Spam-Report: * -1.3 ALL_TRUSTED Passed through trusted hosts only via SMTP * 3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100% * [score: 1.] * 2.5 EMPTY_BODY BODY: Message has subject but no body * 0.0 NORMAL_HTTP_TO_IP URI: Uses a dotted-decimal IP address in URL * 0.4 URI_HEX URI: URI hostname has long hexadecimal sequence * 0.0 NUMERIC_HTTP_ADDR URI: Uses a numeric IP address in URL * 1.6 URI_NOVOWEL URI: URI hostname has long non-vowel sequence * -1.8 AWL AWL: From: address is in the auto white-list X-Original-To: rshep...@appl-ecosys.com Delivered-To: rshep...@appl-ecosys.com Received: from salmo.appl-ecosys.com (localhost.localdomain [127.0.0.1]) by salmo.appl-ecosys.com (Postfix) with ESMTP id 8DA0F1026 for rshep...@appl-ecosys.com; Mon, 1 Jun 2009 11:25:44 -0700 (PDT) Received: (from r...@localhost) by salmo.appl-ecosys.com (8.14.3/8.14.2/Submit) id n51IPibx030133; Mon, 1 Jun 2009 11:25:44 -0700 Date: Mon, 1 Jun 2009 11:25:44 -0700 From: r...@salmo.appl-ecosys.com Message-Id: 200906011825.n51ipibx030...@salmo.appl-ecosys.com To: rshep...@appl-ecosys.com Subject: *SPAM* salmo Daily Mail Report for Monday, 01 June 2009 X-Spam-Prev-Subject: salmo Daily Mail Report for Monday, 01 June 2009 Report based on information in /var/log/maillog And this is from ~/procmail/recipes.rc: ## Call SpamAssassin :0fw: spamassassin.lock * 256000 | spamassassin Thanks, Rich
Re: [sa] Re: Identifying Source of False Positives
On Mon, 1 Jun 2009, Charles Gregory wrote: Is there anywhere in the procmail recipe *above* this one that some specila condition has been specified as: :0fwh ...which has the effect of 'filtering' the message down to just its headers? It wouldn't necessarily have to be a recent change to your procmailrc, it might just be a subtle change in the log mail that 'triggers' the rule when it didn't before. Charles, # BEGIN RECIPES # Nuke duplicate messages #:0 Wh: msgid.lock #| $FORMAIL -D 8192 msgid.cache ## Call SpamAssassin :0fw: spamassassin.lock * 256000 | spamassassin The first recipe has been commented out for a while now, so the call to SA is at the top of the list. Next guess: Has this log summary grown in size past some limit that would cause the whole body to be 'truncated'? No. The log summary report (with headers) is 26,000 bytes. Rich
Re: Identifying Source of False Positives
On Mon, 1 Jun 2009, Bowie Bailey wrote: Your biggest problems here are BAYES_99 and EMPTY_BODY. To fix the Bayes problem, sa-learn some of these messages as ham. Make sure you are learning as the right user... Bowie, I just did this on a run from this morning. I'll do so again tomorrow morning with both the mail log and log watch reports. The empty body problem is a more difficult problem. Have procmail save a copy of the raw message somewhere and take a look at it. Make sure there is a blank line between the headers and the body. Run 'spamassassin -D' on this saved message and look for anything unusual in the debug output. There is always a blank line between headers and body. I tried running 'spamassassin -D' on the saved message and nothing happened. Should it take more than a few seconds to complete and return a debug report? Thanks, Rich
Re: Identifying Source of False Positives
On Mon, 1 Jun 2009, John Hardin wrote: And I assume you look at the sapm-uncaught file before learning it? Yes. The messages in there are those I deliberately move there after they've ended up in my inbox because neither the postfix filters nor the spamassassin rules caught them. If some log files got in there and were learned, that could explain the deterioration. That seems very reasonable, but I would have had to move them there myself and I cannot recall doing so. Also, before running sa-learn to classify them as spam I look over the list. So, it's quite possible that they ended up classified as spam unintentionally. Have you kept your spam and ham corpa? I'm not sure. The spam comes from the spam-uncaught file which is cleared each time it's run. The ham comes from various mail lists and they grow over time. Okay, let's key on that one. ## Call SpamAssassin : 0fw: spamassassin.lock * 256000 | spamassassin :0 fw: spamassassin.lock * 256000 * ! ^TO_abuse@ * ! ^List-Id: .*?use...@.]spamassassin\.apache\.org? * ! ^Received: from salmo\.appl-ecosys\.com \(localhost\.localdomain \[127\.0\.0\.1\]) by salmo\.appl-ecosys\.com | /usr/bin/spamc Using spamc creates less load than launching spamassassin from scratch for every email, but you do have to manage the daemon (i.e. restart it if the rules change). I run spamd: 2978 ?Ss12:16 /usr/bin/spamd -d --pidfile=/var/run/spamd.pid 3052 ?S 0:04 spamd child 3054 ?S 0:05 spamd child is this not adequate for a light load? Are your resources really so limited that you want to serialize all email delivery? As a middle ground you might consider per-user lockfiles instead, e.g.: :0 fw: $HOME/.spamassassin.lock I'd also suggest upping the size limit a bit, but that's not a big issue. There are more complex things you can do; you might want to take a look at http://www.impsec.org/~jhardin/antispam/spamassassin.procmail There are only two users on this network and a low mail volume for each of us. The size limit has been at that value for years without a problem. I'll keep teaching SA that the log reports are ham and see if that makes a difference. As I wrote earlier, this is all within the past quarter year, and it's been a PITA since it's taken time and attention away from my business. Thanks, Rich
Re: Identifying Source of False Positives
On Mon, 1 Jun 2009, Theo Van Dinter wrote: My guess is you did something like spamassassin -D filename, where filename gets treated as the argument to -D, so then it was waiting for input. Theo, Yes, this is what I did. If this is the case, try spamassassin -D filename /dev/null. :) Interesting: [785] dbg: rules: running uri tests; score so far=1.2 [785] dbg: rules: compiled uri tests [785] dbg: rules: ran uri rule NORMAL_HTTP_TO_IP == got hit: http://211.129.107.12; [785] dbg: rules: ran uri rule URI_HEX == got hit: http://kemp-5d866973; [785] dbg: rules: ran uri rule NUMERIC_HTTP_ADDR == got hit: http://1898218; [785] dbg: rules: ran uri rule URI_NOVOWEL == got hit: http://jcwpjkp; [785] dbg: rules: ran uri rule __DOS_HAS_ANY_URI == got hit: h [785] dbg: eval: stock info total: 0 [785] warn: rules: failed to run CG_FUJI_JPG test, skipping: [785] warn: (Can't locate object method image_name_regex via package Mail::SpamAssassin::PerMsgStatus at (eval 719) line 1315. [785] warn: ) [785] warn: rules: failed to run CG_DOUBLEDOT_GIF test, skipping: [785] warn: (Can't locate object method image_name_regex via package Mail::SpamAssassin::PerMsgStatus at (eval 719) line 1580. [785] warn: ) [785] warn: rules: failed to run CG_SONY_JPG test, skipping: [785] warn: (Can't locate object method image_name_regex via package Mail::SpamAssassin::PerMsgStatus at (eval 719) line 2601. [785] warn: ) [785] dbg: rules: ran eval rule BAYES_50 == got hit (1) [785] warn: rules: failed to run CG_CANON_JPG test, skipping: [785] warn: (Can't locate object method image_name_regex via package Mail::SpamAssassin::PerMsgStatus at (eval 719) line 4000. [785] warn: ) [785] dbg: rules: running rawbody tests; score so far=3.191 [785] dbg: rules: compiled rawbody tests [785] dbg: rules: running full tests; score so far=3.191 [785] dbg: rules: compiled full tests [785] dbg: util: current PATH is: /root/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/java/bin:/usr/lib/java/jre/bin:/usr/lib/java/bin:/usr/lib/java/jre/bin:/usr/lib/qt/bin:/usr/share/texmf/bin [785] dbg: pyzor: pyzor is not available: no pyzor executable found [785] dbg: pyzor: no pyzor found, disabling Pyzor [785] dbg: rules: running meta tests; score so far=3.191 [785] dbg: rules: compiled meta tests [785] dbg: check: running tests for priority: 500 [785] dbg: dns: harvest_dnsbl_queries [785] dbg: async: select found 4 responses ready (t.o.=0.0) [785] dbg: async: completed in 0.149 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:10.96.127.75 [785] dbg: async: completed in 0.156 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:10.178.19.65 [785] dbg: async: completed in 0.155 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:11.25.147.192 [785] dbg: async: completed in 0.155 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:110.0.55.209 [785] dbg: async: queries completed: 4, started: 0 [785] dbg: async: queries active: URI-DNSBL=62 URI-NS=10 at Mon Jun 1 15:53:13 2009 [785] dbg: dns: harvest_dnsbl_queries - check_tick [785] dbg: async: select found 1 responses ready (t.o.=1.0) [785] dbg: async: completed in 0.158 s: URI-DNSBL, DNSBL:sbl.spamhaus.org.:39.0.58.80 [785] dbg: async: queries completed: 1, started: 0 [785] dbg: async: queries active: URI-DNSBL=61 URI-NS=10 at Mon Jun 1 15:53:13 2009 [785] dbg: dns: harvest_dnsbl_queries - check_tick ... [785] dbg: check: is spam? score=3.191 required=4 [785] dbg: check: tests=ALL_TRUSTED,BAYES_50,EMPTY_BODY,NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR,URI_HEX,URI_NOVOWEL [785] dbg: check: subtests=__DATE_700,__DOS_BODY_MON,__DOS_HAS_ANY_URI,__DOS_RCVD_MON,__DOS_REF_TODAY,__ENV_AND_HDR_FROM_MATCH,__FB_NUM_PERCNT,__HAS_ANY_EMAIL,__HAS_ANY_URI,__HAS_MSGID,__HAS_RCVD,__HAS_SUBJECT,__KAM_MED2,__KAM_NUMBER2,__KAM_TIME4,__MISSING_REF,__MSGID_OK_DIGITS,__MSGID_OK_HOST,__MSOE_MID_WRONG_CASE,__NAKED_TO,__NONEMPTY_BODY,__SANE_MSGID,__TOCC_EXISTS,__hk_obfdomreq2 It suddenly jumps from 1.2 to 3.91 after looking for images. I don't know where to fix that. I think that I need to update SPF, too, because that's compiled against an earlier perl version. Rich
Completing SA Upgrade
I upgraded my server to Slackware-12.2 and perl-5.10.0 This meant building and installing the lastest SA, built against perl-5.10.0 rather than -5.8.8. So, SA-3.2.5 is installed, and several perl modules were upgraded from CPAN to accommodate it. However, ... ... the related sa-learn and other tools are still looking for perl-5.8.8. These are in /etc/mail/spamassassin. What do I do to update these tools, particularly sa-learn so thwy work with perl-5.10.0? TIA, Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
OS Upgrade Broke SpamAssassin; Help Needed to Fix
I (finally) upgraded my main server/workstation from Slackware-11.0 to
-12.0. After rebooting into the newer kernel, the mail system wasn't
working. Postfix was OK, and I traced the problem to SpamAssassin. Version
3.1.7 was installed.
Some perl modules needed to be upgraded, so I did those via CPAN. This did
not fix the problem (memory use about 98%, disk trashing, postfix cleaning
running constantly, MTA connection to localhost (127.0.0.1) being refused,
etc.) so I thought an upgrade of SA was in order.
I tried to upgrade to 3.2.5 via CPAN ('perl -MCPAN -e shell; install
Mail::SpamAssassin'). The above mentioned problems still exist as soon as I
turn on the spam filter in /etc/postfix/master.cf. Turned it off again.
Here are the current versions reported by SA:
[EMAIL PROTECTED] ~]$ /usr/local/bin/spamassassin -V
spamassassin: spamassassin script is v3.001007, but using modules v3.002005
SpamAssassin-3.1.7 is installed in /etc/mail/spamassassin/.
How should I proceed to fix the installation so there's only one copy
(either in /etc/mail/spamassassin or /usr/local/bin) and that's the latest
version? I would like to get this fixed ASAP so I can turn it back on and
still have the MTA working.
TIA,
Rich
--
Richard B. Shepard, Ph.D. | IntegrityCredibility
Applied Ecosystem Services, Inc.|Innovation
http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: OS Upgrade Broke SpamAssassin; Help Needed to Fix
On Mon, 1 Dec 2008, Benny Pedersen wrote: you have installed 3.2.5 from cpan and 3.1.7 in rpm ? Benny, Slackware doesn't use rpms. The slack packages end in .tgz and that's what the package tools work with. Rich -- Richard B. Shepard, Ph.D. | IntegrityCredibility Applied Ecosystem Services, Inc.|Innovation http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Spam with image and tons of nonsense words, works fantastic for spammers
On Thu, 1 Feb 2007, z3r0 wrote: My hosting is cool, but I'm spammed. Never thought custom rules where something beyond common users, but seems like it is. Perhaps living with pharmaceutical messages in inbox is not so bad when you get used to. Er, ... pardon me for jumping into this thread. But, can you create ~/.spamassassin? If so, in that directory, create a file called user_prefs and put your custom rules in there. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
use_report_template
It was suggested to me that in local.cf I replace use_terse_report with use_report_template. When I turn on the latter, --lint reports it cannot find any report template to use. I've grep'd all the files in /etc/mail/spamassassin without finding anything relevant, and find clear_report_template in /usr/share/spamassassin/10_misc.cf. Where do I put report_template so it can be used? Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: use_report_template
On Mon, 29 Jan 2007, Theo Van Dinter wrote: You can put the report template in local.cf. use_report_template isn't a valid option though. You should read perldoc Mail::SpamAssassin::Conf. Look for report. Thank you, Theo. I looked at docs and the wiki on the web site without finding anything useful. So, I'll read the perldoc. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Ideas to Identify Base64-encoded Spam?
With your help the amount of spam getting past the various filters in my inbox (and that of my fiancee) has dropped dramatically. I appreciate learning from all of you. The past couple of days has seen the arrival of a new mutant species of spam: the empty message with a Windows .exe attachment that is base64 encoded. SpamAssassin is giving them scores of 0.0. I have a postfix filter checking for exposed .exe attachments, but nothing seems to catch these guys. To add insult to injury, I have a postfix body check for '/Empty or malformed message/' that did nothing when the original message came in, but prevented me from send it on to the list here. :-( Below are the headers from one example, with the uucoded part removed. If I tell pine to look at the attachments, and start to save it, the name comes up with a .exe extension. If anyone has suggestions on how to identify and reject this format, please share them with me. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863 === From [EMAIL PROTECTED] Sun Jan 28 04:31:09 2007 Return-Path: [EMAIL PROTECTED] X-Original-To: [EMAIL PROTECTED] Delivered-To: [EMAIL PROTECTED] Received: by salmo.appl-ecosys.com (Postfix, from userid 1006) id E6FC7DE; Sun, 28 Jan 2007 04:31:08 -0800 (PST) X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on salmo.appl-ecosys.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=4.0 tests=BAYES_50 autolearn=no version=3.1.7 Received: from abjn32.neoplus.adsl.tpnet.pl (abjn32.neoplus.adsl.tpnet.pl [83.7.155.32]) by salmo.appl-ecosys.com (Postfix) with SMTP id 210A258 for [EMAIL PROTECTED]; Sun, 28 Jan 2007 04:29:55 -0800 (PST) Received: from egvvx ([116.82.221.212]) by abjn32.neoplus.adsl.tpnet.pl (8.13.4/8.13.4) with SMTP id l0SCY7oD053228; Sun, 28 Jan 2007 13:34:07 +0100 Message-ID: [EMAIL PROTECTED] Date: Sun, 28 Jan 2007 13:29:22 +0100 From: Dooley Dinah [EMAIL PROTECTED] User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: [EMAIL PROTECTED] Subject: Love at First Sight Content-Type: multipart/related; boundary=010904090903010104020004 [ Empty or malformed message. Displaying raw text. ] --010904090903010104020004 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit --010904090903010104020004 Content-Type: application/x-msdownload; name=flash postcard.exe Content-Transfer-Encoding: base64 Content-Disposition: inline; filename=flash postcard.exe
Re: Ideas to Identify Base64-encoded Spam?
On Sun, 28 Jan 2007, John D. Hardin wrote: Please don't ask SA to become an antivirus or attachment file type security policy enforcement tool. There are already very effective tools to do perform those tasks. We run only linux here, so I ignore Microsoft virii and the like. But, when I get an empty message with an attached .exe file I see it as another form of obfuscated spam. Empty message bodies aren't legitimate SA targets? Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
RE: Ideas to Identify Base64-encoded Spam?
On Sun, 28 Jan 2007, Giampaolo Tomassoni wrote: Yes, they are. But I see often legitimate messages like this. They are probably used when sending something to somebody while having a voice conversation with him/her. I did it, too. Giampaolo, In which case, nothing is lost if the message doesn't appear in our inboxes. If you could turn network tests on, you probably may at least reduce the number of accepted viral e-mails: most of them come from infected dialup hosts and get the score they deserve. OK. I understood most of them to come from poorly secured zombies that were taken over by the spammers. My mail logs show a lot of such junk from cable and DSL connections. All Winduhs, of course. Anyway, this may work to you: bodyEMPTY_BODY m'^[^\n]+\n\s*$' describeEMPTY_BODY Message has subject but no body score EMPTY_BODY 0.001 Your score mileage may vary... :) Thank you. I'm taking it for a test drive. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Interpreting Error/Warning Message
On Fri, 26 Jan 2007, Rich Shepard wrote: Thank you, Theo. I found a binary 'languages' file in /etc/mail/spamassassin/rules/ and copied it to /usr/share/spamassassin. Next time I run sa-learn I'll check it. I ran some ham through the slicer without the warnings appearing, so I did the correct thing. Thanks, Theo, Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Re: Drug spam, some caught some not - none caught by drug rules
On Fri, 26 Jan 2007, Ben Wylie wrote: I recommend the KAM rules list which can be found here: http://www.peregrinehw.com/downloads/SpamAssassin/contrib/KAM.cf This catches the drugs names in these emails. Ben, Where do I put this file so it's seen and used by SpamAssassin? Thanks, Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Re: Drug spam, some caught some not - none caught by drug rules
On Fri, 26 Jan 2007, Ben Wylie wrote: On top of these rules, I have written a rule to give 4 points to any email with an .exe attachment as there have been a lot of these. With the above rules and the 4 for having an exe attachment, it hits a rating of 12. The rule i have for detecting the exe attachment, is this: full EXE_ATTACH /file(?:name)?=\.*\.exe/i score EXE_ATTACH 4.0 I'm not sure if there is a better way of writing it, but it works for me. Thank you. I've appended that to local.cf. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Re: Drug spam, some caught some not - none caught by drug rules
On Fri, 26 Jan 2007, Rich Shepard wrote: Where do I put this file so it's seen and used by SpamAssassin? Nevermind. I put it in /usr/share/spamassassin/ with all the other .cf files. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Drug spam, some caught some not - none caught by drug rules
On Fri, 26 Jan 2007, Jim Maul wrote: Those are the DEFAULT rules. Do not add/remove/modify anything in this folder. custom rules go in /etc/mail/spamassassin/ OK. I'll put the new ones there. You really need to have a better understanding of the basics of SA. I'd suggest going over the documentation again. Specifically: http://wiki.apache.org/spamassassin/WhereDoLocalSettingsGo Sure will -- this weekend. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Interpreting Error/Warning Message
I just ran my weekly sa-learn on my spam-uncaught mail file. This is part of the output (repeated probably for each message in the file): Use of uninitialized value in hash element at /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/Plugin/TextCat.pm line 411. Use of uninitialized value in join or string at /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/Plugin/TextCat.pm line 422. Use of uninitialized value in join or string at /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/Plugin/TextCat.pm line 501. Does this mean I need to re-install TextCat.pm? I also received this at the beginning of the processing: textcat: languages filename not defined These are related, I assume. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Interpreting Error/Warning Message
On Fri, 26 Jan 2007, Rich Shepard wrote: Does this mean I need to re-install TextCat.pm? That's not it. It's up to date. So, I've no idea what to do about those warning messages. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Interpreting Error/Warning Message
On Fri, 26 Jan 2007, Theo Van Dinter wrote: This means that the plugin can't find the languages file, which will normally be installed in the default rules dir (/usr/share/spamassassin). Thank you, Theo. I found a binary 'languages' file in /etc/mail/spamassassin/rules/ and copied it to /usr/share/spamassassin. Next time I run sa-learn I'll check it. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Enhancing Detection of Certain Spam
On Wed, 24 Jan 2007, Doc Schneider wrote: I always run sa-update -D to see what is happening. Thank you, Doc. Copying the script to /usr/local/bin/ also made a difference. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Drug spam, some caught some not - none caught by drug rules
On Thu, 25 Jan 2007, Matt Kettler wrote: The proper command would be: spamassassin -D bayes message1 2 debug1.txt OK. I have a spam message that made it to my inbox today. Empty body, the spam base64 encoded. SA gave it a score of 0 this morning. I've run it through the debug process per the above, but I've no idea how to interpret the results or learn from them what -- if anything -- should be tweaked. How should I make the message and debug output tarball available? Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Drug spam, some caught some not - none caught by drug rules
On Thu, 25 Jan 2007, Andy Figueroa wrote: Rich, if you can post the output as text files to a web site somewhere and just send the link/url, that's the kindest way to to this. And then if I knew what I was doing, I'd go look at them and analyze them for you. Thought it won't be me, I'm sure someone will. Andy et al.: You can use wget http://www.appl-ecosys.com/temp-files/analyzed-spam.tgz. I'll leave it there for a day. Any insight into how to better trap this type of spam would be welcome. I have a few other representative types, too. But, Friday evening I run sa-learn on my spam-uncaught message file and delete them. Thanks, Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
RE: Perl Help With FuzzyOCR Needed
On Tue, 23 Jan 2007, Gary V wrote: Are you using SpamAssassin version 3.1.4 or newer? If not, you need to. Gary, SpamAssassin-3.1.7. What version of FuzzyOcr? 3.5.1 Are you trying to load the plugin from more than one place - in other words if you are trying to load it via an entry in v310.pre, comment that out and instead use the supplied loadplugin entry in FuzzyOcr.cf. I have an 'init.pre,' not a v310.pre. But, that's where I put the loading instructions. But, I will now ... and that did the trick. Thank you. Last issue from lint: same problem with ImageInfo. I have no ImageInfo.cf, so from where should I load that? Other than from /etc/mail/spamassassin/init.pre, that is. Thanks, Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Fixing Broken Rules
When I run spamassassin --lint I see a number of rule-related warnings. However, when I look in /etc/mail/spamassassin/rules/ I can't find the specific text referenced. Here's the pertinent lint output: [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_AHBL_RHSBL .4 is not valid for score, skipping: score DNS_FROM_AHBL_RHSBL .4 [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_RFC_ABUSE .4 is not valid for score, skipping: score DNS_FROM_RFC_ABUSE .4 [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_RFC_BOGUSMX .4 is not valid for score, skipping: score DNS_FROM_RFC_BOGUSMX .4 [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_RFC_DSN .4 is not valid for score, skipping: score DNS_FROM_RFC_DSN .4 [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_RFC_POST .4 is not valid for score, skipping: score DNS_FROM_RFC_POST .4 [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_RFC_WHOIS .4 is not valid for score, skipping: score DNS_FROM_RFC_WHOIS .4 When I grep for the entire string -- including the score -- I get no hits. The DNS rules are in 20_dnsbl_tests.cf. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
RE: Perl Help With FuzzyOCR Needed
On Wed, 24 Jan 2007, Gary V wrote: I placed ImageInfo.pm in the SA Plugin directory /./Mail/SpamAssassin/Plugin Gary, Aha! Found it. I had it in /usr/lib/perl5/5.8.8/. There are several directories with perl modules in them. Gets me confused as to what belongs where. Now this lint error is gone. Thanks. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Fixing Broken Rules
On Wed, 24 Jan 2007, Theo Van Dinter wrote: .4 isn't a valid number and so can't be parsed. Try 0.4. Theo, When I look for '.4' in the rules files I cannot find it. Otherwise I would have changed it. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Fixing Broken Rules
On Wed, 24 Jan 2007, Theo Van Dinter wrote: If you run in debug mode, you'll see all the files that SA is using for configs. At least one of those files will have the invalid config lines. My guess is something in the site-wide config directory (/etc/mail/spamassassin). Theo, Here is the result: [EMAIL PROTECTED] /etc/mail/spamassassin]# ./spamassassin -D [30898] dbg: logger: adding facilities: all [30898] dbg: logger: logging level is DBG [30898] dbg: generic: SpamAssassin version 3.1.7 [30898] dbg: config: score set 0 chosen. [30898] dbg: util: running in taint mode? yes [30898] dbg: util: taint mode: deleting unsafe environment variables, resetting PATH [30898] dbg: util: PATH included '/root/bin', keeping [30898] dbg: util: PATH included '/usr/java/j2sdk1.4.1_02/bin', which doesn't exist, dropping [30898] dbg: util: PATH included '/usr/java/apache-ant-1.5.3-1/bin', which doesn't exist, dropping [30898] dbg: util: PATH included '/opt/qt/bin', which doesn't exist, dropping [30898] dbg: util: PATH included '/usr/local/sbin', keeping [30898] dbg: util: PATH included '/usr/sbin', keeping [30898] dbg: util: PATH included '/sbin', keeping [30898] dbg: util: PATH included '/usr/local/bin', keeping [30898] dbg: util: PATH included '/usr/bin', keeping [30898] dbg: util: PATH included '/bin', keeping [30898] dbg: util: PATH included '/usr/X11R6/bin', keeping [30898] dbg: util: PATH included '/usr/games', which doesn't exist, dropping [30898] dbg: util: PATH included '/opt/www/htdig/bin', keeping [30898] dbg: util: PATH included '/usr/lib/java/bin', keeping [30898] dbg: util: PATH included '/usr/lib/java/jre/bin', keeping [30898] dbg: util: PATH included '/usr/lib/qt/bin', keeping [30898] dbg: util: PATH included '/usr/share/texmf/bin', keeping [30898] dbg: util: final PATH set to: /root/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/opt/www/htdig/bin:/usr/lib/java/bin:/usr/lib/java/jre/bin:/usr/lib/qt/bin:/usr/share/texmf/bin [30898] dbg: message: MIME PARSER START [30898] dbg: message: main message type: text/plain [30898] dbg: message: parsing normal part [30898] dbg: message: added part, type: text/plain [30898] dbg: message: MIME PARSER END [30898] dbg: dns: is Net::DNS::Resolver available? yes [30898] dbg: dns: Net::DNS version: 0.59 BTW, you previously mentioned /etc/mail/spamassassin/rules/ ... What is this directory? It's not standard. I upgraded SpamAssassin-3.1.0 to -3.1.7 using CPAN. That left four files in /etc/mail/spamassassin/, none of which was an executable. Then I went to the site's download page and grabbed the tarball. Built and installed from that. The tarball put the rules/ subdirectory there. In /usr/share/spamassassin I have 25_dkim.cf, 60_whitelist_dk.cf, 60_whitelist_dkim.cf, and sa-update-pubkey.txt. If directories and files are mis-located, point me to a list of the correct locations and I'll move them around. Thanks, Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Fixing Broken Rules
On Wed, 24 Jan 2007, Rich Shepard wrote: [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_AHBL_RHSBL .4 is not valid for score, skipping: score DNS_FROM_AHBL_RHSBL .4 [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_RFC_ABUSE .4 is not valid for score, skipping: score DNS_FROM_RFC_ABUSE .4 [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_RFC_BOGUSMX .4 is not valid for score, skipping: score DNS_FROM_RFC_BOGUSMX .4 [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_RFC_DSN .4 is not valid for score, skipping: score DNS_FROM_RFC_DSN .4 [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_RFC_POST .4 is not valid for score, skipping: score DNS_FROM_RFC_POST .4 [22906] warn: config: SpamAssassin failed to parse line, DNS_FROM_RFC_WHOIS .4 is not valid for score, skipping: score DNS_FROM_RFC_WHOIS .4 I was looking in the wrong file, it should be 50_scores.cf. And, in that file there are: score DNS_FROM_AHBL_RHSBL 0 0.306 0 0.231 score DNS_FROM_RFC_ABUSE 0 0.479 0 0.200 score DNS_FROM_RFC_BOGUSMX 0 2.034 0 1.945 score DNS_FROM_RFC_DSN 0 2.872 0 2.597 score DNS_FROM_RFC_POST 0 1.440 0 1.708 score DNS_FROM_RFC_WHOIS 0 0.879 0 1.447 score DNS_FROM_SECURITYSAGE 0 2.001 0 1.513 So there are no '.4' scores in that set. Does this help identify the problem? Thanks, Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Fixing Broken Rules -- FIXED!
On Wed, 24 Jan 2007, Adam Lanier wrote: Did you do a grep for 'DNS_FROM_AHBL_RHSBL' (for example) in /etc/mail/spamassassin? Adam, I _thought_ that I had, but I'm trying to simultaneously fix problems with three applications, so I missed doing this. I'm betting you have a 'score DNS_FROM_AHBL_RHSBL .4' in local.cf or some other cf file in that directory. You win! But, in my defense, I did not put those there. Regardless, I put a leading 0 on each .4 and the problem has been fixed. Now, ... to get the last glitches from FuzzyOCR and I'll be done with this problem set. Many thanks, Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Enhancing Detection of Certain Spam
The quantity of spam for various varieties of quicker-dicker-upper pills into my inbox has increased over the past couple of weeks. The filters in /usr/share/spamassassin/20_drugs.cf don't seem to be up to full strength. Are there newer rules (I'm running 3.1.7) to cover the changing landscape of the spammer's attempted obfuscation of the names? Or, are there other tricks I should learn to tune the Baysian system? Suggestions greatly appreciated. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Enhancing Detection of Certain Spam
On Wed, 24 Jan 2007, Daryl C. W. O'Shea wrote: Use sa-update. Thank you, Daryl. I guess that I'm up to date on those rules so I need to tweak elsewhere. I read in the man page that the default directory is /var/lib/spamassassin/3.001007. That directory does not exist here. I have /usr/share/spamassassin which contains rules (it had a few and I copied over the rest from /etc/mail/spamassassin/rules. If the directory structure created by my manual installation of the 3.1.17 upgrade, where can I learn what that structure should be? Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Re: Enhancing Detection of Certain Spam
On Wed, 24 Jan 2007, Daryl C. W. O'Shea wrote: Have you actually run sa-update? I just did so. Nothing was returned. I have no idea what rules you're copying, where you're copying them to, or why your even copying them anywhere. I originally updated with CPAN. That left no executable binary file. Then I updated from the tarball. That made a directory, rules/, in /etc/mail/spamassassin. When I looked further, I found /usr/share/spamassassin that had four old rule files, so I copied from /etc/mail/spamassassin/rules to /usr/share/spamassassin. /usr/share/spamassassin is the rules that ship with the tarball and is only used if you haven't updated with sa-update OK. /var/lib/spamassassin/versionis the update rules for version OK. I'll try sa-update again in a few days. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Perl Help With FuzzyOCR Needed
I'm trying to set up FuzzyOCR as a plug-in to SpamAssassin. Wrote to the author several days ago but have not received a response. The errors I'm seeing appear to be perl issues or OS issues, not specifically related to the application. Here is the error message I see: plugin: failed to parse plugin /etc/mail/spamassassin/FuzzyOcr.pm: Can't locate FuzzyOcr/Logging.pm in @INC (@INC contains: /etc/mail/spamassassin /usr/lib/perl5/site_perl/5.8.8/i486-linux /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/5.8.8/i486-linux /usr/lib/perl5/5.8.8 /usr/lib/perl5/site_perl) at /etc/mail/spamassassin/FuzzyOcr.pm line 24. BEGIN failed--compilation aborted at /etc/mail/spamassassin/FuzzyOcr.pm line 24. plugin: failed to create instance of plugin FuzzyOcr: Can't locate object method new via package FuzzyOcr at (eval 30) line 1. I changed the perms on FuzzyOcr.pm and Logging.pm to 755. I don't understand why perl failed to parse the plugin /etc/mail/spamassassin/FuzzyOcr.pm since a copy of that module is in that directory. Further, FuzzyOcr/Logging.pm is also in /etc/mail/spamassassin. The error at line 24 is: use FuzzyOcr::Logging qw(debuglog errorlog warnlog infolog); Since I don't know perl, I'm lost here. Rich -- Richard B. Shepard, Ph.D. |The Environmental Permitting Applied Ecosystem Services, Inc.| Accelerator(TM) http://www.appl-ecosys.com Voice: 503-667-4517 Fax: 503-667-8863
Incoming mail writes on console
Setup: Slackware-10.0, 2.4.26 kernel Postfix-2.1.4 procmail SpamAssassin-2.64 SA invoked from within procmail MUA: pine-4.60 This is an unplanned upgrade from my former Red Hat 7.3/postfix-2.0.19/SpamAssassin-something when the hard drives failed after the power supply failed on the running manchine. Anyway, I believe that I have everything properly configured, but incoming mail to my inbox causes the system's speaker to beep and the text overwrites the console with parts of the incoming message. It's quite messy. Even when I'm in X Window, the messages show up on the underlying console and I see them when I kill X to log out. At first I and the folks I consulted on this thought it might be a syslog error, but it's not the /var/log/maillog messages that show up on the console, but the message itself (partially, not fully). I don't know that this is specifically related to SpamAssassin, but I suspect that it might be. I've searched the Web with google but haven't found anything useful. All help greatly appreciated. TIA, Rich -- Dr. Richard B. Shepard, President Applied Ecosystem Services, Inc. (TM) http://www.appl-ecosys.com
