Re: Custom rule in local.cf
On 09/06/2011 10:58 AM, Axb wrote: On 2011-09-06 10:43, J4K wrote: Hi, I know that this is probably the hundredth time I have emailed to the list about my custom rules. Usually, someone points out the blindingly obvious when I fail to note it. This has been going on for months. I'll have a last hack at this, and then be quiet on this topic for at least a month, and then pester you all again. Apologies in advance. I have a rule in the local.cf. I am trying to use it to catch mail show in http://pastebin.com/exWh652x It does not hit. The two hits should be the Subject: Virtual Assistant Position and the email address hitting web-newcarer\.(com|net)\,Thank describe PRIVATE_RULE1 English language job opportunity body __PR1/(Employment opportunity|Job offer match, respond to apply|Employment you've been searching|Job opportunity|Career opportunity inside|Position opening in your area|Work offer inside|Vacancy - apply online|Job ad - see details! Sent through Search engine|Get a New Job Today|Working Part Time|Virtual Manager Vacancy|Virtual Assistant Position|Start New Employment Today)/i uri __PR2 /(joblists\.(com|net)|web-newcarer\.(com|net)|web-newcarer\.(com|net)\,Thank|gb-totaljob\.com)/i meta PRIVATE_RULE1 (__PR1 __PR2) scorePRIVATE_RULE1 2.5 \,Thank in a uri rule? you don't need the __PR1 rule untested.. this should be enough to catch the stuff - no realy need for the body strings in __PR1 rawbody RBODY_JOB_DOMAINS /\@(?:joblists|web-newcarer|gb-totaljob)\./ h2h Well, I put the Thank in there because I hoped it would catch the silly email address that they put in. More than likely the reason it failed is because of my incorrectly defined rule. Oh well, nice to keep you on your toes! Thank-you for introducing me to rawbody rules. describe RBODY_JOB_DOMAINS English language job opportunity rawbody RBODY_JOB_DOMAINS /\@(?:joblists|web-newcarer|gb-totaljob)\./ scoreRBODY_JOB_DOMAINS 2.5
Re: Custom rule in local.cf
On 09/06/2011 12:07 PM, Axb wrote: On 2011-09-06 12:03, J4K wrote: On 09/06/2011 10:58 AM, Axb wrote: On 2011-09-06 10:43, J4K wrote: Hi, I know that this is probably the hundredth time I have emailed to the list about my custom rules. Usually, someone points out the blindingly obvious when I fail to note it. This has been going on for months. I'll have a last hack at this, and then be quiet on this topic for at least a month, and then pester you all again. Apologies in advance. I have a rule in the local.cf. I am trying to use it to catch mail show in http://pastebin.com/exWh652x It does not hit. The two hits should be the Subject: Virtual Assistant Position and the email address hitting web-newcarer\.(com|net)\,Thank describe PRIVATE_RULE1 English language job opportunity body __PR1/(Employment opportunity|Job offer match, respond to apply|Employment you've been searching|Job opportunity|Career opportunity inside|Position opening in your area|Work offer inside|Vacancy - apply online|Job ad - see details! Sent through Search engine|Get a New Job Today|Working Part Time|Virtual Manager Vacancy|Virtual Assistant Position|Start New Employment Today)/i uri __PR2 /(joblists\.(com|net)|web-newcarer\.(com|net)|web-newcarer\.(com|net)\,Thank|gb-totaljob\.com)/i meta PRIVATE_RULE1 (__PR1 __PR2) scorePRIVATE_RULE1 2.5 \,Thank in a uri rule? you don't need the __PR1 rule untested.. this should be enough to catch the stuff - no realy need for the body strings in __PR1 rawbody RBODY_JOB_DOMAINS /\@(?:joblists|web-newcarer|gb-totaljob)\./ h2h Well, I put the Thank in there because I hoped it would catch the silly email address that they put in. More than likely the reason it failed is because of my incorrectly defined rule. Oh well, nice to keep you on your toes! Thank-you for introducing me to rawbody rules. .-) http://wiki.apache.org/spamassassin/WritingRules describe RBODY_JOB_DOMAINS English language job opportunity rawbody RBODY_JOB_DOMAINS /\@(?:joblists|web-newcarer|gb-totaljob)\./ scoreRBODY_JOB_DOMAINS 2.5 you could score those wayy higher no chance of FPs unless some dumb user forwards that to another Agreed. I intend to whack it up further once I have seen it hitting for a day. I'll know tomorrow morning because the sending pattern is very early in the morning CEST.
Quicky custom rule in local.cf question - dbg: rules: PRIVATE_PHONICA2 merged duplicates: PRIVATE_RULE1
Hi there, I've got these two rules in the local.cf. describe PRIVATE_RULE1 English language job opportunity body __PR1/(Employment opportunity|Job offer match, respond to apply|Employment you've been searching|Job opportunity|Career opportunity inside|Position opening in your area|Work offer inside|Vacancy - apply online|Job ad - see details! Sent through Search engine|Get a New Job Today|Working Part Time|Virtual Manager Vacancy)/i uri __PR2 /(\.(com|net)|\.(com|net)|\.com)/i meta PRIVATE_RULE1 (__PR1 __PR2) scorePRIVATE_RULE1 2.5 describe PRIVATE_PHONICA2 Opt-out mailing list will not honour un-subscriptions. body __PP1/(New Release Update)/i uri __PP2 /(\.com|\.com)/i meta PRIVATE_PHONICA2 (__PR1 __PR2) scorePRIVATE_PHONICA2 0.1 Spamassassin -D -lint records this: Sep 1 15:45:56.313 [11484] dbg: rules: PRIVATE_PHONICA2 merged duplicates: PRIVATE_RULE1 What is this really telling me, and why is there a connection between the two rules? I think that these don't anyway, because I have not yet seen these in a test message containing the domains nor bodies. Best wishes, Simon.
Re: Quicky custom rule in local.cf question - dbg: rules: PRIVATE_PHONICA2 merged duplicates: PRIVATE_RULE1
On 09/01/2011 04:33 PM, John Wilcock wrote: Le 01/09/2011 16:23, J4K a écrit : meta PRIVATE_RULE1 (__PR1 __PR2) ... meta PRIVATE_PHONICA2 (__PR1 __PR2) Spamassassin -D -lint records this: Sep 1 15:45:56.313 [11484] dbg: rules: PRIVATE_PHONICA2 merged duplicates: PRIVATE_RULE1 What is this really telling me, and why is there a connection between the two rules? You forgot to change the subrule names in the meta for the phonica rule... John. Thank-you John and Bowie for being my eyes. Glasses have been order in the post.
[Q] Bayes dB: ratio of spam and ham heavily in favour of ham
Afternoon gentlemen, Seems the Bayes dB has become lop-sided in favour of ham. SA is doing its job as there is little spam coming through these recently. I had hoped we could keep it one third spam and two thirds spam. Does the slant shown below (nspam verses nham) cause any problems with correct Bayes classification of newly rated messages? # sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0640 0 non-token data: nspam 0.000 0 7001 0 non-token data: nham 0.000 0 366899 0 non-token data: ntokens 0.000 0 1085514555 0 non-token data: oldest atime 0.000 0 1314019965 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 0 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count Cheers, S
Re: Self addressed spam
On 08/10/2011 12:08 PM, Marcin Mirosław wrote: W dniu 10.08.2011 12:00, akrohnke pisze: Hello, Currently one of our clients are getting spam that looks like it comes from the sender itself. Spamassassin only occasionally catches it. Hello! It should be done at smtp level. if (sender domain is my domain) and sender didn't authenticated then reject mail . How does this work on a server with 1,000 virtual domains on it?
malware.blocklist.cf : www.malware.com.br unavailable
Hi, I noticed that the site that provided the malware.blocklist.cf has been unavailable since at least the 8th of August. URL for the file was on http://www.malware.com.br/cgi/submit?action=list_sa The FQDN no longer resolves to an address. I have tried our local DNS, Level3 4.2.2.2 and Google 8.8.4.4. Its not there :( Does anyone know why they are unavailable? (Or even, is the site available to you?) Regards, S
SOLVED Re: malware.blocklist.cf : www.malware.com.br unavailable
On 08/09/2011 10:50 AM, J4K wrote: Hi, I noticed that the site that provided the malware.blocklist.cf has been unavailable since at least the 8th of August. URL for the file was on http://www.malware.com.br/cgi/submit?action=list_sa The FQDN no longer resolves to an address. I have tried our local DNS, Level3 4.2.2.2 and Google 8.8.4.4. Its not there :( Does anyone know why they are unavailable? (Or even, is the site available to you?) Regards, S Finally found that they changed their name a few months ago, and finally they turned off the .com.br site. http://www.malwarepatrol.net/ wget http://www.malwarepatrol.net//cgi/submit?action=list_sa;
[Q] Skip Bayes classification of emails with a certain From: address
Morning all, I had a route around in the SA dox, and in /usr/share/spamassassin/*.cf, to try and find out how to disable Bayes classification for particular From: address. In particular, I don't want Bayes to classify, or put into the database, messages from this mailing list. You can see that this thought was triggered by an earlier email. There are plenty of examples of how to turn on Bayes, but not to stop it from classifying certain From: addresses. Perhaps better with a Received-SPF: pass (athena.apache.org: local policy) would be better. Any clues? Regards, S
Re: improving the score for specific types of spam
On 07/13/2011 02:43 PM, Martin Gregorie wrote: On Wed, 2011-07-13 at 14:06 +0200, J4K wrote: I assume you tested it as well as running it through lint (spamassassin spam_sample.txt), so is it firing on samples of that type of spam? Comments: As written the rule won't work because __PR2 assumes that the domain name starts at the beginning of the URI but you said that the URIs typically contain a user name and '@'. Also, I'd probably generalise __PR2 to something like: uri __PR2 /(joblists.com|gb-totaljob.com)/i on the assumption that when you wrote 'europ-joblist.com' you meant 'europ-joblists.com'. This change will probably run faster and possibly catch more spam too, especially if there is a Canadian or Scandinavian office. Martin Thank-you Martin. I modified the rule as suggested. I I ran it through spamassassin test.txt, but the rule was not triggered even though the Subject was: Vacancy - apply online, and the content contained the email address: tren...@totaljoblists.net I looked with -D, and there is no mention of PRIVATE_RULE1. Odd.
60_whitelist_spf.cf - whitelist SPF
Hi everyone, Is 60_whitelist_spf.cf the correct place to whitelist SPF for a sender? def_whitelist_from_spf *.junkemailfilter.com It looks correct as per http://spamassassin.apache.org/full/3.3.x/doc/Mail_SpamAssassin_Plugin_SPF.txt Also, I have added their delivery servers to the list of trusted_networks, because they act as the secondary MX. Why am I doing this? trusted_networks because they legitimately deliver messages. I read that forwarding the Email will break SPF, so it makes sense to white-list junkemailfilter. Am I doing something wise, or about to make a gigantic blunder? Regards, S
Re: [Q] Writing rule for career opportunity type messages
On 06/29/2011 09:55 PM, Lawrence @ Rogers wrote: On 29/06/2011 4:58 PM, JKL wrote: select count(spam_count) from bayes_vars Run this query SELECT username,spam_count,ham_count FROM bayes_vars This will give a list of usernames that have been used to learn ham and spam into SpamAssassin's Bayes MySQL DB. For a site-wide installation, this should only return one result. To answer your previous question, I meant to simply add the bayes_sql_override_username setting to your local.cf and restart spamassassin If you are using Postfix with the postfix username, set it as bayes_sql_override_username postfix This ensures that all future e-mails are labeled as being learned from the postfix user, regardless of whether you did it manually using sa-learn via ssh or another interface, or auto-learning is used. For one site-wide Bayes installation, this is what you want. Regards, Lawrence Hi there, This is the table I have in mysql, and the one I intend to populate with data:- mysql describe bayes_vars; ++--+--+-+++ | Field | Type | Null | Key | Default| Extra | ++--+--+-+++ | id | int(11) | NO | PRI | NULL | auto_increment | | username | varchar(200) | NO | UNI | || | spam_count | int(11) | NO | | 0 || | ham_count | int(11) | NO | | 0 || | token_count| int(11) | NO | | 0 || | last_expire| int(11) | NO | | 0 || | last_atime_delta | int(11) | NO | | 0 || | last_expire_reduce | int(11) | NO | | 0 || | oldest_token_age | int(11) | NO | | 2147483647 || | newest_token_age | int(11) | NO | | 0 || ++--+--+-+++ 10 rows in set (0.00 sec) The configuration I intend to use for Bayes is: START local.cf --- rewrite_header Subject *SPAM* report_safe 0 report_hostname xxx.xxx.com dns_available yes use_dcc 1 dcc_path /usr/local/bin/dccproc dcc_home /var/dcc use_pyzor 1 pyzor_path /usr/bin/pyzor pyzor_timeout 5 use_razor2 1 razor_config /etc/razor/razor-agent.conf razor_timeout 5 required_score 6.0 use_bayes 1 skip_rbl_checks 1 bayes_auto_learn 0 # bayes_auto_learn_threshold_nonspam0.1 # bayes_auto_learn_threshold_spam 13.0 bayes_expiry_max_db_size30 bayes_auto_expire 1 bayes_sql_override_username postfix # I don't understand what this setting does, nor why its postfix. Postfix has no intereaction with SA in my set-up as postfix pipes the mail into dovecot,and dovecot handles the spamc portion before filing the email. |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username |shamster_user |bayes_sql_password shamster||_password| ifplugin Mail::SpamAssassin::Plugin::Shortcircuit shortcircuit USER_IN_WHITELIST on shortcircuit SUBJECT_IN_WHITELISTon shortcircuit USER_IN_BLACKLIST on shortcircuit SUBJECT_IN_BLACKLISTon loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody endif score RDNS_DYNAMIC 2.639 0.363 1.663 1.700 meta __PILL_PRICE_1 (0) meta __PILL_PRICE_2 (0) meta __PILL_PRICE_3 (0) END local.cf --- N.B Yes, I know there are some custom rules in the local.cf and these'll be lost after an upgrade of SA, but I have reasonable backups. * Questions Does the configuration above look correct? Will SA only write into the table bayes_vars, or will it touch other tables?
Re: [Q] Writing rule for career opportunity type messages
On 06/30/2011 11:09 AM, J4K wrote: On 06/29/2011 09:55 PM, Lawrence @ Rogers wrote: On 29/06/2011 4:58 PM, JKL wrote: select count(spam_count) from bayes_vars Run this query SELECT username,spam_count,ham_count FROM bayes_vars This will give a list of usernames that have been used to learn ham and spam into SpamAssassin's Bayes MySQL DB. For a site-wide installation, this should only return one result. To answer your previous question, I meant to simply add the bayes_sql_override_username setting to your local.cf and restart spamassassin If you are using Postfix with the postfix username, set it as bayes_sql_override_username postfix This ensures that all future e-mails are labeled as being learned from the postfix user, regardless of whether you did it manually using sa-learn via ssh or another interface, or auto-learning is used. For one site-wide Bayes installation, this is what you want. Regards, Lawrence Hi there, This is the table I have in mysql, and the one I intend to populate with data:- mysql describe bayes_vars; ++--+--+-+++ | Field | Type | Null | Key | Default| Extra | ++--+--+-+++ | id | int(11) | NO | PRI | NULL | auto_increment | | username | varchar(200) | NO | UNI | || | spam_count | int(11) | NO | | 0 || | ham_count | int(11) | NO | | 0 || | token_count| int(11) | NO | | 0 || | last_expire| int(11) | NO | | 0 || | last_atime_delta | int(11) | NO | | 0 || | last_expire_reduce | int(11) | NO | | 0 || | oldest_token_age | int(11) | NO | | 2147483647 || | newest_token_age | int(11) | NO | | 0 || ++--+--+-+++ 10 rows in set (0.00 sec) The configuration I intend to use for Bayes is: START local.cf --- rewrite_header Subject *SPAM* report_safe 0 report_hostname xxx.xxx.com dns_available yes use_dcc 1 dcc_path /usr/local/bin/dccproc dcc_home /var/dcc use_pyzor 1 pyzor_path /usr/bin/pyzor pyzor_timeout 5 use_razor2 1 razor_config /etc/razor/razor-agent.conf razor_timeout 5 required_score 6.0 use_bayes 1 skip_rbl_checks 1 bayes_auto_learn 0 # bayes_auto_learn_threshold_nonspam0.1 # bayes_auto_learn_threshold_spam 13.0 bayes_expiry_max_db_size30 bayes_auto_expire 1 bayes_sql_override_username postfix # I don't understand what this setting does, nor why its postfix. Postfix has no intereaction with SA in my set-up as postfix pipes the mail into dovecot,and dovecot handles the spamc portion before filing the email. |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username |shamster_user |bayes_sql_password shamster||_password| ifplugin Mail::SpamAssassin::Plugin::Shortcircuit shortcircuit USER_IN_WHITELIST on shortcircuit SUBJECT_IN_WHITELISTon shortcircuit USER_IN_BLACKLIST on shortcircuit SUBJECT_IN_BLACKLISTon loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody endif score RDNS_DYNAMIC 2.639 0.363 1.663 1.700 meta __PILL_PRICE_1 (0) meta __PILL_PRICE_2 (0) meta __PILL_PRICE_3 (0) END local.cf --- N.B Yes, I know there are some custom rules in the local.cf and these'll be lost after an upgrade of SA, but I have reasonable backups. * Questions Does the configuration above look correct? Will SA only write into the table bayes_vars, or will it touch other tables? Seems that some process butchered part of the config by discovering some pipe characters. |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username |shamster_user |bayes_sql_password shamster||_password| Above should have read: |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username sa_user bayes_sql_password sa_user_password| Other question: If the above looks correct, is that somethin else that I ought to enable? e.g plugins for mysql, or a particular perl module that I might have omitted? Regards, S.
Re: [Q] Writing rule for career opportunity type messages
On 06/30/2011 11:38 AM, Benny Pedersen wrote: On Thu, 30 Jun 2011 11:09:18 +0200, J4K wrote: loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody should not be in a cf file but in a pre file, check other pre files to enable it Thank-you. Moved this into v320.pre
Re: [Q] Writing rule for career opportunity type messages
On 06/30/2011 11:37 AM, J4K wrote: On 06/30/2011 11:09 AM, J4K wrote: On 06/29/2011 09:55 PM, Lawrence @ Rogers wrote: On 29/06/2011 4:58 PM, JKL wrote: select count(spam_count) from bayes_vars Run this query SELECT username,spam_count,ham_count FROM bayes_vars This will give a list of usernames that have been used to learn ham and spam into SpamAssassin's Bayes MySQL DB. For a site-wide installation, this should only return one result. To answer your previous question, I meant to simply add the bayes_sql_override_username setting to your local.cf and restart spamassassin If you are using Postfix with the postfix username, set it as bayes_sql_override_username postfix This ensures that all future e-mails are labeled as being learned from the postfix user, regardless of whether you did it manually using sa-learn via ssh or another interface, or auto-learning is used. For one site-wide Bayes installation, this is what you want. Regards, Lawrence Hi there, This is the table I have in mysql, and the one I intend to populate with data:- mysql describe bayes_vars; ++--+--+-+++ | Field | Type | Null | Key | Default| Extra | ++--+--+-+++ | id | int(11) | NO | PRI | NULL | auto_increment | | username | varchar(200) | NO | UNI | || | spam_count | int(11) | NO | | 0 || | ham_count | int(11) | NO | | 0 || | token_count| int(11) | NO | | 0 || | last_expire| int(11) | NO | | 0 || | last_atime_delta | int(11) | NO | | 0 || | last_expire_reduce | int(11) | NO | | 0 || | oldest_token_age | int(11) | NO | | 2147483647 || | newest_token_age | int(11) | NO | | 0 || ++--+--+-+++ 10 rows in set (0.00 sec) The configuration I intend to use for Bayes is: START local.cf --- rewrite_header Subject *SPAM* report_safe 0 report_hostname xxx.xxx.com dns_available yes use_dcc 1 dcc_path /usr/local/bin/dccproc dcc_home /var/dcc use_pyzor 1 pyzor_path /usr/bin/pyzor pyzor_timeout 5 use_razor2 1 razor_config /etc/razor/razor-agent.conf razor_timeout 5 required_score 6.0 use_bayes 1 skip_rbl_checks 1 bayes_auto_learn 0 # bayes_auto_learn_threshold_nonspam0.1 # bayes_auto_learn_threshold_spam 13.0 bayes_expiry_max_db_size30 bayes_auto_expire 1 bayes_sql_override_username postfix # I don't understand what this setting does, nor why its postfix. Postfix has no intereaction with SA in my set-up as postfix pipes the mail into dovecot,and dovecot handles the spamc portion before filing the email. |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username |shamster_user |bayes_sql_password shamster||_password| ifplugin Mail::SpamAssassin::Plugin::Shortcircuit shortcircuit USER_IN_WHITELIST on shortcircuit SUBJECT_IN_WHITELISTon shortcircuit USER_IN_BLACKLIST on shortcircuit SUBJECT_IN_BLACKLISTon loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody endif score RDNS_DYNAMIC 2.639 0.363 1.663 1.700 meta __PILL_PRICE_1 (0) meta __PILL_PRICE_2 (0) meta __PILL_PRICE_3 (0) END local.cf --- N.B Yes, I know there are some custom rules in the local.cf and these'll be lost after an upgrade of SA, but I have reasonable backups. * Questions Does the configuration above look correct? Will SA only write into the table bayes_vars, or will it touch other tables? Seems that some process butchered part of the config by discovering some pipe characters. |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username |shamster_user |bayes_sql_password shamster||_password| Above should have read: |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username sa_user bayes_sql_password sa_user_password| Other question: If the above looks correct, is that somethin else that I ought to enable? e.g plugins for mysql, or a particular perl module that I might have omitted? Regards, S. Regarding
Now: Bayes with mysql Was: [Q] Writing rule for career opportunity type messages
On 06/30/2011 01:45 PM, J4K wrote: On 06/30/2011 11:37 AM, J4K wrote: On 06/30/2011 11:09 AM, J4K wrote: On 06/29/2011 09:55 PM, Lawrence @ Rogers wrote: On 29/06/2011 4:58 PM, JKL wrote: select count(spam_count) from bayes_vars Run this query SELECT username,spam_count,ham_count FROM bayes_vars This will give a list of usernames that have been used to learn ham and spam into SpamAssassin's Bayes MySQL DB. For a site-wide installation, this should only return one result. To answer your previous question, I meant to simply add the bayes_sql_override_username setting to your local.cf and restart spamassassin If you are using Postfix with the postfix username, set it as bayes_sql_override_username postfix This ensures that all future e-mails are labeled as being learned from the postfix user, regardless of whether you did it manually using sa-learn via ssh or another interface, or auto-learning is used. For one site-wide Bayes installation, this is what you want. Regards, Lawrence Hi there, This is the table I have in mysql, and the one I intend to populate with data:- mysql describe bayes_vars; ++--+--+-+++ | Field | Type | Null | Key | Default| Extra | ++--+--+-+++ | id | int(11) | NO | PRI | NULL | auto_increment | | username | varchar(200) | NO | UNI | || | spam_count | int(11) | NO | | 0 || | ham_count | int(11) | NO | | 0 || | token_count| int(11) | NO | | 0 || | last_expire| int(11) | NO | | 0 || | last_atime_delta | int(11) | NO | | 0 || | last_expire_reduce | int(11) | NO | | 0 || | oldest_token_age | int(11) | NO | | 2147483647 || | newest_token_age | int(11) | NO | | 0 || ++--+--+-+++ 10 rows in set (0.00 sec) The configuration I intend to use for Bayes is: START local.cf --- rewrite_header Subject *SPAM* report_safe 0 report_hostname xxx.xxx.com dns_available yes use_dcc 1 dcc_path /usr/local/bin/dccproc dcc_home /var/dcc use_pyzor 1 pyzor_path /usr/bin/pyzor pyzor_timeout 5 use_razor2 1 razor_config /etc/razor/razor-agent.conf razor_timeout 5 required_score 6.0 use_bayes 1 skip_rbl_checks 1 bayes_auto_learn 0 # bayes_auto_learn_threshold_nonspam0.1 # bayes_auto_learn_threshold_spam 13.0 bayes_expiry_max_db_size30 bayes_auto_expire 1 bayes_sql_override_username postfix # I don't understand what this setting does, nor why its postfix. Postfix has no intereaction with SA in my set-up as postfix pipes the mail into dovecot,and dovecot handles the spamc portion before filing the email. |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username |shamster_user |bayes_sql_password shamster||_password| ifplugin Mail::SpamAssassin::Plugin::Shortcircuit shortcircuit USER_IN_WHITELIST on shortcircuit SUBJECT_IN_WHITELISTon shortcircuit USER_IN_BLACKLIST on shortcircuit SUBJECT_IN_BLACKLISTon loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody endif score RDNS_DYNAMIC 2.639 0.363 1.663 1.700 meta __PILL_PRICE_1 (0) meta __PILL_PRICE_2 (0) meta __PILL_PRICE_3 (0) END local.cf --- N.B Yes, I know there are some custom rules in the local.cf and these'll be lost after an upgrade of SA, but I have reasonable backups. * Questions Does the configuration above look correct? Will SA only write into the table bayes_vars, or will it touch other tables? Seems that some process butchered part of the config by discovering some pipe characters. |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username |shamster_user |bayes_sql_password shamster||_password| Above should have read: |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username sa_user bayes_sql_password sa_user_password| Other question: If the above looks correct, is that somethin else that I ought to enable? e.g plugins for mysql, or a particular perl module that I might
Re: [Q] Writing rule for career opportunity type messages
On 06/30/2011 01:45 PM, J4K wrote: On 06/30/2011 11:37 AM, J4K wrote: On 06/30/2011 11:09 AM, J4K wrote: On 06/29/2011 09:55 PM, Lawrence @ Rogers wrote: On 29/06/2011 4:58 PM, JKL wrote: select count(spam_count) from bayes_vars Run this query SELECT username,spam_count,ham_count FROM bayes_vars This will give a list of usernames that have been used to learn ham and spam into SpamAssassin's Bayes MySQL DB. For a site-wide installation, this should only return one result. To answer your previous question, I meant to simply add the bayes_sql_override_username setting to your local.cf and restart spamassassin If you are using Postfix with the postfix username, set it as bayes_sql_override_username postfix This ensures that all future e-mails are labeled as being learned from the postfix user, regardless of whether you did it manually using sa-learn via ssh or another interface, or auto-learning is used. For one site-wide Bayes installation, this is what you want. Regards, Lawrence Hi there, This is the table I have in mysql, and the one I intend to populate with data:- mysql describe bayes_vars; ++--+--+-+++ | Field | Type | Null | Key | Default| Extra | ++--+--+-+++ | id | int(11) | NO | PRI | NULL | auto_increment | | username | varchar(200) | NO | UNI | || | spam_count | int(11) | NO | | 0 || | ham_count | int(11) | NO | | 0 || | token_count| int(11) | NO | | 0 || | last_expire| int(11) | NO | | 0 || | last_atime_delta | int(11) | NO | | 0 || | last_expire_reduce | int(11) | NO | | 0 || | oldest_token_age | int(11) | NO | | 2147483647 || | newest_token_age | int(11) | NO | | 0 || ++--+--+-+++ 10 rows in set (0.00 sec) The configuration I intend to use for Bayes is: START local.cf --- rewrite_header Subject *SPAM* report_safe 0 report_hostname xxx.xxx.com dns_available yes use_dcc 1 dcc_path /usr/local/bin/dccproc dcc_home /var/dcc use_pyzor 1 pyzor_path /usr/bin/pyzor pyzor_timeout 5 use_razor2 1 razor_config /etc/razor/razor-agent.conf razor_timeout 5 required_score 6.0 use_bayes 1 skip_rbl_checks 1 bayes_auto_learn 0 # bayes_auto_learn_threshold_nonspam0.1 # bayes_auto_learn_threshold_spam 13.0 bayes_expiry_max_db_size30 bayes_auto_expire 1 bayes_sql_override_username postfix # I don't understand what this setting does, nor why its postfix. Postfix has no intereaction with SA in my set-up as postfix pipes the mail into dovecot,and dovecot handles the spamc portion before filing the email. |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username |shamster_user |bayes_sql_password shamster||_password| ifplugin Mail::SpamAssassin::Plugin::Shortcircuit shortcircuit USER_IN_WHITELIST on shortcircuit SUBJECT_IN_WHITELISTon shortcircuit USER_IN_BLACKLIST on shortcircuit SUBJECT_IN_BLACKLISTon loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody endif score RDNS_DYNAMIC 2.639 0.363 1.663 1.700 meta __PILL_PRICE_1 (0) meta __PILL_PRICE_2 (0) meta __PILL_PRICE_3 (0) END local.cf --- N.B Yes, I know there are some custom rules in the local.cf and these'll be lost after an upgrade of SA, but I have reasonable backups. * Questions Does the configuration above look correct? Will SA only write into the table bayes_vars, or will it touch other tables? Seems that some process butchered part of the config by discovering some pipe characters. |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username |shamster_user |bayes_sql_password shamster||_password| Above should have read: |bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:spamassassin:localhost bayes_sql_username sa_user bayes_sql_password sa_user_password| Other question: If the above looks correct, is that somethin else that I ought to enable? e.g plugins for mysql, or a particular perl module that I might
Re: Bayes and mysql Was: [Q] Writing rule for career opportunity type messages
[SNIP]
Hi there,
This is the table I have in mysql, and the one I intend to populate with
data:-
mysql describe bayes_vars;
++--+--+-+++
| Field | Type | Null | Key | Default|
Extra |
++--+--+-+++
| id | int(11) | NO | PRI | NULL |
auto_increment |
| username | varchar(200) | NO | UNI |
||
| spam_count | int(11) | NO | | 0
||
| ham_count | int(11) | NO | | 0
||
| token_count| int(11) | NO | | 0
||
| last_expire| int(11) | NO | | 0
||
| last_atime_delta | int(11) | NO | | 0
||
| last_expire_reduce | int(11) | NO | | 0
||
| oldest_token_age | int(11) | NO | | 2147483647
||
| newest_token_age | int(11) | NO | | 0
||
++--+--+-+++
10 rows in set (0.00 sec)
The configuration I intend to use for Bayes is:
START local.cf ---
rewrite_header Subject *SPAM*
report_safe 0
report_hostname xxx.xxx.com
dns_available yes
use_dcc 1
dcc_path /usr/local/bin/dccproc
dcc_home /var/dcc
use_pyzor 1
pyzor_path /usr/bin/pyzor
pyzor_timeout 5
use_razor2 1
razor_config /etc/razor/razor-agent.conf
razor_timeout 5
required_score 6.0
use_bayes 1
skip_rbl_checks 1
bayes_auto_learn 0
# bayes_auto_learn_threshold_nonspam0.1
# bayes_auto_learn_threshold_spam 13.0
bayes_expiry_max_db_size30
bayes_auto_expire 1
bayes_sql_override_username postfix
# I don't understand what this setting does, nor why its postfix.
Postfix has no intereaction with SA in my set-up as postfix pipes the
mail into dovecot,and dovecot handles the spamc portion before filing
the email.
|bayes_store_module Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn DBI:mysql:spamassassin:localhost
bayes_sql_username |shamster_user
|bayes_sql_password shamster||_password|
ifplugin Mail::SpamAssassin::Plugin::Shortcircuit
shortcircuit USER_IN_WHITELIST on
shortcircuit SUBJECT_IN_WHITELISTon
shortcircuit USER_IN_BLACKLIST on
shortcircuit SUBJECT_IN_BLACKLISTon
loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody
endif
score RDNS_DYNAMIC 2.639 0.363 1.663 1.700
meta __PILL_PRICE_1 (0)
meta __PILL_PRICE_2 (0)
meta __PILL_PRICE_3 (0)
END local.cf ---
N.B Yes, I know there are some custom rules in the local.cf and these'll
be lost after an upgrade of SA, but I have reasonable backups.
* Questions
Does the configuration above look correct?
Will SA only write into the table bayes_vars, or will it touch other
tables?
Seems that some process butchered part of the config by discovering some
pipe characters.
[SNIP]
Other question: If the above looks correct, is that somethin else that I
ought to enable? e.g plugins for mysql, or a particular perl module
that I might have omitted?
Regards, S.
Regarding local.cf
Should the password be quoted such as in single quotes?
The password has many strange chars in it e.g
bayes_sql_passwordfg$%-)_()(Wsuisrt{^%TEST
RTFM problem... Apologies.
Jun 30 16:10:11.628 [2220] dbg: bayes: found bayes db version 3
Jun 30 16:10:11.628 [2220] dbg: bayes: Using userid: 186
Jun 30 16:10:11.628 [2220] dbg: bayes: not available for scanning,
only 0 spam(s) in bayes DB 200
Solved by feeding one piece of spam to init the database:
sa-learn --spam gtube.txt
However, I added some messages, but the detail from --dump magic shows
nothing:
# sa-learn --ham cur/
Learned tokens from 25 message(s) (26 message(s) examined)
# sa-learn --dump magic
0.000 0 3 0 non-token data: bayes db version
0.000 0 0 0 non-token data: nspam
0.000 0 0 0 non-token data: nham
0.000 0 0 0 non-token data: ntokens
0.000 0 2147483647 0 non-token data: oldest atime
0.000 0 0 0 non-token data: newest atime
0.000 0 0 0 non-token data: last journal
sync atime
0.000 0 0 0 non-token data: last expiry atime
0.000 0 0 0 non-token data: last expire
atime delta
0.000 0 0 0 non-token data: last expire
reduction count
I checked if the
Re: Bayes and mysql Was: [Q] Writing rule for career opportunity type messages
On 06/30/2011 05:27 PM, Axb wrote:
spamassassin --lint -D bayes
Hi Axb,
Spamd runs as root.
# spamassassin --lint -D bayes
Jun 30 17:32:10.858 [2775] dbg: FuzzyOcr: focr_bin_helper:
'pnmnorm,pnminvert,ppmtopgm'
Jun 30 17:32:10.858 [2775] info: FuzzyOcr: Adding 3 new helper apps
Jun 30 17:32:10.858 [2775] dbg: FuzzyOcr: focr_bin_helper: 'tesseract'
Jun 30 17:32:10.858 [2775] info: FuzzyOcr: Adding 1 new helper apps
Jun 30 17:32:10.859 [2775] info: FuzzyOcr: Starting preprocessor parser
for file /etc/mail/spamassassin/FuzzyOcr.preps...
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: preprocessor normalize {
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: command = pnmnorm
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: }
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: preprocessor invert {
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: command = pnminvert
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: }
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: preprocessor ppmtopgm {
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: command = ppmtopgm
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: }
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: preprocessor maketiff {
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: command = pnmtotiff
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: args = -color -truecolor
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line: }
Jun 30 17:32:10.860 [2775] info: FuzzyOcr: Starting scanset parser for
file /etc/mail/spamassassin/FuzzyOcr.scansets...
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line scanset ocrad {
Jun 30 17:32:10.860 [2775] dbg: FuzzyOcr: line command = $ocrad
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line args = -s5 $input
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line }
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line scanset ocrad-invert {
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line command = $ocrad
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line args = -s5 -i $input
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line }
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line scanset
ocrad-decolorize-invert {
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line preprocessors = ppmtopgm
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line command = $ocrad
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line args = -s5 -i $input
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line }
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line scanset ocrad-decolorize {
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line preprocessors = ppmtopgm
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line command = $ocrad
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line args = -s5 $input
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line }
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line scanset gocr {
Jun 30 17:32:10.861 [2775] dbg: FuzzyOcr: line command = $gocr
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line args = -i $input
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line }
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line scanset gocr-180 {
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line command = $gocr
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line args = -l 180 -d 2 -i $input
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line }
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line scanset tesseract {
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line preprocessors = maketiff
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line command = $tesseract
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line args = $input $output
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line force_output_in = $output.txt
Jun 30 17:32:10.862 [2775] dbg: FuzzyOcr: line }
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Searching in:
/usr/local/netpbm/bin
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Searching in: /usr/local/bin
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Searching in: /usr/bin
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Using gifsicle =
/usr/bin/gifsicle
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Using giffix = /usr/bin/giffix
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Using giftext = /usr/bin/giftext
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Using gifinter =
/usr/bin/gifinter
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Using giftopnm =
/usr/bin/giftopnm
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Using jpegtopnm =
/usr/bin/jpegtopnm
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Using pngtopnm =
/usr/bin/pngtopnm
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Using bmptopnm =
/usr/bin/bmptopnm
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Using tifftopnm =
/usr/bin/tifftopnm
Jun 30 17:32:12.697 [2775] info: FuzzyOcr: Using ppmhist = /usr/bin/ppmhist
Jun 30 17:32:12.698 [2775] info: FuzzyOcr: Using pamfile = /usr/bin/pamfile
Jun 30 17:32:12.698 [2775] info: FuzzyOcr: Using ocrad = /usr/bin/ocrad
Jun 30 17:32:12.698 [2775] info: FuzzyOcr: Using gocr = /usr/bin/gocr
Jun 30 17:32:12.698 [2775] info: FuzzyOcr: Using pnmnorm = /usr/bin/pnmnorm
Jun 30 17:32:12.698 [2775] info: FuzzyOcr: Using pnminvert =
/usr/bin/pnminvert
Jun 30 17:32:12.698 [2775] info: FuzzyOcr: Using ppmtopgm =
/usr/bin/ppmtopgm
Jun 30
Re: Bayes and mysql Was: [Q] Writing rule for career opportunity type messages
On 06/30/2011 05:54 PM, Axb wrote: ok.. you said Spamd runs as root. in that case: bayes_sql_override_username spamd then as per Bowie: sa-learn --username=spamd --ham /path/to/ham sa-learn --username=spamd --spam /path/to/spam then sa-learn --dump magic Ahh, I meant that spamd was started as root. spamd is running with --username=spamd, and the childs all drop down to this UID. Apologies for the confusion. # sa-learn --username=spamd --ham .HAM/cur/ Learned tokens from 717 message(s) (764 message(s) examined) # sa-learn --username=spamd --spam .Junk/cur Learned tokens from 311 message(s) (368 message(s) examined) # sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 0 0 non-token data: nspam 0.000 0 0 0 non-token data: nham 0.000 0 0 0 non-token data: ntokens 0.000 0 2147483647 0 non-token data: oldest atime 0.000 0 0 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 0 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count
Re: Bayes and mysql Was: [Q] Writing rule for career opportunity type messages
On 06/30/2011 06:02 PM, Axb wrote: Please only reply to the list... On 2011-06-30 18:00, J4K wrote: # sa-learn --username=spamd --ham .HAM/cur/ Learned tokens from 717 message(s) (764 message(s) examined) # sa-learn --username=spamd --spam .Junk/cur Learned tokens from 311 message(s) (368 message(s) examined) pls run sa-learn -D --username=spamd --spam .Junk/cur and post the output Found one problem: Jun 30 18:05:03.559 [3091] dbg: bayes: _put_tokens: SQL error: UPDATE command denied to user 'xxx'@'localhost' for table 'bayes_token' Added update privs for the mysql user onto the table. Re-ran the sa-learn (with username). I have attached the debug output as a file. [sa-learn.txt] # sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 0 0 non-token data: nspam 0.000 0 0 0 non-token data: nham 0.000 0 0 0 non-token data: ntokens 0.000 0 2147483647 0 non-token data: oldest atime 0.000 0 0 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 0 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count sa-learn -D --username=spamd --ham cur Jun 30 18:12:11.327 [3156] dbg: logger: adding facilities: all Jun 30 18:12:11.327 [3156] dbg: logger: logging level is DBG Jun 30 18:12:11.327 [3156] dbg: generic: SpamAssassin version 3.3.1 Jun 30 18:12:11.327 [3156] dbg: generic: Perl 5.010001, PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/spamassassin, LOCAL_STATE_DIR=/var/lib/spamassassin Jun 30 18:12:11.327 [3156] dbg: config: timing enabled Jun 30 18:12:11.328 [3156] dbg: config: score set 0 chosen. Jun 30 18:12:11.329 [3156] dbg: util: running in taint mode? yes Jun 30 18:12:11.329 [3156] dbg: util: taint mode: deleting unsafe environment variables, resetting PATH Jun 30 18:12:11.329 [3156] dbg: util: PATH included '/usr/local/sbin', keeping Jun 30 18:12:11.329 [3156] dbg: util: PATH included '/usr/local/bin', keeping Jun 30 18:12:11.329 [3156] dbg: util: PATH included '/usr/sbin', keeping Jun 30 18:12:11.330 [3156] dbg: util: PATH included '/usr/bin', keeping Jun 30 18:12:11.330 [3156] dbg: util: PATH included '/sbin', keeping Jun 30 18:12:11.330 [3156] dbg: util: PATH included '/bin', keeping Jun 30 18:12:11.330 [3156] dbg: util: final PATH set to: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin Jun 30 18:12:11.333 [3156] dbg: dns: is Net::DNS::Resolver available? yes Jun 30 18:12:11.333 [3156] dbg: dns: Net::DNS version: 0.66 Jun 30 18:12:11.334 [3156] dbg: config: using /etc/spamassassin for site rules pre files Jun 30 18:12:11.334 [3156] dbg: config: read file /etc/spamassassin/init.pre Jun 30 18:12:11.334 [3156] dbg: config: read file /etc/spamassassin/v310.pre Jun 30 18:12:11.334 [3156] dbg: config: read file /etc/spamassassin/v312.pre Jun 30 18:12:11.334 [3156] dbg: config: read file /etc/spamassassin/v320.pre Jun 30 18:12:11.334 [3156] dbg: config: read file /etc/spamassassin/v330.pre Jun 30 18:12:11.334 [3156] dbg: config: using /var/lib/spamassassin/3.003001 for sys rules pre files Jun 30 18:12:11.334 [3156] dbg: config: using /var/lib/spamassassin/3.003001 for default rules dir Jun 30 18:12:11.335 [3156] dbg: config: read file /var/lib/spamassassin/3.003001/updates_spamassassin_org.cf Jun 30 18:12:11.335 [3156] dbg: config: using /etc/spamassassin for site rules dir Jun 30 18:12:11.335 [3156] dbg: config: read file /etc/spamassassin/65_debian.cf Jun 30 18:12:11.335 [3156] dbg: config: read file /etc/spamassassin/FuzzyOcr.cf Jun 30 18:12:11.335 [3156] dbg: config: read file /etc/spamassassin/clamav.cf Jun 30 18:12:11.335 [3156] dbg: config: read file /etc/spamassassin/local.cf Jun 30 18:12:11.337 [3156] dbg: config: read file /etc/spamassassin/malware.blocklist.cf Jun 30 18:12:11.338 [3156] dbg: config: read file /etc/spamassassin/sql.cf Jun 30 18:12:11.338 [3156] dbg: config: using /root/.spamassassin/user_prefs for user prefs file Jun 30 18:12:11.338 [3156] dbg: config: read file /root/.spamassassin/user_prefs Jun 30 18:12:11.351 [3156] dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC Jun 30 18:12:11.355 [3156] dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC Jun 30 18:12:11.362 [3156] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC Jun 30 18:12:11.365 [3156] dbg: plugin: loading Mail::SpamAssassin::Plugin::RelayCountry from @INC Jun 30 18:12:11.366 [3156] dbg: plugin: loading Mail::SpamAssassin::Plugin::DCC from @INC Jun 30 18:12:11.370 [3156] dbg: dcc: network tests on, registering DCC Jun 30 18:12:11.371 [3156] dbg: plugin: loading Mail::SpamAssassin
[Q] Writing rule for career opportunity type messages
Dear all, Over the past few months I noticed an increase in 'Start New Employment Today | Career Opportunity' style email. The rules I use, that are pretty much stock rules, correctly tag the email as spam. Usually the Spam score hovers between 5.5 and 6.9. I would like to add a rule that adds more points to these specific messages. I do not really want to increase the scoring for the current rules it triggers, as it affects other spam that hits these rules. [I don't know if this is good or bad, so my logic might be flawed.] I would rather target the actual messages content. Alternatively, I could lower the spam reject threshold on spamass-milter, but this is a sledgehammer action, but then where would I stop and give up. Are there any rules around that target these particular types of emails? An example of one such message, and the current spam reports follows. Otherwise, what are the chances of me writing my own scoring that targets these types of messages? Best regards, S. X-Spam-Status: Yes, score=6.1 required=5.0 tests=SPF_SOFTFAIL, T_URIBL_BLACK_OVERLAP,UNPARSEABLE_RELAY,URIBL_BLACK,URIBL_DBL_SPAM, URIBL_WS_SURBL shortcircuit=no autolearn=no version=3.3.1 X-Spam-Report: * 1.7 URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist * [URIs: europe-hire.net] * 1.7 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist * [URIs: europe-hire.net] * 1.8 URIBL_BLACK Contains an URL listed in the URIBL blacklist * [URIs: europe-hire.net] * 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) * 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines * 0.0 T_URIBL_BLACK_OVERLAP T_URIBL_BLACK_OVERLAP Good afternoon! I'm willing to introduce myself as a Human Resources manager of one of the leading investment companies. This company is connected with different areas of activity, such as: * real estate * logistics * private undertaking service * etc. At the present time we have vacancies to be filled by European residents only: - payment 2300 ˆ +bonus - part-time job - free timetable If you find this interesting, we look forward to getting to know you and kindly ask you to provide us your contact details. br...@europe-hire.net Attention! We need just the people residing in Europe. If you meet our requirement we would love to work with you.
[Q] SA mis-configuration question: Empty dirs username being created in /var/mail
Dear everyone,
I wonder if I have some misconfiguration in my SA set-up, and if
someone would be able to send me off in the right direction, or has seen
it before. As shown below, some messages are scanned by SA with just
the user name, while others are scanned using the full login.
Fri Mar 11 11:28:56 2011 [2252] info: spamd: clean message (0.0/6.0) for
bob.pieters:5002 in 1.2 seconds, 22828 bytes.
Fri Mar 11 11:28:57 2011 [2253] info: spamd: clean message (-0.0/5.0)
for bob.piet...@klunky.co.uk:5002 in 1.1 seconds, 22390 bytes.
Fri Mar 11 11:40:59 2011 [2252] info: spamd: clean message (0.0/6.0) for
bob.pieters:5002 in 1.4 seconds, 51665 bytes.
Fri Mar 11 11:41:00 2011 [2253] info: spamd: clean message (0.0/5.0) for
bob.piet...@klunky.co.uk:5002 in 1.1 seconds, 51214 bytes.
Fri Mar 11 11:51:30 2011 [2252] info: spamd: clean message (1.1/6.0) for
bob.pieters:5002 in 6.3 seconds, 11829 bytes.
Fri Mar 11 11:58:05 2011 [2252] info: spamd: clean message (0.0/6.0) for
bob.pieters:5002 in 1.3 seconds, 28802 bytes.
Fri Mar 11 11:58:07 2011 [2253] info: spamd: clean message (-0.0/5.0)
for bob.piet...@klunky.co.uk:5002 in 1.2 seconds, 28198 byt
SA runs with these options (taken from the /etc/default/spamassassin
file on Debian Squeeze):-
OPTIONS=--create-prefs -x -q --max-children 3 --sql-config
--nouser-config --username spamd --helper-home-dir ${SAHOME} -s
/var/log/spamd.log --virtual-config-dir=${SAHOME}/users/%d/%u
I noticed that erroneous directories are created via dovecot in
/var/vmail like:
/var/vmail/bob.piet...@klunky.co.uk/
/var/vmail/bob.pieters/
however, all directories created sans the domain name (user name only)
remain empty. I have deleted these now and again, and eventually these
are re-created. I imagine that its dovecot that is responsible for
creating these directories, but is it SA that is passing the wrong email
address? SA runs as a milter, and as a pipe from postfix into dovecot:
# grep spam /etc/postfix/master.cf
dovecot-spamass unix - n n - - pipe
flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -u ${recipient} -e
/usr/lib/dovecot/deliver -d ${recipient}
Does anyone know why this would happen. I've noticed it for quite
sometime, but never got around to sorting it out. I dpn't know if this
is really a problem with my SA set-up, or a postfix or dovecot
mis-configuration. Apologies for one of those long-winded questions on
a Friday afternoon.
Best wishes, S.
Re: Should Emails Have An Expiration Date
On 03/01/2011 04:13 PM, Jari Fredriksson wrote: On 1.3.2011 0:06, Matt wrote: Looking at top 8 newest messages from my personnel email account: [Spammy subjects deleted] None of them are SPAM. If I wanted to unsubscribe from them I would. Its just if I do not read them within 30 days why keep them. I want to keep those for 3 months for SpamAssassin auto-mass-check corpus. That is a decision by me, not the sender. My script will automatically delete them when they reach the desired age. Some more points of view: If someone sends me an Email, then its kept according to my policy, not theirs. If they did not wish to adhere to my policy then they should have contacted me before they send the particular Email to me. Next, someone will ask for a X-Do-Not-Delete-Before-Date: header. Whatever next! Regards,s -- I won't accept your confidentiality agreement, and your Emails are kept. --
[Q] Adjusting Rule Scores - Which file?
Hi, I am interested in raising the score for the rule RDNS_DYNAMIC. However, I cannot find it in any of the files under /etc/spamassassin. I thought that it would be listed somewhere in this directory. In which file is this located? * Why do I want to raise the bar for RDNS_DYNAMIC? I won't use the pbl from spamhaus, nor other expressions that reject email sent from dynamic address ranges. I think if someone [hobbyist] wishes to run his own server and not pay for a static address, then I won't block email from them. However, I would like to add some extra points for sending from one. 99% of spam received here from the dynamic ranges get blocked by postfix for numerous reasons; Some being an invalid rcpt email, some Subjects, RFC compliance, and so on). Regards, S
Re: [Q] Adjusting Rule Scores - Which file?
On 02/17/2011 04:45 PM, John Hardin wrote: On Thu, 17 Feb 2011, J4K wrote: You do not want to alter the distributed files, as any alterations would be lost on the next upgrade. That rule doesn't appear in any of your local customization files (under /etc/spamassassin) because you've never customized it before. Just put score RDNS_DYNAMIC 1.00 into your local spamassassin .cf file under /etc/spamassassin and restart the daemon. Thank-you everyone for telling me. Is 1.00 higher than the default. How could I list the default?
Re: [Q] Adjusting Rule Scores - Which file?
On 02/17/2011 05:33 PM, Bowie Bailey wrote: On 2/17/2011 10:51 AM, J4K wrote: On 02/17/2011 04:45 PM, John Hardin wrote: On Thu, 17 Feb 2011, J4K wrote: You do not want to alter the distributed files, as any alterations would be lost on the next upgrade. That rule doesn't appear in any of your local customization files (under /etc/spamassassin) because you've never customized it before. Just put score RDNS_DYNAMIC 1.00 into your local spamassassin .cf file under /etc/spamassassin and restart the daemon. Thank-you everyone for telling me. Is 1.00 higher than the default. How could I list the default? $ grep RDNS_DYNAMIC /var/lib/spamassassin/3.003001/updates_spamassassin_org/50_scores.cf score RDNS_DYNAMIC 2.639 0.363 1.663 0.982 If you are using Bayes and Network tests, then the last number is what you want. So the default (for SA 3.3.1) is 0.982. Take a look at the man page for Mail::SpamAssassin::Conf for more info on the scores. Great. Thanks again. I shall play around with the scores for this.
Re: Irony
Not a chance. Philip Prindeville philipp_s...@redfish-solutions.com wrote: On 2/7/11 1:28 AM, Matus UHLAR - fantomas wrote: On Tue, 1 Feb 2011 09:49:36 -0500 Michael Scheidellmichael.scheid...@secnap.com wrote: because HELO doesn't match RDNS. On 01.02.11 09:54, David F. Skoll wrote: Rejecting on that basis would also cause tons of false-positives. It's also violation of all SMTP RFCs (former and current), because they explicitly say that the sender MUST NOT reject smtp session just because HELO string does not match resolved FQDN. Does anyone else reject messages where the rDNS maps to more than one PTR record?
[Q] sa-compile: not compiling; 'spamassassin --lint' check failed!
Hi chaps, I added the FuzzyOCR (via Debian repos) into sa a few days ago, and finally got around to running sa-compile which looks like it fails. I tried with the -D option, but the verbosity was too great for me to make head nor tail of it. I don't know if this is FuzzyOCR or Where could I start to look? Regards, S # sa-compile Feb 15 13:54:52.424 [14842] info: config: failed to parse line, skipping, in /etc/spamassassin/local.cf: dcc_add_header 1 Feb 15 13:54:52.777 [14842] info: generic: base extraction starting. this can take a while... Feb 15 13:54:52.777 [14842] info: generic: extracting from rules of type body_0 100% [=] 9213.57 rules/sec 00m00s DONE 100% [ ] 322.06 bases/sec 00m08s DONE Feb 15 13:55:01.576 [14842] info: body_0: 1432 base strings extracted in 9 seconds sa-compile: not compiling; 'spamassassin --lint' check failed!
Re: [Q] sa-compile: not compiling; 'spamassassin --lint' check failed!
On 02/15/2011 02:19 PM, Lawrence @ Rogers wrote: On 15/02/2011 9:27 AM, J4K wrote: spamassassin --lint This may seem obvious, but did you run spamassassin --lint like sa-compile suggested? I assume DCC is probably not loaded, or disabled in your setup. Open up /etc/spamassassin/local.cf, find this line dcc_add_header 1 Comment it out. The line should look like this when you are done #dcc_add_header 1 Save your changes and run spamassassin --lint again. This time there should be no complaints from it. If there are not, try sa-compile again. Regards, Lawrence Hi Lawrance, DCC has always been running. I have no idea why SA always complains about this, but it was an not a problem before. dcc 3130 0.0 0.0 19808 196 ?Ss Jan24 0:00 /usr/local/bin/dccifd -I dcc -Q -Goff And it is logging away (not a lot) Feb 15 14:20:32 logout dccifd[3131]: grey-query is only available when greylisting is enabled with -G and not disabled with 'grey-off' Also, I thought that one only had to run spamassassin --lint once. I did a month ago. Afterwards, I thought that only sa-compile had to be run. Its pretty moot any way, because now after running spamassassin -lint, sa-compile still fails with the same error. # spamassassin --lint Feb 15 14:34:22.759 [15147] warn: config: failed to parse line, skipping, in /etc/spamassassin/local.cf: dcc_add_header 1 Feb 15 14:34:24.860 [15147] warn: lint: 1 issues detected, please rerun with debug enabled for more information # sa-compile Feb 15 14:34:33.485 [15149] info: config: failed to parse line, skipping, in /etc/spamassassin/local.cf: dcc_add_header 1 Feb 15 14:34:34.660 [15149] info: generic: base extraction starting. this can take a while... Feb 15 14:34:34.661 [15149] info: generic: extracting from rules of type body_0 100% [=] 25240.75 rules/sec 00m00s DONE 100% [=] 102.57 bases/sec 00m53s DONE Feb 15 14:35:28.356 [15149] info: body_0: 4108 base strings extracted in 54 seconds sa-compile: not compiling; 'spamassassin --lint' check failed!
Re: [Q] sa-compile: not compiling; 'spamassassin --lint' check failed!
On 02/15/2011 02:43 PM, Lawrence @ Rogers wrote: On 15/02/2011 10:07 AM, J4K wrote: Its pretty moot any way, because now after running spamassassin -lint, sa-compile still fails with the same error. Hi, Just because DCC is running doesn't mean SA is configured to use it. Can you post the following: - Output of spamassassin --lint - Contents of /etc/spamassassin/local.cf Those should help pinpoint the exact problem. Regards, Lawrence Hi Lawrence, Here you go. Simon. # spamassassin --lint Feb 15 14:34:22.759 [15147] warn: config: failed to parse line, skipping, in /etc/spamassassin/local.cf: dcc_add_header 1 Feb 15 14:34:24.860 [15147] warn: lint: 1 issues detected, please rerun with debug enabled for more information cat /etc/spamassassin/local.cf | grep -v ^# rewrite_header Subject *SPAM* report_safe 0 dns_available yes use_dcc 1 dcc_path /usr/local/bin/dccproc dcc_home /var/dcc dcc_add_header 1 use_pyzor 1 pyzor_path /usr/bin/pyzor pyzor_timeout 5 use_razor2 1 razor_config /etc/razor/razor-agent.conf razor_timeout 5 required_score 6.0 use_bayes 1 skip_rbl_checks 1 bayes_auto_learn 1 bayes_expiry_max_db_size30 bayes_auto_expire 1 ifplugin Mail::SpamAssassin::Plugin::Shortcircuit shortcircuit USER_IN_WHITELIST on shortcircuit SUBJECT_IN_WHITELISTon shortcircuit USER_IN_BLACKLIST on shortcircuit SUBJECT_IN_BLACKLISTon loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody endif # Mail::SpamAssassin::Plugin::Shortcircuit
Re: [Q] sa-compile: not compiling; 'spamassassin --lint' check failed!
On 02/15/2011 03:21 PM, Michael Scheidell wrote: On 2/15/11 8:46 AM, J4K wrote: use_dcc 1 dcc_path /usr/local/bin/dccproc dcc_home /var/dcc dcc_add_header 1 just like lint says, dcc_add_header is NOT valid. locate DCC.pm /usr/local/lib/perl5/site_perl/5.10.1/Mail/SpamAssassin/Plugin/DCC.pm /var/amavis/etc/DCC.pm mx1# grep dcc_add /usr/local/lib/perl5/site_perl/5.10.1/Mail/SpamAssassin/Plugin/DCC.pm no dcc_add_header in the plugin. Removed. It took a while for this part to sink in. sa-compile runs well now. Thank-you. and stop will the piss yellow background. Good point. Jaundice in just not in these days.
X-IronPort-AV: E=Sophos;i=4.60,386,1291590000; d=scan'208;a=41500553
Good morning everyone (almost the week-end), Is X-IronPort-AV added by SA, or from something else (DCC Clamav ? ) I just noticed that all email from a certain company was flagged with X-IronPort-AV, and I wonder why this is so. I have searched on the usual engine, and saw refereces to this header, but not to the programme. Regards, S. Original Message Return-Path:...@tele2.com X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on logout.simonloewen.info X-Spam-Level: X-Spam-Status: No, score=0.0 required=4.0 tests=HTML_MESSAGE shortcircuit=no autolearn=ham version=3.3.1 X-Spam-Virus: No Delivered-To: xxx...@klunky.co.uk Received: from xx.tele2.se (.tele2.se [193.12.60.45]) by klunky.co.uk (Postfix) with ESMTP id E4119843B7 for xx...@klunky.co.uk; Thu, 27 Jan 2011 16:40:47 +0100 (CET) X-IronPort-AV: E=Sophos;i=4.60,386,129159; d=scan'208;a=41500553 Received: from xx.tele2.com ([172.16.33.32]) by .tele2.se with ESMTP; 27 Jan 2011 16:40:47 +0100 In-Reply-To: OFB15345B3.8FFF0406-ONC125781E.004F7DEC-C125781E.0053C4F3@LocalDomain References: OFB15345B3.8FFF0406-ONC125781E.004F7DEC-C125781E.0053C4F3@LocalDomain X-Disclaimed: 38106 To: MIME-Version: 1.0 Subject:Correction 29-Jan (Re: Saturday Borrel 5-Feb-2011 in Amstelveen) X-KeepSent: 986ACF3D:EADBDE0D-C1257825:00550505; type=4; name=$KeepSent Message-ID: of986acf3d.eadbde0d-onc1257825.00550505-c1257825.00565...@tele2.com From: xx Date: Thu, 27 Jan 2011 16:40:45 +0100 Content-Type: multipart/alternative; boundary==_alternative 0056520DC1257825_=
Re: X-IronPort-AV: E=Sophos;i=4.60,386,1291590000; d=scan'208;a=41500553
On 01/28/2011 10:13 AM, Giles Coochey wrote: On 28/01/2011 10:11, Giles Coochey wrote: On 28/01/2011 10:02, J4K wrote: Good morning everyone (almost the week-end), Is X-IronPort-AV added by SA, or from something else (DCC Clamav ? ) I just noticed that all email from a certain company was flagged with X-IronPort-AV, and I wonder why this is so. I have searched on the usual engine, and saw refereces to this header, but not to the programme. X-IronPort-AV: E=Sophos;i=4.60,386,129159; d=scan'208;a=41500553 Sophos is an anti-virus company... I would check their product list... probably one of these http://www.sophos.com/products/enterprise/email/security-and-control/appliances/ Tell a lie, probably the Cisco Ironport: http://www.ironport.com/products/ I imagine it uses the Sophos engine though Cheers Giles. Then I presume its not my server adding the header. It must be Tele2's server somewhere along the way. S.
Re: spamhaus dbl considered safe for mta blocking?
On 01/27/2011 05:58 AM, Sahil Tandon wrote: On Sat, 2011-01-22 at 12:43:55 -0500, Michael Scheidell wrote: oh, and to be safe: reject_rhsbl_helo dbl.spamhaus.org=127.0.1.2, reject_rhsbl_client dbl.spamhaus.org=127.0.1,2, reject_rhsbl_sender dbl.spamhaus.org=127.0.1,2, Sound advice to advocate good practices, but in more recent version of Postfix, this should not be required. Wietse cleaned up the reject_rhsbl code to hopefully avoid these false positives. or it might reject: mail from: idiot@23.45.67.5 That should be rejected even before the RHSBL checks with: 501 5.1.7 Bad sender address syntax (127.255.255.255 is returned if you pass it an ip address) 127.0.1.255 is returned for IP queries to the SpamHaus DBL. withing seconds of putting on a 2000 user box, got hits. (just using _sender) looked up the sender's name and found 27 spams sent today that SA had to deal with (no more!) Glad to hear it's working well for you - I'm having a similar experience! I'm using spamhaus and junkmailfilter. At the moment, what is missed by junkmailfilter is often caught by spamhaus (Obviously, because of the postfix settings!): (postfix 2.7.n) reject_rbl_client hostkarma.junkemailfilter.com=127.0.0.2, reject_rbl_client sbl-xbl.spamhaus.org
Re: score=100.0 required=3.0 tests=SHORTCIRCUIT,,USER_IN_BLACKLIST
On 01/27/2011 01:55 PM, Florescu, Dan Alexandru wrote: Fire up what? Correct me if I'm wrong, but as far as I know SA does not reject at SMTP session level. I myself am using it with amavis and I have: $sa_quarantine_cutoff_level = 12.0; which will drop any spammy message with that score or above it. If you don't have any option you can use header_checks and drop spams there, although you should be warned that false positives will be dropped also. I think SA delivers by default spam-marked messages to avoid these false positives. /^X-Spam-Level: / DISCARD This is a spam message (score 12) Shortcircuit is useful as it will not run any other tests (less cpu usage) if it is sure that message is spam. -Original Message- From: Bowie Bailey [mailto:] Sent: Monday 24, January 01, 2011 19:32 To: users@spamassassin.apache.org Subject: Re: score=100.0 required=3.0 tests=SHORTCIRCUIT,,USER_IN_BLACKLIST On 1/24/2011 11:50 AM, J4 wrote: Hi all, Just would like to check that my settings are correct. The rcpt was blacklisted, yet the spam was delivered. I had thought that it would have been rejected during the SMTP session via spamass-milter, but I did not see it fire in the logs. Perhaps I have missed something in the spam-milter set-up and integration with postfix Regards, S Original Message Return-Path: si...@baduser.bomb X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on logout.niceemailserver.test X-Spam-Flag: YES X-Spam-Level: ** X-Spam-Status:Yes, score=100.0 required=3.0 tests=SHORTCIRCUIT, USER_IN_BLACKLIST shortcircuit=spam autolearn=disabled version=3.3.1 X-Spam-Virus: _CLAMAVRESULT_ X-Spam-Report:* 0.0 SHORTCIRCUIT Not all rules were run, due to a shortcircuited rule * 100 USER_IN_BLACKLIST From: address is in the user's black-list SA marked it as spam. If it was not rejected, you should check your spamass-milter settings. What I thought I might have missed was the -r -1 option. The man page states that if -r -1 is used, then this could reject based on user preferences. I have spamassassin talking back to a mysql table for userprefs. Thus options are now set to this:- OPTIONS=-u nobody -m -r -1 -i 127.0.0.1 -e -f -p /var/spool/postfix/spamass/spamass.sock However, spam was not rejected, although I think that this might cause unnecessary backscatter in the case of probably forged From addresses, which is a little unfair. I would prefer USER_IN_BLACKLIST reject with a message, because users who black list specific email addresses do so not because its spam, but because they simply won't want email from the address. I this is the wrong way to go about it, but its what people do. I could add this into seive (dovecot) to discard the message, but I would prefer the original sender to receive the reject message during the SMTP session in this specific case. Its not the end of the world if this cannot be done simply, but perhaps the blacklisted people really has to send an important message. At least they know they have been rejected. Otherwise, I shall let the Email through. /On 01/27/2011 01:59 PM, Giles Coochey wrote: / /spamass-milter can reject according to SA results at the SMTP session level. The OP mentioned that. / Yep, I did. Regards, S
