Way to set user-prefs without a database?
Hey there, We have a couple of user accounts (really, role aliases) that need a different required_score from our global defaults. Since they're role accounts, they don't have a homedir. We're using a milter that passes the whole username (including domain name) along, anyway. Is there a dead-simple way to make this work using only the config files, or do I have to go to the trouble of setting up all of mysql just to make this happen? Best, -Dan Mahoney -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Good rules for PGP-Signed/Encrypted mail?
Hey all, The Day Job (and some of you may know what job that is) does enough PGP related stuff that we've had encrypted messages get dropped on occasion, and we'd like to whitelist this stuff. It looks like Mail::Spamassassin::Plugin::OpenPGP is way way old and has requirements that aren't exactly standard in our packaging system (BSD), so a rules-only approach might be nice. Does anyone see any problems with the *SYNTAX* of the rules at? https://svn.apache.org/repos/asf/spamassassin/trunk/rulesrc/sandbox/khopesh/20_khop_blessed.cf That would break under a modern spamassassin? (Yes, yes, I know we're not validating the messages/keys themselves, but I'd like a message to security-officer@ to NOT get dropped on the floor, and since this isn't a widespread rule, it's not likely we'll be specifically targeted knowing this rule is in place.) -Dan -- Happy, Sad, Happy, Sad, Happy, Sad, Happy, Intruiged! I've never been so in touch with my emotions! -AndrAIa as Hexadecimal, Reboot Episode 3.2.3 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
sa-learn from a cronjob?
All, Most of my users aren't command-line friendly. I'd like to basically have my IMAP server default to handing out two imap mailboxes that get auto-crontabbed to training bayes. Ideally, I'd also like to make it so that things dropped in the learn_spam folder are deleted, and stuff in the learn_ham folder (mistake-based training) are de-tagged and moved back to the inbox. Alternatively, a single learned folder would do. Perl's Mail::Box seems like a heavy tool for this simple task. Does anyone else have any recommendations? -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Weighted MIRRORED.BY files?
On Sun, 24 Mar 2013, Mark Martinec wrote:
On Sunday March 24 2013 05:57:49 Dan Mahoney, System Admin wrote:
sa-update also uses a mirror file which lists all of the URLs where the
update can be downloaded from, optionally including weights for different
mirrors.
But there's no documentation or examples given for weighting. Anyone
closer to the code know what this would look like?
$ curl http://spamassassin.apache.org/updates/MIRRORED.BY
http://sa-update.dnswl.org/ weight=1
http://www.sa-update.pccc.com/ weight=5
http://sa-update.secnap.net/ weight=5
sa-update has the following in its comments:
# choose a random integer between 0 and the total weight of all mirrors
# loop through the mirrors from largest to smallest weight
# if random number is largest weight, use it
# otherwise, random number -= largest, remove mirror from list, try again
# eventually, there'll just be 1 mirror left in $mirrors[0] and it'll be used
#
sub choose_mirror {
my($mirror_list) = @_;
[...]
I'll add this on to the wiki.
-Dan
--
Dan Mahoney
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---
Weighted MIRRORED.BY files?
Hey there. The SA wiki says: sa-update also uses a mirror file which lists all of the URLs where the update can be downloaded from, optionally including weights for different mirrors. But there's no documentation or examples given for weighting. Anyone closer to the code know what this would look like? -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Yahoo single link spam
On Fri, 22 Feb 2013, Kevin A. McGrail wrote:
On 2/22/2013 3:27 PM, David F. Skoll wrote:
On Fri, 22 Feb 2013 12:20:22 -0800
Marc Perkel supp...@junkemailfilter.com wrote:
We need a rule to catch this. It looks like more data than it is but
it's really little more than a single link. Like to see a rule that
identifies it.
Our product lets you make compound rules. It should not be very hard
to translate this to SpamAssassin:
HeaderMatches RegExp ^To:(.*?@.*?){5} AND
Envelope Sender Ends with@yahoo.com AND
MessageSize 6000
Well, ok... the MessageSize condition is tricky. And this rule does
kick up some false-positives, but overall it works pretty well for us.
Here's the current version I'm using based on 3.4.0 trunk:
#YAHOO COMPROMISED ACCOUNT SPAMS - SCORED HIGH BECAUSE THESE ARE COMPROMISED
ACCOUNTS WHICH MAKES ALL OF YAHOO!'s PROCEDURES QUESTIONABLE
header __KAM_YAHOO1From =~
/\@(yahoo.com|yahoo.com.id|rocketmail.com)/i
header __KAM_YAHOO2Subject =~ /^(FOR |Hey$|hi$|look at
this$|great!?$|amazing!?|the best!?$|excellent!?$|very good!?$|great!?$)/
body__KAM_YAHOO3/\d{1,2}[\\\/]\d{1,2}[\\\/]\d{2,4}
\d{1,2}\:\d{1,2}\:\d{1,2} (AM|PM)/
header __KAM_YAHOO4From:name =~ /Connor Hopkins/i
metaKAM_YAHOO (__KAM_YAHOO1 + __KAM_YAHOO2 + __KAM_YAHOO3 +
__KAM_YAHOO4 + __KAM_BODY_LENGTH_LT_128 + MISSING_SUBJECT = 3)
describeKAM_YAHOO Compromised Yahoo! Accounts Sending Spam
score KAM_YAHOO 9.0
Just to add a late reply to the game, I'm still getting these. Kevin, it
looks like your rules YAHOO1 and YAHOO3 are still appropriate, but neither
of the others. I think there's a few other things I've noticed that I
don't know how to match:
the body doesn't contain the link, it pretty much IS the link.
However, I don't know how to write a rule that says contains a link and
NOTHING ELSE. I also don't know how to write rules that say the
text/plain portion contains a link, and the text/html portion contains
more. I'm not aware of how body gets interpreted in
multipart/alternative messages. Kevin, if you're able to tell me more
about this, I'm happy to learn.
Writing rules is easy for some, but I'm more about solving the problem.
The answer isn't many people write many custom rulesets, it's surbl
catches up faster or yahoo acknowledges the problem.
While yahoo's abuse reporting procedures leave much to be desired, this is
actually one of the reasons I was asking about a channel to autoreport
mail to spamcop (and yahoo, if they were willing to take it, but they
don't seem to be -- blog post coming on that, soon).
-Dan
--
One...plus two...plus one...plus one.
-Tim Curry, Clue
Dan Mahoney
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---
Re: [sa-list] Re: How to log detected locale/language?
On Fri, 8 Mar 2013, Axb wrote: On 03/08/2013 04:46 PM, Dan Mahoney, System Admin wrote: Hey there all, It seems a pretty core function in SA is the ok_languages and ok_locales function. I'd like to be able to turn on LOGGING of detected locales before I set which are ok (or specifically, which are less ok) I'm sure there's a knob for this somewhere, can anyone tell me where? Nice someone documented this: http://spamassassin.apache.org/full/3.3.x/doc/Mail_SpamAssassin_Conf.txt _LANGUAGES_ so now what? a few lines later it tells us what to do add_header all X-BLAHTYPE _LANGUAGES_ add that to your local.cf and reload SA, glue, coffee machine. does this do what you want? Mostly, but I can't figure out how to get the LOCALE (which is purely characterset based) to work. What's the logging macro for that one? -Dan -- I wish the Real World would just stop hassling me! -Matchbox 20, Real World, off the album Yourself or Someone Like You Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
How to log detected locale/language?
Hey there all, It seems a pretty core function in SA is the ok_languages and ok_locales function. I'd like to be able to turn on LOGGING of detected locales before I set which are ok (or specifically, which are less ok) I'm sure there's a knob for this somewhere, can anyone tell me where? -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
pyzor 401/unauthorized?
I was in the process of linting my SA config when I discovered that the pyzor servers are handing back this response to all commands: /usr/local/bin/pyzor --homedir /usr/local/etc/mail/spamassassin/.pyzor ping public.pyzor.org:24441 (401, 'Unauthorized: User is not authorized to request the operation.') As opposed to the myriad of other issues I've seen on this list where the user can't set pyzor_home correctly or firewall issues, I'm pretty sure I'm doing things right (I don't get a backtrace or anything) and this appears to be server-side. -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Supporting spamcop quick reporting
On Tue, 19 Feb 2013, Andrzej A. Filip wrote: On 02/19/2013 03:47 AM, Dan Mahoney, System Admin wrote: Spamcop has an undocumented feature that they allow you (if they trust you) to quick report spam, where you send to a different mail address, and it's reported instantly, without having to hit the web interface. When you do this, you are still free to report spam in the usual way (with the confirm screen) by using your usual reporting-address. [...] AFAIK/AFAIR: Spamcop.net quick reporting automatically sens reports/LARTs about spam reported via SMTP _based on mail routing only_ (no reports/LARTs about spamvertized web sites). It has been intended for spamtraps' catch. Reporting again the usual way could to easily create duplicate reports. By this I meant (and apologies if anyone else misconstrued it), that when spamcop enables this feature, you may use either to process a given message, but not both. However, as I now read here: http://forum.spamcop.net/scwik/QuickReporting You're quite right, it doesn't parse the body, which is a shame. See my reply to kevin for more information. -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Supporting spamcop quick reporting
On Tue, 19 Feb 2013, Kevin A. McGrail wrote: On 2/18/2013 9:47 PM, Dan Mahoney, System Admin wrote: Hey there, Spamcop has an undocumented feature that they allow you (if they trust you) to quick report spam, where you send to a different mail address, and it's reported instantly, without having to hit the web interface. When you do this, you are still free to report spam in the usual way (with the confirm screen) by using your usual reporting-address. How hard would it be to extend spamassassin's report syntax to allow this? Unfortunately, I'm not seeing a good way to pass config-options to spamd, so that's out. (I suppose this email could be interpreted as a case of is this useful?). Running the report against spamassassin locally would lose me the other learning (bayes, etc). Creating an alternate user with the quick-reporting mail address sent is similarly problematic (althouth I *might* be able to do this by playing with the userpref sql query). I'm open to any other ideas people have come up with. Hi Dan, Looking a this in a high level, I think you are referring to spamc's reporting feature. I am. I receive email for my entire domain, and I have several mailboxes which meet spamcop's definition of traps -- they have NEVER been used to receive legit mail, and were basically made up by list-sellers to pad lists, and are not even close (typographically) to any other email addresses I've got. They have real names and other such demographic information, and are doctors, apparently, based on the crap they get. For a while, I tried reaching out to the people mailing me (who looked legit) and tried to tell them okay, this is the first time I'm seeing mail to this address, you got scammed by whomever sold you this list). But bulk-mailers (legit or not) deal in volume, and can-spam basically says they don't have to care. Faced with this, I had three options: 1) Unsubscribe, basically self-listwashing. 2) Route the mail to /dev/null. 3) Allow these email addresses to act like a poisoned fruit, and serve as a marker of the spam and irresponsible list-buyers, and act as a sigil with razor/pyzor/spamcop. With #3, the annoyance is that I now send to spamc -C report, but get a steady stream of emails that say spamcop has accepted one email for processing. And of course, because spamcop wants their mail to be fresh it means I'm dealing with a constant stream of having to log in and click through. Aside: What's more braindead, on Spamcop's end, is that while they won't accept mail over two days old, if you don't go in and click report/cancel, it will wait for you in the queue, for weeks. (And from what they tell me, they don't parse the mail until you hit report now, so they cite CPU overhead on doing advanced expiry). They seem to have missed the bit that they have the date-of-submission without having to parse the body. /Aside. However, that's likely not the best avenue unless you are just trying to send spamcop examples of algorithmically determined spam. I wonder if it is time for a separate reporting binary and perhaps build on the existing collaboration reporting in spamc/d and add RPS::Mail::EventReporter for reputation collaboration. I would be in favor of this. It would also seem that DCC's reputation code/reporting should have support in the latest version of SA. As I now read that spamcop's quick reporting isn't as thorough as their manual report, I'm somewhat less interested, but better support in a tool could change that. -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Supporting spamcop quick reporting
On Tue, 19 Feb 2013, Andrzej A. Filip wrote: On 02/19/2013 08:53 PM, Dan Mahoney, System Admin wrote: On Tue, 19 Feb 2013, Andrzej A. Filip wrote: On 02/19/2013 03:47 AM, Dan Mahoney, System Admin wrote: Spamcop has an undocumented feature that they allow you (if they trust you) to quick report spam, where you send to a different mail address, and it's reported instantly, without having to hit the web interface. When you do this, you are still free to report spam in the usual way (with the confirm screen) by using your usual reporting-address. [...] AFAIK/AFAIR: Spamcop.net quick reporting automatically sens reports/LARTs about spam reported via SMTP _based on mail routing only_ (no reports/LARTs about spamvertized web sites). It has been intended for spamtraps' catch. Reporting again the usual way could to easily create duplicate reports. By this I meant (and apologies if anyone else misconstrued it), that when spamcop enables this feature, you may use either to process a given message, but not both. However, as I now read here: http://forum.spamcop.net/scwik/QuickReporting You're quite right, it doesn't parse the body, which is a shame. See my reply to kevin for more information. One option would be co create via HTTP reporter automatically sending reports/LARTs about spam without possibly spamvertised URLs. It should be quite simple to implement if you are ready to wait extra 5-7s per every spam reported. an interesting thought, but I'm not sure what you mean there. Do you mean as a means of sending the initial report instead of email, or do you mean as a means of both reporting the spam, AND ack'ing it? (as if I had pasted it in). -Dan -- [23:49:00] LarpGM: Did my little TP comment scare you off? [23:49:22] ilzarion: no, the shrieking retarded child eating people did -Feb 06, 2001, times apparent. Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Supporting spamcop quick reporting
Hey there, Spamcop has an undocumented feature that they allow you (if they trust you) to quick report spam, where you send to a different mail address, and it's reported instantly, without having to hit the web interface. When you do this, you are still free to report spam in the usual way (with the confirm screen) by using your usual reporting-address. How hard would it be to extend spamassassin's report syntax to allow this? Unfortunately, I'm not seeing a good way to pass config-options to spamd, so that's out. (I suppose this email could be interpreted as a case of is this useful?). Running the report against spamassassin locally would lose me the other learning (bayes, etc). Creating an alternate user with the quick-reporting mail address sent is similarly problematic (althouth I *might* be able to do this by playing with the userpref sql query). I'm open to any other ideas people have come up with. -Dan -- this is too stupid even for irc -mtreal, EFnet #macintosh, 09/15/2K, 12:33 AM Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Still no apparent fix on ipv6 spamd?
On Tue, 25 Sep 2012, Kevin A. McGrail wrote: On 9/25/2012 5:02 PM, Dan Mahoney, System Admin wrote: I mentioned this on the mailing lists a few years ago. I notice that there still doesn't seem to be a clean way to just make spamd listen on all (v4 and v6) addresses by default, nor is there a way to listen on multiple addresses with multiple -A options. This means that if you want to listen on v6, none of your v4 clients can connect. I also note that like all standard resolver libraries, if you specify a hostname to spamc, it tries the v6 variant first -- so the default behaviors between spamc and spamd are still conflicting. Nor is there an option in spamc to say use this hostname, but only try v4. Has anyone come up with patches for the above, or is the solution really to just hard-code the ipv4 address everywhere when doing a remote-connect (or perhaps define alternate v4-only hostnames for your spamd hosts). Hi Dan! I'm working on packaging an RC for 3.4.0 and ipv6 is a big focus of this release. Can you open a bug about these issues with as much information as you can, please? 6840 (docs) 6841 (spamd) 6842 (spamc) -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Still no apparent fix on ipv6 spamd?
All, I mentioned this on the mailing lists a few years ago. I notice that there still doesn't seem to be a clean way to just make spamd listen on all (v4 and v6) addresses by default, nor is there a way to listen on multiple addresses with multiple -A options. This means that if you want to listen on v6, none of your v4 clients can connect. I also note that like all standard resolver libraries, if you specify a hostname to spamc, it tries the v6 variant first -- so the default behaviors between spamc and spamd are still conflicting. Nor is there an option in spamc to say use this hostname, but only try v4. Has anyone come up with patches for the above, or is the solution really to just hard-code the ipv4 address everywhere when doing a remote-connect (or perhaps define alternate v4-only hostnames for your spamd hosts). -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Still no apparent fix on ipv6 spamd?
On Tue, 25 Sep 2012, Kevin A. McGrail wrote: On 9/25/2012 5:02 PM, Dan Mahoney, System Admin wrote: I mentioned this on the mailing lists a few years ago. I notice that there still doesn't seem to be a clean way to just make spamd listen on all (v4 and v6) addresses by default, nor is there a way to listen on multiple addresses with multiple -A options. This means that if you want to listen on v6, none of your v4 clients can connect. I also note that like all standard resolver libraries, if you specify a hostname to spamc, it tries the v6 variant first -- so the default behaviors between spamc and spamd are still conflicting. Nor is there an option in spamc to say use this hostname, but only try v4. Has anyone come up with patches for the above, or is the solution really to just hard-code the ipv4 address everywhere when doing a remote-connect (or perhaps define alternate v4-only hostnames for your spamd hosts). Hi Dan! I'm working on packaging an RC for 3.4.0 and ipv6 is a big focus of this release. Can you open a bug about these issues with as much information as you can, please? You got it. Later today, probably. Do you prefer one bug or multiple (there's at least four or five issues in this)? -Dan -- Pika Pika Pika! -Pikachu, of Pokemon fame. Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Spamhaus Whitelist
All, Has anyone come up with a ruleset yet to score against the new spamhaus whitelists, and deduct points appropriately? -Dan -- Let me tell you something about regrowing your dead wife Lucy, Harry. It's probably illegal, potentially dangerous, and definitely crazy. -Harry nods- Vincent Spano, as Boris in Creator. Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Learing spam/ham with Pine
On Wed, 3 Nov 2010, John Hardin wrote: On Wed, 3 Nov 2010, Pat Traynor wrote: I've been running Spamassassin on my linux server for some time, and I use Pine to read my mail. Hello, fellow fossil! Aah, yonder fossils. I've found, by the way, that if you're not using Alpine, you sure should be. Better bits, and some cool new features. I consider myself a pine power user. So, things to know. 1) If you're using pine and not procmail, you're missing out. Learn it, live it, love it. 2) While John's methods for learning and reporting spam work, I've found that the best way to do it is per-message within spamassassin, via spamd/spamc. The pipe command returns MUCH faster in this config. (Make no mistake, there's nothing wrong with periodically feeding your spam and ham folders to bayes as well, but I have a personal policy of report what gets through the filters to improve the system. 3) While you're at it, if you're using spamc/spamd, take whatever account they're registered under, and tie them to DCC/Pyzor/Razor -- if you're reporting, you might as well get the most bang for your buck. If you're not using them, then register yourself a Pyzor/Razor/DCC account. It's quick and easy. Anyway, you'll need to go into your pine options and turn on the following: * Enable Aggregate Command Set (this allows you to select and act on multiple messages at once) * Enable unix pipe command From there, you can simply take any message (or any group of messages) and press | to start a pipe, and set the following options: * Raw Message * Uncaptured output * If working with multiple messages, set the new pipe option so each message gets fed to a separate copy of the command, and for the command itself, I put: /usr/local/bin/spamc -d quark.gushi.org --reporttype=report or /usr/local/bin/spamassassin --report If you want to correct a false positive, chance --report to revoke -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Spamhaus Whitelist
On Sat, 6 Nov 2010, David F. Skoll wrote:
On Sat, 06 Nov 2010 00:41:53 -0700
Bill Landry b...@inetmsg.com wrote:
You could also test the envelope sender:
header SPAMHAUS_ENV eval:check_rbl_envfrom('SPAMHAUS_ENV',
'_vouch.dwl.spamhaus.org.')
But that's an abuse... you should not be using Vouch-by-reference unless
either DKIM or SPF returns a pass. Otherwise, you've just told
spammers who they should pretend to be to get their spam in your inbox.
Yeah, I read that and suspected this needed more complex config than just
another Whitelist.
-Dan
--
Happy, Sad, Happy, Sad, Happy, Sad, Happy, Intruiged! I've never been so
in touch with my emotions!
-AndrAIa as Hexadecimal, Reboot Episode 3.2.3
Dan Mahoney
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---
Pyzor occasionally dying when called from spamassassin.
Hey there, I just enabled pyzor as part of spamassassin (freebsd 6.4, pyzor built from ports), and occasionally get this message in my logs: Jul 9 05:40:59 quark spamd[11607]: spamd: connection from prime.gushi.org [72.9.101.130] at port 51280 Jul 9 05:40:59 quark spamd[11607]: spamd: processing message 80052004218074290153548c4434576868b5c94f5dd661c0...@pd164.marketingfx.info for minn:58 Jul 9 05:41:05 quark spamd[11607]: pyzor: [11983] error: TERMINATED, signal 15 (000f) I've got spamd at maximum logging, does anyone offhand know what this error means? -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Minor Doc Issue on spamc config file?
Hey all, In my spamc config file I have: -d 72.9.101.140 -l --connect-retries=10 --retry-sleep=30 However, procmail scripts that I was using to report, via spamc -C report, were simply returning the message. When I added -d 72.9.101.140, the message was properly reported. The manpage states: Existing command line switches will override any settings in the configuration file. What I took from this was that if I specified, say, -d on the command line, it would override what's in the file. But what this really seems to mean is that you need to specify ALL options when using the command line. In other words, your command line can be really long (specifying all options from the command line, plus all options that would be in your config file) or really short (spamc or spamd -F configfile). Is this by design? With most other programs, it would seem only the options specified on the command line would be overridden (like -d 127.0.0.1 in my example above). -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Passing preferences to spamd?
All, It seems the only way to pass a preference from spamc to spamd is by having a different user-id. In my specific case, I'd like to report to spamcop using their quick UID for some mails, but keep all my preferences otherwise the same (so I still get the benefits of bayes, dcc, awl, etc). (Think of this in terms of the -o options to ssh) Since I'm using the DB backend, I could do some tricks, like modify the query to template one set of userprefs over another, I suppose, but it would be nice to have a unified way. It seems there's no way to override an additional pref on the command line with any of (spamc, spamd, spamassassin) -- you have to override the whole file, and sometimes even more than that, in the case of spamc/spamd. -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Passing preferences to spamd?
On Thu, 8 Jul 2010, Karsten Bräckelmann wrote: On Wed, 2010-07-07 at 18:09 -0400, Dan Mahoney, System Admin wrote: It seems the only way to pass a preference from spamc to spamd is by having a different user-id. In my specific case, I'd like to report to spamcop using their quick UID for some mails, but keep all my preferences otherwise the same (so I still get the benefits of bayes, dcc, awl, etc). Correct. You cannot pass anything spamd / spamassassin (the CLI tool) accepts as options via spamc. Since I'm using the DB backend, I could do some tricks, like modify the query to template one set of userprefs over another, I suppose, but it would be nice to have a unified way. User prefs in DB? Then having specific sets of user_prefs (and only what's allowed in there, no spamd options obviously) should be simple. Have a look at the spamc -u username option. Yes, I saw that...what I need to do is come up with a clever way of saying if I pass an impossible username, such as danm_reporting, the query does the right thing. (Since the mysql command language has a split function, I should be able to do this without touching the spamassassin code.) I might have to also modify the sql queries for the bayes/awl backends as well, so they know danm_report is really danm, since reporting also includes learning. This also opens up the possibility of creating a more strict setup for different email addresses, so -u danm_strict could have a required_score of 2, for addresses that are more agressively spammed. (Yes, this would take multiple passes through spamd or some special procmail logic). IIRC it works with DB backend. From memory, since I once tried long ago, it does not work if you're using $HOME based user_prefs and running spamc as an ordinary user. If you want to see the behavior for yourself, please use the netcat trick I mentioned in a previous thread of yours. Run netcat listening on one port, and make spamc use that port -- you'll see the simple protocol headers, including the User to use by spamd, if possible. Knowing the headers is good, but being able to know what they look like, and knowing how to get the application to set them are two different problems, from my point of view. There are people who debug with syslog and -v, there are people who debug with tcpdump, and there are people who debug with strace. I'm the syslog type. It seems there's no way to override an additional pref on the command line with any of (spamc, spamd, spamassassin) -- you have to override the whole file, and sometimes even more than that, in the case of spamc/spamd. It is possible with spamassassin, as fine-grained as you want with any setting. See the --cf option in man spamassassin-run. At that point, I could in fact use spamassassin to point at the DB server where my bayes and awl, etc, live, the only real difference is in which cpu parses the message, the end effect is the same. My biggest problem with this, as I had brought up a while ago, is that I'd need to run spamassassin setGID, and put the db files in a different config that's not world-readable by all. I once wrote a small-but-useful tool which publishes your user_prefs to a database (and also fetches), asked on this list if people wanted it for CONTRIB. No reply. -Dan -- Hate fedora with a white hot burning passion right now though ... damn thing is Linux-XP(tm) -Bill Nolan 2/24/04 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Adding headers on spamassassin ignores?
Hey all, From what I've gathered, there's both a recommended way to call spamassassin/spamd from procmail with a message-size-limit, as well as an overrideable builtin-default (-s option to spamc). These both cause the usual spamassasin headers to be missing from messages. The greater problem is, that if for some reason spamassasin doesn't run (for example, a spamc timeout(*)) it produces exactly the same effect. Is there a way to have spamassasin/dspamd not scan messages above a certain size, but still add headers (i.e. x-spam-status: skipped)? I can do it in procmail, and add a header that means something to me (and face the additional problems of communicating this nuance to my users), but it would be nice if SA had a standard way. -Dan (*) with it's brilliant try 3 times, 1 second apart retry timer. -- If you need web space, give him a hard drive. If you need to do something really heavy, build him a computer. -Ilzarion, late friday night Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Adding headers on spamassassin ignores?
On Mon, 5 Jul 2010, Karsten Bräckelmann wrote: On Mon, 2010-07-05 at 02:31 -0400, Dan Mahoney wrote: The greater problem is, that if for some reason spamassasin doesn't run (for example, a spamc timeout(*)) it produces exactly the same effect. Is there a way to have spamassasin/dspamd not scan messages above a certain size, but still add headers (i.e. x-spam-status: skipped)? I can No, SA cannot add headers in case the message size exceeds the spamc threshold, because in that case spamc does not pass on the message to spamd at all. do it in procmail, and add a header that means something to me (and face the additional problems of communicating this nuance to my users), but it would be nice if SA had a standard way. With procmail, the spamc -s option actually should be irrelevant to you, unless *raising* the limit. Why have procmail pipe the message to a filter, if we know it will be passed back unhandled? :0 fw * 512000 | spamc Now there are two ways to add various skipped headers. A trivial one is negating the size condition. :0 fw * 511999 | formail -A X-Spam-Status: Skipped, too large A more fancy variant starts by using the spamc -x option in the above recipe, disabling the default safe fallback of returning an exit code of 0 regardless. Now errors will result in an actual error exit code, while the unprocessed message still is passed back. See man spamc. A procmail recipe to handle this must follow the spamc filter recipe immediately, and looks like this, using the procmail error flag. :0 e fw | formail -A X-Spam-Status: Error processing mail (*) with it's brilliant try 3 times, 1 second apart retry timer. If this is merely about timing issues, where restarting spamd might cause spamc to give up before the daemon is back, you could simply adjust these. Both retry times as well as numbers of attempts are configurable. To do that globally, without even touching your procmail recipes, you can use spamc.conf in your sysconfig dir. Again, see man spamc. I have done so, and that may alleviate some of the problem. However, while you've given me some procmail-based shortcuts and saved me a bit of research, my point was that it would be very nice if the messages and flags you mention were *standard* parts of spamd, and not just coded into my (and only my) procmailrc. It's probably fairly trivial to have spamc add only this header under these conditions, and it would make it more compatible with third-party tools that are out there, and as a bonus, spamc could keep within the principal of least surprise by requiring an extra command line option to add these headers, so not as to break existing scripts. -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Autoreplies from RT are hitting on ANY_BOUNCE_MESSAGE
On Mon, 28 Jun 2010, Yet Another Ninja wrote: On 2010-06-28 11:33, Dan Mahoney, System Admin wrote: Hey there, Perhaps this is by design, but rt replies are, strictly speaking, not bounce messages. Message attached, let me know if it looks normal. -Dan from what I see it looks normal if someone really makes an effort to tune SA scores. my 50_scores.cf deault says: score ANY_BOUNCE_MESSAGE 0.1 score SHORTCIRCUIT 0 Even so, why is it matching, when it's not a bounce. It's either something inaccurate in spamassassin, or something RT is doing that it shouldn't be. It it's the latter, I'll attempt to fix rt. If the former, perhaps SA should. -Dan -- You recreate the stars in the sky with cows? -Furrball, March 7 2005, on Katamari Damacy Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Autoreplies from RT are hitting on ANY_BOUNCE_MESSAGE
Hey there, Perhaps this is by design, but rt replies are, strictly speaking, not bounce messages. Message attached, let me know if it looks normal. -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org --- From s...@isc.org Thu Jun 3 20:29:04 2010 From: ISC Systems via RT s...@isc.org To: d...@prime.gushi.org Date: Fri, 4 Jun 2010 00:28:53 + Subject: SPAM(120.1) [ISC-Ops #28368] AutoReply: Live from new york Spam detection software, running on the system quark.gushi.org, has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: Greetings, This message has been automatically generated in response to the creation of a trouble ticket regarding: Live from new york, a summary of which appears below. There is no need to reply to this message right now. Your ticket has been assigned an ID of [ISC-Ops #28368]. [...] Content analysis details: (120.1 points, 5.0 required) pts rule name description -- -- 0.1 BOUNCE_MESSAGE MTA bounce message 100 SHORTCIRCUIT Not all rules were run, due to a shortcircuited rule 20 ANY_BOUNCE_MESSAGE Message is some kind of bounce message [ Part 2: original message before SpamAssassin ] X-Envelope-To: UNKNOWN From: ISC Systems via RT s...@isc.org To: d...@prime.gushi.org Date: Fri, 4 Jun 2010 00:28:53 + Subject: [ISC-Ops #28368] AutoReply: Live from new york Greetings, This message has been automatically generated in response to the creation of a trouble ticket regarding: Live from new york, a summary of which appears below. There is no need to reply to this message right now. Your ticket has been assigned an ID of [ISC-Ops #28368]. Please include the string: [ISC-Ops #28368] in the subject line of all future correspondence about this issue. To do so, you may reply to this message. Thank you, s...@isc.org - It's ISC live. -Dan -- Zaren Christ almighty... my EYES! They're melting! -Zaren, Efnet #macintosh, in response to: www.geocities.com/CollegePark/Classroom/1944 The WEBSITE DESIGN class that gave my fiancee a D. Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Learning and reporting with spamc in a single step?
On Mon, 28 Jun 2010, Karsten Bräckelmann wrote: On Sun, 2010-06-27 at 16:52 -0400, Dan Mahoney, System Admin wrote: Can spamc do this, or must it be forked to tee or something. Ideally I'd like to both report and learn in a single step (such as in a pipe from alpine). I note that spamassassin -r also has the option to learn (by default!), but spamc doesn't for some reason. Or if it does, the manpage neglects to mention it. Hmm, man spamc shows -L learn type and -C report type right next to each other. Yours doesn't? It shows them top to bottom, but does not say whether they're exclusive or not. As for the usage summary... %spamc -V SpamAssassin Client version 3.2.3 compiled with SSL support (OpenSSL 0.9.7e-p1 25 Oct 2004) SYNOPSIS spamc [options] message is less than helpful in determining which options work together. If you actually can use both options at the same time, I don't know. Maybe you wanna try it, and let us know. :) I wonder what the logs show (or are supposed to show) during these operations. -Dan -- You're a daddy. I'm a mommy. She's our baby. Deal with it. -Cali, 11/7/02, about 1:35 AM Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Does spamd support ipv6 yet?
I previously asked this question and was told the best answer might be to wait for 3.3. Was there ever support ratified for ipv6 including proper -A ipv6 access lists, and proper ability to listen on both the ipv6 default and the v4 default at the same time, when specifying -i? I'm not sure which bugs to look at to ascertain this. -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Learning and reporting with spamc in a single step?
Can spamc do this, or must it be forked to tee or something. Ideally I'd like to both report and learn in a single step (such as in a pipe from alpine). I note that spamassassin -r also has the option to learn (by default!), but spamc doesn't for some reason. Or if it does, the manpage neglects to mention it. In a perfect world, I'd also be able to choose the express or manual spamcop methods, which use different reporting addresses, but if I need to run two commands anyway from my .procmailrc, I might as well use spamc for one and spamassassin (with an alternate config file) for the other. -Dan Mahoney -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Adding remote-ip/ESMTPID/X-Envelope to logging output?
On Sun, 27 Dec 2009, Shane Williams wrote: One way to find what you want is to grab the msg id (or mid) from the spamd line, and grep for that out of the sendmail log for the remote IP. As I recall when I wrote something that searched like this, I had to do some special character quoting on some of the mid's, but did finally get it working. Yeah, it's possible to have my parser do that kind of cross-correlation, and I imagine it's what I'll have to do (along with using syslog to send files from one server to the other), as well as keeping them local. I just figured if there was a single local.cf tweak I could add that would change my logline, that might be the easier and more correct way, as getting the score, plus all the rules matched, plus the long-messageid, plus the short messageID is rather complex. (Even within just spamd it requires looking at multiple lines) For example, this line contains the score: Dec 28 02:37:35 quark spamd[9203]: spamd: identified spam (20.1/5.0) for danm:58 in 0.4 seconds, 3920 bytes. But this one, which contains almost everything else of use, does NOT contain the decimal score, and there's nothing there at ALL to cross-correlate them (and this is running in debugmode). I can guess based on the size, score, scantime and uid, but those are hardly unique, especially during a deluge. Dec 28 02:37:35 quark spamd[9203]: spamd: result: Y 20 - ANY_BOUNCE_MESSAGE,BOUNCE_MESSAGE scantime=0.4,size=3920,user=danm,uid=58,required_score=5.0,rhost=prime.gushi.org,raddr=72.9.101.130,rport=53762,mid=200912280733.nbs7xfkj049...@prime.gushi.org,bayes=0.001253,autolearn=disabled,shortcircuit=spam Somewhere within the spamd guts there's a print/printf line that prints that last line, and is supplied a list of variables. I mean only to add a couple more. I'm quite surprised it's not a tunable. I'm also surprised that, unline sendmail, spamd doesn't put a single token in EVERY logline, even if that's a unique messageid known only to SA (as is the case with sendmail logs). -Dan On Sat, 26 Dec 2009, Dan Mahoney, System Admin wrote: Hey there, Background: Sendmail with spamd running on a different box, spamc called from global procmail file. I'm doing some nightly log-combing to look for interesting patterns, including against other network traffic (like erroneous DNS lookups, I think I might be on to something). However, one of the annoying things about spamassassin's logging is that it fails to log the remote connecting ip, even though it places it in special places in the logs: take for example: Dec 26 08:41:51 quark spamd[87490]: spamd: connection from prime.gushi.org [72.9.101.130] at port 62430 Dec 26 08:41:51 quark spamd[87490]: spamd: processing message ca65a2f0912260536l2d0edcdy28f748beb74ea...@mail.gmail.com for danm:58 Dec 26 08:41:53 quark spamd[87490]: FuzzyOcr: Scan canceled, message has less than -5 points (-6.601). Dec 26 08:41:53 quark spamd[87490]: spamd: clean message (-6.6/5.0) for danm:58 in 1.9 seconds, 3788 bytes. Dec 26 08:41:53 quark spamd[87490]: spamd: result: . -6 - AWL,BAYES_00,RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS scantime=1.9,size=3788,user=danm,uid=58,required_score=5.0,rhost=prime.gushi.org,raddr=72.9.101.130,rport=62430,mid=ca65a2f0912260536l2d0edcdy28f748beb74ea...@mail.gmail.com,bayes=0.00,autolearn=ham,shortcircuit=no From those logs, there's nothing at all that tells me what the relaying ip is, even though it's special to spamd, used to determine the ASN, etc. The sendmail logs (grepped for that messageid) are more useful: Dec 26 08:38:23 mail.info prime sm-mta[27423]: nBQDcLck027423: from=asterisk-users-boun...@lists.digium.com, size=2735, class=0, nrcpts=1, msgid=ca65a2f0912260536l2d0edcdy28f748beb74ea...@mail.gmail.com, proto=ESMTP, daemon=MTA, relay=lists.digium.com [216.207.245.17] But again, those give me half the picture (and are on two different machines), and I'd need the long msgid line to correlate them. Is logging output configurable that I could add the value of the relay= line into the output? Or perhaps the value of X-Envelope-To? Also, does spamc have any concept of the short (ESMTP) messageid, as defined by sendmail's queues? (nBQDcLck027423). In terms of parsing logs, this is a much more useful correlation point, since it's that identifier that every other milter uses, and every other thing that writes to maillog uses. (But I understand if it's not possible since the API is different). For example, grepping for that self-same messageid, other than spamc, gives me the whole story. Sender, recipient, every milter it's been through. Dec 26 08:38:23 mail.info prime sm-mta[27423]: nBQDcLck027423: from=asterisk-users-boun...@lists.digium.com, size=2735, class=0, nrcpts=1, msgid=ca65a2f0912260536l2d0edcdy28f748beb74ea...@mail.gmail.com, proto=ESMTP, daemon=MTA, relay=lists.digium.com [216.207.245.17] Dec 26 08:38:23 mail.info prime sm-mta[27423
Adding remote-ip/ESMTPID/X-Envelope to logging output?
Hey there, Background: Sendmail with spamd running on a different box, spamc called from global procmail file. I'm doing some nightly log-combing to look for interesting patterns, including against other network traffic (like erroneous DNS lookups, I think I might be on to something). However, one of the annoying things about spamassassin's logging is that it fails to log the remote connecting ip, even though it places it in special places in the logs: take for example: Dec 26 08:41:51 quark spamd[87490]: spamd: connection from prime.gushi.org [72.9.101.130] at port 62430 Dec 26 08:41:51 quark spamd[87490]: spamd: processing message ca65a2f0912260536l2d0edcdy28f748beb74ea...@mail.gmail.com for danm:58 Dec 26 08:41:53 quark spamd[87490]: FuzzyOcr: Scan canceled, message has less than -5 points (-6.601). Dec 26 08:41:53 quark spamd[87490]: spamd: clean message (-6.6/5.0) for danm:58 in 1.9 seconds, 3788 bytes. Dec 26 08:41:53 quark spamd[87490]: spamd: result: . -6 - AWL,BAYES_00,RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS scantime=1.9,size=3788,user=danm,uid=58,required_score=5.0,rhost=prime.gushi.org,raddr=72.9.101.130,rport=62430,mid=ca65a2f0912260536l2d0edcdy28f748beb74ea...@mail.gmail.com,bayes=0.00,autolearn=ham,shortcircuit=no From those logs, there's nothing at all that tells me what the relaying ip is, even though it's special to spamd, used to determine the ASN, etc. The sendmail logs (grepped for that messageid) are more useful: Dec 26 08:38:23 mail.info prime sm-mta[27423]: nBQDcLck027423: from=asterisk-users-boun...@lists.digium.com, size=2735, class=0, nrcpts=1, msgid=ca65a2f0912260536l2d0edcdy28f748beb74ea...@mail.gmail.com, proto=ESMTP, daemon=MTA, relay=lists.digium.com [216.207.245.17] But again, those give me half the picture (and are on two different machines), and I'd need the long msgid line to correlate them. Is logging output configurable that I could add the value of the relay= line into the output? Or perhaps the value of X-Envelope-To? Also, does spamc have any concept of the short (ESMTP) messageid, as defined by sendmail's queues? (nBQDcLck027423). In terms of parsing logs, this is a much more useful correlation point, since it's that identifier that every other milter uses, and every other thing that writes to maillog uses. (But I understand if it's not possible since the API is different). For example, grepping for that self-same messageid, other than spamc, gives me the whole story. Sender, recipient, every milter it's been through. Dec 26 08:38:23 mail.info prime sm-mta[27423]: nBQDcLck027423: from=asterisk-users-boun...@lists.digium.com, size=2735, class=0, nrcpts=1, msgid=ca65a2f0912260536l2d0edcdy28f748beb74ea...@mail.gmail.com, proto=ESMTP, daemon=MTA, relay=lists.digium.com [216.207.245.17] Dec 26 08:38:23 mail.info prime sm-mta[27423]: nBQDcLck027423: Milter insert (1): header: X-DomainKeys: Sendmail DomainKeys Filter v1.0.2 prime.gushi.org nBQDcLck027423 Dec 26 08:38:23 mail.info prime sm-mta[27423]: nBQDcLck027423: Milter insert (1): header: Authentication-Results: prime.gushi.org; dkim=none (no signature)\n\theader.i=unknown; x-dkim-adsp=none Dec 26 08:38:23 mail.info prime sm-mta[27423]: nBQDcLck027423: Milter insert (1): header: X-DKIM: Sendmail DKIM Filter v2.8.3 prime.gushi.org nBQDcLck027423 Dec 26 08:38:23 mail.info prime sm-mta[27423]: nBQDcLck027423: Milter insert (1): header: Authentication-Results: prime.gushi.org; sender-id=pass header.sender=asterisk-users-boun...@lists.digium.com; spf=pass smtp.mfrom=asterisk-users-boun...@lists.digium.com Dec 26 08:38:23 mail.info prime sm-mta[27423]: nBQDcLck027423: Milter insert (1): header: X-SenderID: Sendmail Sender-ID Filter v1.0.0 prime.gushi.org nBQDcLck027423 Dec 26 08:38:24 mail.info prime sm-mta[27423]: nBQDcLck027423: Milter add: header: X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.0.1 (prime.gushi.org [72.9.101.130]); Sat, 26 Dec 2009 08:41:49 -0500 (EST) Dec 26 08:38:28 mail.info prime sm-mta[27436]: nBQDcLck027423: to=d...@prime.gushi.org, delay=00:00:05, xdelay=00:00:03, mailer=local, pri=33624, dsn=2.0.0, stat=Sent Thoughts? -Dan Mahoney -- When I'm lost, and confused, and trying to make a U-turn, nothing annoys me more than someone telling me to watch out for the tombstone! How often does that happen, Fab? -David Feld Tom Fabry, sometime in High School. Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: A rant about FUZZY_OCR
On Mon, 27 Apr 2009, Henrik K wrote: Nothing of this makes sense. If you don't have a test server, too bad. If you don't trust the score-changing values too bad. It all worked for me. It's a great idea, but I'd like to see it mature some first, especially with respect to its documentation, test emails, word list, and live testing. If was quickly developed to an ongoing problem. The problem disappeared years ago. It was mature enough for 99% of users at that time. Though it did add lots of complexity and stricter MTA rules etc handled the job just fine also. The problem exists now, there is PNG spam, and there will continue to be, because it gets through. Right now the only way I find this blocked is if spamcop blocks it. Ideally, what I'd probably like to see with regard to fuzzyOCR are: 1) Just patch it enough to work with 3.2 and 3.3 -- I don't have the internals know-how to do this, and I don't know if Decoder still reads this list. 2) A debug mode, whereby the plugin would note its own score, possibly by applying an equal negative value. 3) Wordlists loadable from userprefs, if not bayes. 4) A recommended configuration, along with shortcircuit documentation. -Dan -- Ca. Tas. Tro. Phy. -John Smedley, March 28th 1998, 3AM Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: A rant about FUZZY_OCR
On Mon, 27 Apr 2009, Jo Rhett wrote: On Apr 27, 2009, at 1:16 PM, Dan Mahoney, System Admin wrote: The problem exists now, there is PNG spam, and there will continue to be, because it gets through. Right now the only way I find this blocked is if spamcop blocks it. Just as a point of reference, I'd like to note that we haven't bothered with FuzzyOCR here and absolute none of the spam which reaches my inbox is a PNG or JPG or GIF spam. SA does block it, and it does so without FuzzyOCR. That said, we have jacked the scores for e-mail with images and no text and that might be why. We never, ever receive valid e-mail with no text in it. The spam I've been getting contains text, lots of it. Markov-chain like crap that is 100 percent nonrelevant to the image. -Dan -- She's NOT my girlfriend! -Dan Mahoney, Quite a bit recently. Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Image spam and failing rule
On Sat, 25 Apr 2009, John Hardin wrote: On Sat, 25 Apr 2009, Gary Forrest wrote: We are receiving the same image spam many times, random text within the body. FuzzyOCR. It seems Spammers are trying image spam again, after giving up on it for a year or so. Is there a version of FuzzyOCR that's actually supported with the current SA release? Or under active development at all? -Dan -- Man, this is such a trip -Dan Mahoney, October 25, 1997 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Code Rot?
Hey all, While there's a decent amount of spamassassin list traffic to imply otherwise, is the SA project falling dormant? the sare-rules claim they won't be updated due to lives, wives, and hockey. the fuzzyOCR project claims the only thing that works with 3.2 is the SVN version, and on the same page claims you shouln't really expect the SVN version to work. The wiki pages show the last release as almost a year ago, with no notice of any betas, pending releases, or whatnot. Many commercial products have happily used SA in their core offering, is that where the future of development is? -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: [sa-list] Re: [sa-list] Re: Spamd and ipv6
On Wed, 3 Dec 2008, SM wrote: At 18:23 02-12-2008, Byung-Hee HWANG wrote: Are you using FreeBSD or NetBSD? If so, i understand you. Unfortunately, SA developers do not care about IPv6 yet. So here SA program at first do action with 127.0.0.1 than ::1, i guess ;; This was tested on a BSD system. SpamAssassin developers are sharing their code for free.If we need a specific feature or find a bug, we can always send a patch. If you read the URL I posted previously, you will see that the developers have been working on IPv6 support. fwiw, I wasn't trying to sound abrasive, simply requesting that since the expected behavior is that if the behavior is that the client should try v6, then v4 -- that the server should have options to bind that way (assume I am running a spamd server that serves both v4 and v6 users). Right now I cannot multi-stack bind (is that being worked on?) or bind to multiple addresses (is that also being worked on?). Alternatively, there should be an flag in the client to control whether it connects on v4 or v6, and the default should be consistent with how the server functions by default. If v6 support in the server isn't done yet, then v4 should be the default. Of course, opening a bug on this won't help since it's slated for fixing and ostensibly already have bugs open. I suggested there might also be docbugs, but since this support is coming in the next release, amending the docs in the current version wouldn't help. I've found bug reports to be a *terrible* method of communication unless someone on a list who knows the product better than I says yeah, that's a bug, open one. -Dan -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Spamd and ipv6
On Fri, 5 Dec 2008, Dan Mahoney, System Admin wrote: Also, sorry about the subject headers. I think I've fixed my procmail recipe. -Dan -- I love you forever eternally. -Connaian Expression Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: [sa-list] Re: Spamd and ipv6
On Mon, 1 Dec 2008, SM wrote: At 23:01 30-11-2008, Dan Mahoney, System Admin wrote: So then, you're saying the behavior for ipv4 and ipv6 is somehow different? If you start spamd without specifying the IP addresses to listen on, spamd will listen on the 127.0.0.1 IP address only. And on an ip6 enabled system, where will spamc localhost try to connect to first? 127.0.0.1 or ::1? You should have the IO::Socket::INET6 and Socket6 Perl modules installed to have IPv6 support in spamd. I have both modules present: quark# perl -e 'use IO::Socket::INET6' quark# perl -e 'use Socket6' You can start spamd as follows: spamd -i 2001:DB8:1:1::1 Yes, but there's no way to listen on *both* addresses -- however, it's completely possible to listen on all ip4 addresses -- I'm just looking for a switch that will say all ip4 AND all ip6. Also, would be useful if I could specify to listen on :: or [::] (neither worked when I tried it.) Again, consistent behavior between v4 and v6 is what I'm looking for. spamd only allows connections from 127:0.0.1. You can allow connections from other IP addresses with the -A parameter. You may have to patch Mail::SpamAssassin::NetSet. See https://issues.apache.org/SpamAssassin/show_bug.cgi?id=4964 Additionally, even when I get this working, I am unable to specify ipv6 addresses to -A, either with or without square brackets. That part of the code is IPv4 specific. Listening on v6 is pointless if I can't restrict. Is the correct answer open another bug? Or from these commit messages, should I simply assume the next 3.3 will have these (I see jm's note that the patches shouldn't cleanly apply to 3.2.x.)? As stated, I've fixed this (for now) by changing my spamc args to have the v4 address. -Dan -- Your future hasn't been written yet; no one's has. So make it a good one! -Doc Emmet L. Browne, Back to the Future III Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Spamd and ipv6
Since getting my hosts natively speaking ipv6, I've been seeing a lot of initial timeouts connecting to spamc, because I believe it's apparently trying ipv6 first. spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 3): Connection refused spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 3): Connection refused spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 3): Connection refused spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 3): Connection refused spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 3): Connection refused spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 3): Connection refused spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 3): Connection refused spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 3): Connection refused However, I cannot get the -A systax for spamd to accept connections from a given address, nor does it appear to be listening on said address: quark# netstat -na | grep LIST | grep 783 tcp4 0 0 *.783 *.*LISTEN I'm running a recent enough version that v6 *should* be supported. Versions: SpamAssassin Server version 3.2.5 running on Perl 5.8.8 with SSL support (IO::Socket::SSL 1.13) with zlib support (Compress::Zlib 2.008) Any ideas? -- I can feel it, comin' back again...Like a rolling thunder chasin' the wind... -Dan Mahoney, JS, JB SL, May 10th, 1997, Approx 1AM Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Spamd and ipv6
On Sun, 30 Nov 2008, SM wrote: At 21:45 30-11-2008, Dan Mahoney, System Admin wrote: Since getting my hosts natively speaking ipv6, I've been seeing a lot of initial timeouts connecting to spamc, because I believe it's apparently trying ipv6 first. spamc: connect to spamd on 2001:470:1f07:a7f::1 failed, retrying (#1 of 3): Connection refused [snip] However, I cannot get the -A systax for spamd to accept connections from a given address, nor does it appear to be listening on said address: quark# netstat -na | grep LIST | grep 783 tcp4 0 0 *.783 *.*LISTEN Use the -i parameter to specify the IPv6 address. The -A parameter to specify the host which can connect to spamd and not the IP address on which spamd should listen on. So then, you're saying the behavior for ipv4 and ipv6 is somehow different? I am starting spamd with -i but no ip specified, according to the docs: If you specify no IP address after the switch, spamd will listen on all interfaces. (This is equal to the address 0.0.0.0). All Interfaces != 0.0.0.0 At the very least, this is a docbug and should be amended to say all ipv4 interfaces. No mention is made of whether or not multiple -i arguments can be specified, but from my research, only the first -i is used, and you cannot comma-separate. This is a second docbug, or a functionality that should be added to listen on v4 and v6 simultaneously. Additionally, even when I get this working, I am unable to specify ipv6 addresses to -A, either with or without square brackets. Behaviorally, spamc *tries v6 by default* but spamd requires hoop-jumping. This is a consistency problem and should also be looked into. V6 is coming, fast. Things like this are worth chasing down. Let me know if you need me to run any other debugs or anything. If you need access to my systems, please just say the word. I like having something to offer in the solution of a problem, other than just complaints :) -Dan -- Man, this is such a trip -Dan Mahoney, October 25, 1997 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Blogger URLs
Hello all, A lot of the spam I'm seeing sneak past spamassassin has a blogger url in it (this seems to be a new favorite for spammers). I've got about 200 such spams that have managed to sneak past (no idea how many of the 2 spams in my confirmed-kills folder also match). So, that said: Can someone do a spam-versus-ham comparison for included links to blogger.com (I don't have the corpus handy, nor do I know how to set up a proper test.) If it proves high enough, would a rule be possible? Also, would it be possible to make spamassassin -r smart about reporting such links straight to the feedback form here: http://help.blogger.com/?page=troubleshooter.csproblem=ItemType=spamcontact_type=SpamSubmit=Continue -Dan -- Be happy. Try not to hurt each other. Hope you fall in love. --Mallory, Family Ties Finale (on the meaning of life) Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Blogger URLs
On Sun, 20 Apr 2008, Theo Van Dinter wrote: On Sun, Apr 20, 2008 at 12:39:29PM -0400, Dan Mahoney, System Admin wrote: Can someone do a spam-versus-ham comparison for included links to blogger.com (I don't have the corpus handy, nor do I know how to set up a proper test.) It's not really going to help you, you'd need to know the #s for your mail flow. Okay, so presumably then -- in my normal mail flow, there were all of six -- and those were with a client who was specifically giving me the URL to point her site at it. Is there a tool, with the standard SA distribution, that can let me do a comparison analysis? I found some tools here http://wiki.apache.org/spamassassin/StatsAndAnalyzers that gives me a whole bunch of after-the-fact info (i.e. on a live pool), but not something to let me say okay, here's a defined rule, find all the X's and Y's (but I'm sure the SA team has something for such). Didn't find anything in the rule submission guidelines either. If it proves high enough, would a rule be possible? Sure, go ahead, it's your setup. :) Also, would it be possible to make spamassassin -r smart about reporting such links straight to the feedback form here: http://help.blogger.com/?page=troubleshooter.csproblem=ItemType=spamcontact_type=SpamSubmit=Continue You could write a plugin to do it, but generally this is what spamcop is for imo. I've found spamcop to be a problem for two reasons: 1) It's an ANNOYING process. And in theory I could automate it, but that circumvents the whole idea. 2) A plugin specifically targeted for blogger could check for the standard error messages -- not report if so, etc etc. I've also had issues with spamcop not following the links right (for example, a popular ploy is to load the IMAGES in spam from other sources which SC doesn't follow) The possibility of catering the reporting protocols to different sites (i.e. the major free sites have their own reporting systems that might be better used). It's beyond the scope of this thread, but are there any docs on how to write a reporting protocol? -Dan -- Randomly Selected Tagline: Everyone looks like they're wearing the game board from Sorry. - Dennis Miller -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
[no subject]
-- Man, this is such a trip -Dan Mahoney, October 25, 1997 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: RCVD_IN_DNSWL_LOW
On Fri, 26 Oct 2007, Matthias Leisi wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alex Woick schrieb: [Spamcop] I understand the two step reporting process too, and I too find it annoying and timeconsuming to ack my (manually reviewed) 50 spams per day to them, so I ceased to do it. There exist scripts for ack'ing automatically, but this is not the intention of this process, so this is no alternative for me. I don't speak for Spamcop, but I do speak for dnswl.org. From our experience I can tell that a manual review process is very important to ensure data quality. At least in the context of dnswl.org, there is little value in reporting for the sake of reporting alone -- there needs to be some quality control involved, or otherwise we run a high risk of including unwanted IP addresses. Having said that, we of course welcome all reports on false positives, especially on IP addresses with a low, med or hi score, and we welcome all notifications of mailservers we do not yet know about. It's rather simple, really. If I'm auto-reporting spams with a score of (let's say, 15...enough that regardless of the DNSWL score's negative it would still be enough to auto-learn as spam to DNSWL (and DNSWL is passing complaints onto the original mailserver, which seems a logical thing) this serves as a reminder to the original mail server (let us say, in this case, two things). This is the kind of thing that I would suggest be an enhancement to SA (but off by default for privacy reasons), on the spamd side, at the same time as bayes auto-learning happens. 1) That they are sending spam that risks their whitelist rating. and 2) That the email they are sending is probably too spammish ANYWAY, if it's of a high enough threshhold ABOVE the DNSWL score to still be reported. If you are a spammer, this allows you not only to listwash, but also to scrub and detail your email so it hits less SA rules -- of course, if you are any kind of pro spammer, presumably you are running your mails through at least a standard SA install anyway to test them. If on the other hand you are a legitimate user of this service, *and* you are a producer of regular volumes of email, locally originated, that has some spammish tendencies (badly formed HTML parts, or being sent by a non-malicious script, then it allows you to correct other means of those false positive. Naturally, if DNSWL isn't reporting back to the mailserver user, none of the above applies. Manually reporting, on the other hand, is something that I would tie into the spamassassin -r functions, and much LIKE spamcop or the others, I'd suggest one or two extra pieces of data: Some kind of a reporting ID, which determined the severity of the report (i.e. anonymous reports were given less credence). And if the reports were going to be given back to the original mailserver again, some option to have the identifying data stripped. Also, the ability to view the number of reports for a given server helps as well. -Dan - -- Matthias -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFHIggQxbHw2nyi/okRAludAKC14sT7Ff3Ax4L9zpC/fWHx/xyUAwCfSUZ1 WB4q6mV08fa4Yhyx+aUtbEs= =3yG4 -END PGP SIGNATURE- -- Amerikanskaya firma Transceptor Technology pristupila k poizvodstu komputerov Personal'ni Sputnik Translates as: 'American company Transceptor Technology commenced the production of the computer personal sputnik' --Snap, The Power Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: RCVD_IN_DNSWL_LOW
On Wed, 17 Oct 2007, ram wrote: Sorry I meant like spamcop .. I think I must proof-read my own mail now before Ctrl-Enter :-) The problem with SpamCop is: the two step reporting process makes things a bear to do. I understand the logic behind it, but once or twice I've taken a couple hundred spam emails and spamassassin -r'd it...annoying as hell. I'd like it if they open-sourced their analysis engine so people could use it to report spam privately, but I know it's not happening. -Dan -- there is no loyalty in the business, so we stay away from things that piss people off -The Boss, November 12, 2002 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Rule for TLS verify=OK?
Hey all, In looking through my sendmail logs, I've found that some connecting mail servers actually are correctly configured with a signed, valid cert from one of the major CA's. Is there a rule that can match this, on sendmail, based on the connecting ip on your network edge? This could be used to complement domain-assurance tools like SPF, DKIM or the like, since it not only matches the fact that in order to get one of these certs, the domain owner has had to match at least SOME kind of legitimacy test (even with the most automated signers). This is a length I cannot imagine a spammer going to. Better still, can someone with a better corpus than I confirm some hit/not hit ratios here? -Dan -- unless is a pr0no book he wont even come close to the bandwidth quota -Racer-X, concerning DanMahoney.com's web hits. Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: RCVD_IN_DNSWL_LOW
On Wed, 17 Oct 2007, Henrik Krohns wrote: On Tue, Oct 16, 2007 at 06:16:49PM -0400, Dan Mahoney, System Admin wrote: dnswl.org is either full of it, or not well maintained. I've gotten at least 20 spams which I see are listed in dnswl.org as low trust (which still merits -1.0). Umm, did you actually read their pages? Low Occasional spam occurrences, actively corrected but less promptly. My point was more along the lines of the fact that there's no method (other than manual notification) of doing Active Correction. DNSWL is a cool idea, but could we also come up with some sort of reporting plugin (disabled by default, optional) that could notify them when, say, a spam of score 15 or above also hits their rules. If you dont like it, change the scores. Why not change the system? -Dan -- Why are you wearing TWO grounding straps? -John Evans, Ezzi Computers August 23, 2001 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: RCVD_IN_DNSWL_LOW
On Wed, 17 Oct 2007, Matthias Leisi wrote: I forwarded over 200 of them earlier today (as an attachment -- total email size was about one meg). It would have been from this address. -Dan -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dan Mahoney, System Admin schrieb: dnswl.org is either full of it, or not well maintained. I've gotten at least 20 spams which I see are listed in dnswl.org as low trust (which still merits -1.0). All different IP addresses or some specific network? Could we maybe please add a feature to spamassassin -r (or some other hook to the generic whitelisting code) which reports this to the appropriate whitelist owner? Can you forward such false positives to admins -at- dnswl.org, please? Thanks, - -- Matthias, for dnswl.org -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFHFa31xbHw2nyi/okRAueXAJ9v7bs40kAz4UEry7dCKxYqWVnWFwCgjte/ N/CrJ3V4V3X1H+jkGhf/nb8= =kIQd -END PGP SIGNATURE- -- Oh, and we just recently got an invoice... Congratulations! -JC and DM, regarding Unpredictable Billing, 8/18/2001 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: [sa-list] Re: RCVD_IN_DNSWL_LOW
On Wed, 17 Oct 2007, Henrik Krohns wrote: On Wed, Oct 17, 2007 at 02:48:49AM -0400, Dan Mahoney, System Admin wrote: On Wed, 17 Oct 2007, Henrik Krohns wrote: On Tue, Oct 16, 2007 at 06:16:49PM -0400, Dan Mahoney, System Admin wrote: dnswl.org is either full of it, or not well maintained. I've gotten at least 20 spams which I see are listed in dnswl.org as low trust (which still merits -1.0). Umm, did you actually read their pages? Low Occasional spam occurrences, actively corrected but less promptly. My point was more along the lines of the fact that there's no method (other than manual notification) of doing Active Correction. Sure, I just felt like being rude also. ;) You say at least 20 spam, but since it depends on what your total traffic is, it doesn't mean much. Actually, that was a typo, of sorts...a more accurate metric would be: Over 200 hits on that rule, with spams mostly over scores of ten, since October 8th, with total spam volume ( 5) about 1000. Or...roughly 1/5 to 1/4 of all the spam in the past couple weeks. -Dan -- Is Gushi a person or an entity? Yes -Bad Karma, August 25th 2001, Ezzi Computers, Quoting himself earler, referring to Gushi Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: RCVD_IN_DNSWL_LOW
On Wed, 17 Oct 2007, Matthias Leisi wrote: I forwarded over 200 of them earlier today (as an attachment -- total email size was about one meg). OK, I now could have a look at them (well, a sample of them, not each of the 200 individually). All samples in that set have been forwarded through your livejournal.com account, and consequently sent to your server through a dnswl.org-listed server of livejournal.com (204.9.177.18, see http://www.dnswl.org/search.pl?s=1409). Livejournal's purely a mail forwarding service (i.e. there's no way to POP/IMAP that account) and if they can't effect proper controls on how mail is sent through them, then they shouldn't be trusted at all. On my end, I have degrees of control (false MXes, Blacklists, whitelists, greylists, sender callbacks, etc). I have no such control over the LJ MX'es. I've proposed a reporting plugin on the sa-users list, that allows (both for yourself, as well as other whitelists) for the list-owner to be notified with details of high-spam activity (at which point, I guess, you guys could pass that on to your whitelisted groups, and/or adjust categories accordingly. Please configure your trusted_networks/internal_networks -- like that, Like what? I think I missed what you want me to do. you'll even get the benefit that all RBL lookups, whitelist_from_rcvd etc. profit from the correct information. -Dan -- The first annual 5th of July party...have you been invited? It's a Jack Party. Okay, so Long Island's been invited. --Cali and Gushi, 6/23/02 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: RCVD_IN_DNSWL_LOW
On Wed, 17 Oct 2007, Matthias Leisi wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dan Mahoney, System Admin schrieb: Livejournal's purely a mail forwarding service (i.e. there's no way to POP/IMAP that account) As far as I know, there are mails originating from LJ itself (eg notifications etc)? No, Livejournal also gives you a [EMAIL PROTECTED] email address. Yes, they do also originate mail (for which we have things like SPF (which they do), DomainKeys, DKIM (which they don't, and in fact they may have an error for) -- as well as some of the more esoteric things like HashCash, GnuPG-signing, etc etc.) and if they can't effect proper controls on how mail is sent through them, then they shouldn't be trusted at all. On my end, I have degrees of control (false MXes, Blacklists, whitelists, greylists, sender callbacks, etc). I have no such control over the LJ MX'es. Correct. But by setting (in your local.cf or equivalent) | trusted_networks 204.9.177.18 you are telling SpamAssassin that this relay is not operated by a spammer and that it should apply all black-/whitelist rules etc. to the IP address one more hop away. Then, in the context of SpamAssassin, you regain full control of connection-oriented rules. interesting point, I suppose. Kinda breaks the logic of trusted networks. On the same note, would it not be more useful to, instead of using the static trusted_networks configuration, to use the DNSWL to determine if that logic should be in play? Or some kind of database of known forwarding services that work in such a manner? That's not fully equivalent to having the actual spamming connection to deal with, but as close as it gets -- if you need it closer, you should not use forwarding services. Forwarding services are edge case in spamfiltering. Usually, such a service is itself perfectly trustworthy and not the actual source of spam, and care must be taken not to unduly penalize these services for forwarded spam. The problem therein lies in the fact that LJ notifications (comment notifications, friendslist notifications, account verification emails, etc) are passed through the exact same MXes as the [EMAIL PROTECTED] forwarding service. I've proposed a reporting plugin on the sa-users list, that allows (both for yourself, as well as other whitelists) for the list-owner to be notified with details of high-spam activity (at which point, I guess, you guys could pass that on to your whitelisted groups, and/or adjust categories accordingly. As I've answered before: That's already on the todo list. However, the main problem is not the plugin per se (technically, that is rather simple), but identifying trustworthy submitters. I suppose that depends on what we submit. If it's something verifiable (like, messageID:originating ip:spam level, it's easy). Just as with spamcop, one can choose to omit the message-id so that the spammers cannot track who is the spamtrap and listwash, but such reports could be given a lower precedence. -- You're a nomad billygoat! -Juston, July 18th, 2002 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: RCVD_IN_DNSWL_LOW
On Wed, 17 Oct 2007, Alex Woick wrote: Matthias Leisi schrieb am 17.10.2007 09:46: Correct. But by setting (in your local.cf or equivalent) | trusted_networks 204.9.177.18 you are telling SpamAssassin that this relay is not operated by a spammer and that it should apply all black-/whitelist rules etc. to the IP address one more hop away. Then, in the context of SpamAssassin, you regain full control of connection-oriented rules. That's not fully equivalent to having the actual spamming connection to deal with, but as close as it gets -- if you need it closer, you should not use forwarding services. Good point. I think I start to understand what trusted_network is for and how it works. Currently, I have a provider whose MX receives mail for me and forwards it to my local mail server. Spam detection improved much when I added its IP address to trusted_networks some time ago. Now, I occasionly get spam to my users.sourceforge.net account, just like Dan Mahoney is getting spam to his Livejournal account. Sourceforge is also listed with LOW at dnswl and acts as a forwarder to my own mail server. Since I never get spam from users.sourceforge.net accounts directly but only spam sent to my users.sourceforge.net account from random addresses, I suppose the Sourceforge mail server is trusted in that way that spam doesn't originate from it, and that's the purpose of trusted_network. Just like my Provider forwarding mail to me sent from random originators, but never produces spam itself. Sure, but that means each person who is a member of one of these services has to: * Look up their forwarded email address * Look up the SPF record for that domain -or- * Take a best guess as to the fact that the receiving MX will also be the sending. THEN * Translate that into trusted networks statements, which are GLOBALLY trusted (either per server or per used, but NOT per envelope-recipient) -- which is fine for Livejournal or Sourceforge, I guess, I'd imagine their MXes are pretty dedicated, but I'm sure there's smaller cases. But it might help to have some series of dynamic rule...whereby an address is DNSWL'd with a special code that lists it as a known relay for certain domains, and the trusted_networks logic extends automatically (if the relaying domain matches). Apologies if I've repeated anything already said. -Dan -- there is no loyalty in the business, so we stay away from things that piss people off -The Boss, November 12, 2002 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
RCVD_IN_DNSWL_LOW
dnswl.org is either full of it, or not well maintained. I've gotten at least 20 spams which I see are listed in dnswl.org as low trust (which still merits -1.0). Could we maybe please add a feature to spamassassin -r (or some other hook to the generic whitelisting code) which reports this to the appropriate whitelist owner? -Dan Mahoney -- Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
A compound bounce/(spf/dk/dkim) rule I'd like to see.
In pseudocode... IF (message is a recognizable bounce || message is from )... AND (we can guess the domain being sent to (can't trust the to header, but maybe the X-Envelope-To or some MTA token?) AND the domain being sent TO supports SPF and/or DKIM...(i.e. implying a misdirected bounce) Score a compound rule hit. My logic here is that I would eventually like to compile an rfc-ignorant list of the senders of such bounces, and aid them in not SENDING such bounce messages, or at the very least, set up a ruleset in the future to block bounces from them, based on a low signal/noise ratio. I am not trying at all to claim that this should be something SCORABLE, immediately: I don't think SA's detection of legitimate bounce messages versus illegitmate bounce messages is good enough (please feel free to tell me differently). -Dan Mahoney -- GO HOME AND COOK!!! Donielle Cocossa, Taco Bell, 2:30 AM Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] RE: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)
On Wed, 10 Oct 2007, Bret Miller wrote: sa-update does NOT feed a local blocklist generated by *my* particular corpus of spam emails. Think of it as the RBL equivalent of sitewide-bayes. Or think of it as a way of SA saying when I get twelve spams of score 10+ from ip 208.23.118.172...I will feed the auto-expiring RBL, which *SENDMAIL* works off of, thus keeping my *SPAMASSASSIN* load lower. How do you call SpamAssassin? If whatever calls SpamAssassin in your setup knows what IP the connecting relay has, it can hopefully also do what you describe above. SpamAssassin doesn't really need to support this (through plugins or anything else) for it to be possible (and feasible). And I did something very similar as well. The problem I found is that you need a very large white list to avoid blocking big ISPs for a sudden flood of spam. I ended up rejecting legitimate email far too often from the temporary block. I still like the idea and would do it in a second if I could change the 5xx reject to a 4xx try later type of block. But I can't' without switching to a different MTA. milter-greylist lets me do this (reject 4XX based on a DNSBL). I've found it to be highly customizable, if not a bit of a memory pig. On the other hand, if there is a big ISP who is sending me spam...should they not be blocked, anyway? -Dan -- Long live little fat girls! -Recent Taco Bell Ad Slogan, Literally Translated. (Viva Gorditas) Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Advice on MTA blacklist
On Wed, 10 Oct 2007, David B Funk wrote: On Tue, 9 Oct 2007, Jo Rhett wrote: On Oct 9, 2007, at 4:22 PM, Chris Edwards wrote: Your server then enforces encryption and SMTP-AUTH, and the SSL will (hopefully) defeat any man-in-the-middle attacks by trans-proxies. That's exactly the problem I am reporting. A lot of mail clients don't enforce SSL connections, so man in the middle is silently accepted. Only T-bird can be configured to not work any other way, TTBOMK. Jo you didn't read Chris's statement closely. A conscientious mail server administrator will configure the SERVER to -ONLY- accept encrypted connections for SMTP-AUTH transactions; the server should enforce the encryption requirements. Thus it does not matter what the client wants to do, the server should not let the client continue the SMTP-AUTH transaction until it has completed the STARTTLS operation (or in the case of SMTPS, it's already encrypted). Back to Skip's question, possibly the easiest way to solve his problem would be to run two SMTP servers, one on port 25 with full spam/AV scanning for regular mail traffic, one on ports 587 645 with SMTP-AUTH/TLS for his users' clients to submit messages, on that one have AV scanning and possibly limited spam scanning. Assuming sendmail (and we don't make such assumptions), you can specify different options per-port, such that you don't need to run two mail servers. For example, I have no less than seven virtual daemons configured: Submission agents on 587 and 2525, which require auth, and have encryption optional. Also listens on 127.1. A submission agent on 465 (not 645), configured the same way, but with encryption explicit. Standard daemon on port 25 (and yes, it still supports the optional encryption). As a bonus, my own server any port will present a FQDN, signed certificate (not self-signed). I've actually found other servers out there in the wild that do the same, with a valid cert -- I've got my server configured with the CA root certs so it knows which are true (this doesn't affect ability to relay or anything, but it's cool to see others are doing it). Of course, all this is wildly off the topic, but hey... -Dan -- And, a special guest, from the future, miss Ria Pischell. Miss Pischell, as you all know, is the inventor of the Statiophonic Oxygenetic Amplifiagraphaphonadelaverberator, and it's pretty hard to imagine life without one of those. -Rufus, Bill Ted's Bogus Journey Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)
On Tue, 9 Oct 2007, Steven Kurylo wrote: Or think of it as a way of SA saying when I get twelve spams of score 10+ from ip 208.23.118.172...I will feed the auto-expiring RBL, which *SENDMAIL* works off of, thus keeping my *SPAMASSASSIN* load lower. Thus a spam deluge via a dictionary attack that may take hours is mitigated in the course of X number of mails. I already do something similar, but I haven't bothered to take it quite that far yet. I use fail2ban to parse my exim logs. If an IP address hits more than 5 invalid accounts in 5 minutes, the IP is banned (fail2ban uses iptables) for 24 hours. As well if an IP address, which is listed on spamhause, hits me more than twice in 5 minutes it is banned for 24 hours. Granted neither of these cases usually end up getting messages as far as spamassassin. I've managed to drastically reduce the amount of simultaneous connections using this method; which was overloading the server. The next step would be to add the when I get twelve spams of score 10+ from [...] parsing. Though I hadn't thought of trying my hand at a SA plugin, I may do that. Parsing the SA logs would be easy, but the connecting IP isn't listed there. -Dan -- Man, this is such a trip -Dan Mahoney, October 25, 1997 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)
On Tue, 9 Oct 2007, Steven Kurylo wrote: Parsing the SA logs would be easy, but the connecting IP isn't listed there. As I mentioned, I'm parsing exim's logs. It contains the spam score and the IP address. Oh, that's true enough. I was musing on parsing my own logfiles as opposed to plugins. Not enough info since I'm rejecting at the procmail level, not the MTA (sendmail) level. -Dan -- Ca. Tas. Tro. Phy. -John Smedley, March 28th 1998, 3AM Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)
On Mon, 8 Oct 2007, Matus UHLAR - fantomas wrote: On Sat, 6 Oct 2007, Rob McEwen wrote: FWIW... that IP, 220.226.197.15, is currently listed on four spam blacklists (RBLs): 1) uceprotect 2) no-more-funn 3) psbl 4) ivmSIP.com (mine) On 07.10.07 05:55, Dan Mahoney, System Admin wrote: My problem is: blocklists come and go, and some blocklists, when they go, do things like hang up because they're being flooded, thus slowing my mail processes or flag all mail as spam or hand out stale data that hasn't changed at all in months/years. That's what sa-update is for. Personally, I'd like it if SA came with a blocklist-feeder tool, where upon, say, two auto-learns, a blocklist (or SQL database) could be fed. Why do you think people would use them, when they don't already use sa-update which does the same? sa-update does NOT feed a local blocklist generated by *my* particular corpus of spam emails. Think of it as the RBL equivalent of sitewide-bayes. Or think of it as a way of SA saying when I get twelve spams of score 10+ from ip 208.23.118.172...I will feed the auto-expiring RBL, which *SENDMAIL* works off of, thus keeping my *SPAMASSASSIN* load lower. Thus a spam deluge via a dictionary attack that may take hours is mitigated in the course of X number of mails. Which is what I was (off-topicly) asking for, -Dan -- I'll commit ritual suicide before I whore myself out to Disney. --Emi Bryant April 26, 2004 On the animation industry Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Auto-RBL was: Why did this not hit more? (SPF, DKIM, Ironport,
On Mon, 8 Oct 2007, Rob McEwen wrote: Therefore, I recommend that you re-think your choices here! Don't let your quest for guaranteed long-term perfection keep you from making **substantial** progress today! Rob, Then help rally the SA team to include those RBLs that you mentioned in the stock config. Also, rally them to update the documentation on the wiki on how to configure SA for third-party DNSBL's, because it blows (and refers to years-old versions of SA). Yes, I know the point of a wiki is that ANYONE can update it, but I'm not about to update it with information I don't understand for certain. ((Q: This documentation doesn't seem to cover how to configure dns-blocklists. It says Support for these is built-in but I can't believe that all free BL's is called each time a mail is beeing checked. There must be a way to configure which to use. A: You're right. You might look at the [WWW] Mail::SpamAssassin::Conf documentation page which I admit doesn't really say how to configure which DNSBL to use, or the rules file [WWW] 20_dnsbl_tests.cf, for internal details, but no clear examples of how to configure the inclusion of various DNSBLs either. For the latest list of DNSBLs you want to be using SpamAssassin version 2.63 or 3.0.0-pre2, for the same reason that you wouldn't use an out-of-date virus scanner, but that also doesn't really have anything to do with the question.)) Finally, rally them to pay attention to the topic I'm proposing here, which is: allow users to run their own RBL + feeder so that they can auto-rbl and floodgate themselves (and yes, it allows me to combine your corpus, plus my corpus, plus HIS corpus) in a scoring config, which is FUN...or it lets you say, quite simply SA said you sent too much spam, now sendmail won't listen for X hours per spam run. soapbox While I've had a long history of getting decent responses from the developers on this list some of the time -- nobody has managed to answer the questions I've asked in the previous thread: * can we do something with the ironport headers * can we do something with the SPF softfail which my MTA registered but SA didn't (and why didn't it?) * can we do something with the X-Originating-IP: 127:1 (is it a legit header, or is it there to evade filters?) * can we fix something about the DKIM_POLICY_SIGNSOME, * and after I changed the topic: Can we get a plugin that lets us feed our own blocklists, currently I get dictionary floods that are enough to overload SA (even right now). and many is the time I've just sent an email out to this list on a given topic, seen a lack of useful answer, and shrugged it off. /soapbox -- Check it out, it's just like Christmas. Except it sucks. -Jason Seguerra, 3/2/05 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)
On Sat, 6 Oct 2007, Rob McEwen wrote: Dan, FWIW... that IP, 220.226.197.15, is currently listed on four spam blacklists (RBLs): 1) uceprotect 2) no-more-funn 3) psbl 4) ivmSIP.com (mine) My problem is: blocklists come and go, and some blocklists, when they go, do things like hang up because they're being flooded, thus slowing my mail processes or flag all mail as spam or hand out stale data that hasn't changed at all in months/years. If you put out a popular enough blocklist, you're likely to be blocked, period. Personally, I'd like it if SA came with a blocklist-feeder tool, where upon, say, two auto-learns, a blocklist (or SQL database) could be fed. The docs here: http://wiki.apache.org/spamassassin/DnsBlocklists?highlight=%28dnsbl%29 Are outdated. -Dan The first two are FP-risky for outright blocking, but can be useful in a scoring environment. The latter two are much more safe for outright blocking... particularly ivmSIP.com, which a FP rate that is almost low as the FP rate of SpamHaus's lists. Rob McEwen Dan Mahoney, System Admin wrote: Message at bottom. I checked on this email. My system is right: it is an spf soft-fail. At this point, ninety nine percent of people who set up SPF are going to be setting ~all and not understanding the difference between ~all and -all. And this did constitute a fail (i.e. a forgery), but there's no rule that hit. We've had the debate before, that SPF alone should not stop spam, but here it is: a legitimate domain hijack and SA isn't hitting? Also, what's up with RDNS_NONE? My sendmail won't accept a connection unless your RDNS resolves, or you send in the domain literal format. I did a quick search and found a few bugs on this. We've already been over DKIM_POLICY_SIGNSOME -- I'm still in favor of making a new rule for the implicit policy (DKIM_NOPOLICY or DKIM_POLICY_ASSUMED_SIGNSSONE) rather than the explicit one. Can we also assume the following... The Ironport-Anti-Spam score is bogus but we have no way of checking the result? The Ironport-AV score is probably also bogus? Are valid values for i and a documented somewhere? The X-Originating-IP of 127.0.0.1 is probably accurate (after all, the sending host must have had a 127.1), but useless and either the result of a bug (i.e. a misconfigured mailserver, from which we should not accept), or an intentional attempt to fool filters to believe it's trusted (for those systems that check this header) and should be ignored or a rule created? From [EMAIL PROTECTED] Sat Oct 6 05:40:56 2007 Return-Path: [EMAIL PROTECTED] X-Spam-Checker-Version: SpamAssassin 3.2.2 (2007-07-23) on quark.gushi.org X-Spam-Level: * X-Spam-Status: No, score=1.4 required=5.0 tests=BAYES_50,DKIM_POLICY_SIGNSOME, MISSING_HEADERS,RDNS_NONE autolearn=no version=3.2.2 Received: from rx4.indiatimes.com ([220.226.197.15]) by prime.gushi.org (8.13.8/8.13.8) with ESMTP id l969eqTG063292 for [EMAIL PROTECTED]; Sat, 6 Oct 2007 05:40:54 -0400 (EDT) (envelope-from [EMAIL PROTECTED]) Authentication-Results: prime.gushi.org [EMAIL PROTECTED]; sender-id=softfail; spf=softfail Received: from unknown (HELO tilmb7.indiatimes.com) ([192.168.61.27]) by x1.indiatimes.com with ESMTP; 06 Oct 2007 15:07:38 +0530 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AnoUAJL0BkfAqD0b/2dsb2JhbAAMiRw X-IronPort-AV: i=unknown; a=17144176:sNHT0 Date: Sat, 6 Oct 2007 14:57:11 +0530 (IST) From: Mr.Craig McAfee [EMAIL PROTECTED] Reply-To: Mr.Craig McAfee [EMAIL PROTECTED] Message-ID: [EMAIL PROTECTED] Subject: Attn:YOU HAVE WON A PRIZE (1,700,000.00 Euros)! MIME-Version: 1.0 X-Originating-IP: [127.0.0.1] Content-Type: text/plain; charset=utf-8 X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-3.0 (prime.gushi.org [0.0.0.0]); Sat, 06 Oct 2007 05:40:56 -0400 (EDT) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by prime.gushi.org id l969eqTG063292 X-Envelope-To: [EMAIL PROTECTED] [ The following text is in the utf-8 character set. ] [ Your display is set for the US-ASCII character set. ] [ Some characters may be displayed incorrectly. ] Attention!!! Your email address has emerged as one of the winner in Euromillions FreeDraws.Prize attached is 1,700,000.00 Euros.Contact Mr Mr Denis Ernest Fing.Email:[EMAIL PROTECTED] with the following information:1, Full Names: 2. Address:3. Age:4. Sex:5. Phone /Fax number: and 6. Country: -- My life has changed. What about yours? Log on to the new Indiatimes Mail and Live out of the Inbox! -- Is Gushi a person or an entity? Yes -Bad Karma, August 25th 2001, Ezzi Computers, Quoting himself earler, referring to Gushi Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org --- -- Of course she's gonna be upset! You're dealing with a woman here Dan
Why did this not hit more? (SPF, DKIM, Ironport, X-originating-ip)
Message at bottom. I checked on this email. My system is right: it is an spf soft-fail. At this point, ninety nine percent of people who set up SPF are going to be setting ~all and not understanding the difference between ~all and -all. And this did constitute a fail (i.e. a forgery), but there's no rule that hit. We've had the debate before, that SPF alone should not stop spam, but here it is: a legitimate domain hijack and SA isn't hitting? Also, what's up with RDNS_NONE? My sendmail won't accept a connection unless your RDNS resolves, or you send in the domain literal format. I did a quick search and found a few bugs on this. We've already been over DKIM_POLICY_SIGNSOME -- I'm still in favor of making a new rule for the implicit policy (DKIM_NOPOLICY or DKIM_POLICY_ASSUMED_SIGNSSONE) rather than the explicit one. Can we also assume the following... The Ironport-Anti-Spam score is bogus but we have no way of checking the result? The Ironport-AV score is probably also bogus? Are valid values for i and a documented somewhere? The X-Originating-IP of 127.0.0.1 is probably accurate (after all, the sending host must have had a 127.1), but useless and either the result of a bug (i.e. a misconfigured mailserver, from which we should not accept), or an intentional attempt to fool filters to believe it's trusted (for those systems that check this header) and should be ignored or a rule created? From [EMAIL PROTECTED] Sat Oct 6 05:40:56 2007 Return-Path: [EMAIL PROTECTED] X-Spam-Checker-Version: SpamAssassin 3.2.2 (2007-07-23) on quark.gushi.org X-Spam-Level: * X-Spam-Status: No, score=1.4 required=5.0 tests=BAYES_50,DKIM_POLICY_SIGNSOME, MISSING_HEADERS,RDNS_NONE autolearn=no version=3.2.2 Received: from rx4.indiatimes.com ([220.226.197.15]) by prime.gushi.org (8.13.8/8.13.8) with ESMTP id l969eqTG063292 for [EMAIL PROTECTED]; Sat, 6 Oct 2007 05:40:54 -0400 (EDT) (envelope-from [EMAIL PROTECTED]) Authentication-Results: prime.gushi.org [EMAIL PROTECTED]; sender-id=softfail; spf=softfail Received: from unknown (HELO tilmb7.indiatimes.com) ([192.168.61.27]) by x1.indiatimes.com with ESMTP; 06 Oct 2007 15:07:38 +0530 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AnoUAJL0BkfAqD0b/2dsb2JhbAAMiRw X-IronPort-AV: i=unknown; a=17144176:sNHT0 Date: Sat, 6 Oct 2007 14:57:11 +0530 (IST) From: Mr.Craig McAfee [EMAIL PROTECTED] Reply-To: Mr.Craig McAfee [EMAIL PROTECTED] Message-ID: [EMAIL PROTECTED] Subject: Attn:YOU HAVE WON A PRIZE (1,700,000.00 Euros)! MIME-Version: 1.0 X-Originating-IP: [127.0.0.1] Content-Type: text/plain; charset=utf-8 X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-3.0 (prime.gushi.org [0.0.0.0]); Sat, 06 Oct 2007 05:40:56 -0400 (EDT) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by prime.gushi.org id l969eqTG063292 X-Envelope-To: [EMAIL PROTECTED] [ The following text is in the utf-8 character set. ] [ Your display is set for the US-ASCII character set. ] [ Some characters may be displayed incorrectly. ] Attention!!! Your email address has emerged as one of the winner in Euromillions FreeDraws.Prize attached is 1,700,000.00 Euros.Contact Mr Mr Denis Ernest Fing.Email:[EMAIL PROTECTED] with the following information:1, Full Names: 2. Address:3. Age:4. Sex:5. Phone /Fax number: and 6. Country: -- My life has changed. What about yours? Log on to the new Indiatimes Mail and Live out of the Inbox! -- Is Gushi a person or an entity? Yes -Bad Karma, August 25th 2001, Ezzi Computers, Quoting himself earler, referring to Gushi Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: DK_POLICY_SIGNSOME
On Mon, 6 Aug 2007, Mark Martinec wrote: Rob, When the domainkey policy record for the domain in question says the domain signs some of its email. Heheh.. Yeah, I guessed that much, but, we *don't* sign email. Not DK(IM) or anything else. Yes, this is normal. An absence of a policy record implies a default policy, which is a neutral 'signs some mail'. True, but perhaps, SA could hit a different rule when encountering the EXPLICIT signsome policy versus the IMPLICIT, i.e. DK_POLICY_SIGNSOME_DEFAULT or something similar? (With corresponding explanation tests). -Dan -- Tonite on reboot! People misspelling as many words with sexual connotations as possible... -Keyo-Chan, February 10th 1999, Undernet #reboot Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: [sa-list] Re: Default Plugins?
On Sat, 4 Aug 2007, Theo Van Dinter wrote: On Fri, Aug 03, 2007 at 10:59:31PM -0400, Dan Mahoney, System Admin wrote: Is there some default mechanism loading these things (for example, I notice loadplugin Mail::SpamAssassin::Plugin::DKIM is only in v312.pre), and is it safe to remove the old ones? So then, what if, for example, nothing else had loaded Mail::SpamAssassin::Plugin::DKIM? If nothing loads that plugin, then you don't get the functionality. SA reads *.pre, so as long as a plugin is loaded in one of them, it's loaded. It wasn't in the other files, even in a commented out format? Should there be a Lint of all the possible modules (and worst-case scenario, I get an error if I try to load a module twice) You can't list all the possible modules, since they can live anywhere. You could get a list of the standard/default modules, and any modules that an update channel gives you though. No, but YOU (the SA team) can, in fact, list all of the modules that you are shipping with a specific version of SA, in a commented (and possibly commented out) version of $version.pre. Notes in there such as: 'Mail::SpamAssassin::Plugin::DomainKeys is officialy outdated by Mail::SpamAssassin::Plugin::DKIM' would be nice things too (as presumably, nothing is going to ever REMOVE that old module from its installed location for those of us using the make, make install method, and because SA will still read the three-versions-ago command to LOAD that module. However, I don't know what a lint would do for you. Plugins are optional (*), so not loading them isn't a reportable problem. In fact, that's one of the main benefits of having plugins: being able to not load certain functionality, reducing the amount of resources needed to run SA, etc. Maybe I didn't mean the same thing by LINT you thought I meant? Under BSD, there's a kernel config file called LINT that lists every possible kernel config option (even cross-incompatible ones) so you can at least see and grep for them all. In older versions, this was fully commented. In more recent versions, it's programmatically generated, which means there's no nice human readable comments, but that it's more likely to be all-inclusive. In the case of SA, the printing of such a message/description could come from the self-contained POD documentation. While I feel it's my duty as an admin to know which modules I installed myself, and which were stock (pretty simple, based on which config file loads them from where, in most cases), it's only stated in the included docs that NEW modules are in $version.pre (which doesn't help AT ALL if I missed a version bump, or am installing clean). Even now, there could be functionality I'm missing, simply because I haven't installed every minor version in between. -Dan -- If you aren't going to try something, then we might as well just be friends. We can't have that now, can we? -SK Dan Mahoney, December 9, 1998 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Default Plugins?
Hello all, I've got some stale v3xx.pre files around, and I notice that they load plugins that are NOT loaded by v320.pre Is there some default mechanism loading these things (for example, I notice loadplugin Mail::SpamAssassin::Plugin::DKIM is only in v312.pre), and is it safe to remove the old ones? I can't find a good piece of documentation on the wiki on this, would be happy to add it if I could get a definitive answer. -Dan -- Little tramp sits in her room all day, sewing dolls! Children misbehaving in the basement, and one in the walls, doing his business God knows where! You children will be the death of me, *sniff*. 'Mommy', The People Under The Stairs Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
http://wiki.apache.org/spamassassin/SareChannels
I notice the above page is immutable, for some reason. I noticed, upon trying to use the instructions at http://saupdates.openprotect.com/, that there IS no DNS record for 3.2.2 updates there, and I cannot edit the page to reflect this. Nor is there an easy piece of contact information on that page (I could look through all of openprotect, I suppose, but it's a bit much). Can someone tell me why, to any of the above? -Dan -- Blargy Frap! -mtreal, efnet #macintosh channel, 8.10.98, Approx 3AM Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Default Plugins?
On Fri, 3 Aug 2007, Theo Van Dinter wrote: On Fri, Aug 03, 2007 at 06:49:58PM -0400, Dan Mahoney, System Admin wrote: I've got some stale v3xx.pre files around, and I notice that they load plugins that are NOT loaded by v320.pre Of course. Is there some default mechanism loading these things (for example, I notice loadplugin Mail::SpamAssassin::Plugin::DKIM is only in v312.pre), and is it safe to remove the old ones? All pre files are used. Nothing is automatically loaded. There are multiple files, based on the release where the plugins that are loaded by that file were in. This way, we can add new plugins and the new pre file will get installed, and there's no issue with changing the old pre files (where admins may have added their own config, commented things out, etc.) So no, don't remove old pre files, because they're still being used and important. So then, what if, for example, nothing else had loaded Mail::SpamAssassin::Plugin::DKIM? It wasn't in the other files, even in a commented out format? Should there be a Lint of all the possible modules (and worst-case scenario, I get an error if I try to load a module twice) -Dan -- I wish the Real World would just stop hassling me! -Matchbox 20, Real World, off the album Yourself or Someone Like You Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
DNS timeouts on almost all queries
Any idea what could be causing the following? DNS is against localhost, net::dns is 0.59 I'm seeing a ton of this in my ddebug log: Jan 5 16:37:14 quark spamd[2031]: dns: timeout for sorbs-lastexternal,sorbs after 11 seconds Jan 5 16:37:14 quark spamd[2027]: dns: timeout for rfci_envfrom after 11 seconds Jan 5 16:37:14 quark spamd[2027]: dns: timeout for NO_DNS_FOR_FROM after 11 seconds Jan 5 16:37:14 quark spamd[2031]: dns: timeout for NO_DNS_FOR_FROM after 11 seconds Jan 5 16:37:14 quark spamd[2027]: dns: timeout for bsp-firsttrusted after 11 seconds Jan 5 16:37:14 quark spamd[2031]: dns: timeout for sorbs after 11 seconds Jan 5 16:37:14 quark spamd[2031]: dns: timeout for whois after 11 seconds Jan 5 16:37:14 quark spamd[2031]: dns: timeout for whois,whois-lastexternal after 11 seconds Jan 5 16:37:14 quark spamd[2031]: dns: timeout for rfci_envfrom after 11 seconds Jan 5 16:37:15 quark spamd[2031]: dns: timeout for NO_DNS_FOR_FROM after 12 seconds Jan 5 16:37:15 quark spamd[2031]: dns: timeout for bsp-firsttrusted after 12 seconds Jan 5 16:37:17 quark spamd[2050]: dns: timeout for sorbs-lastexternal,sorbs after 11 seconds Jan 5 16:37:17 quark spamd[2050]: dns: timeout for iadb-firsttrusted after 11 seconds Jan 5 16:37:17 quark spamd[2050]: dns: timeout for whois,whois-lastexternal after 11 seconds Jan 5 16:37:17 quark spamd[2050]: dns: timeout for bsp-firsttrusted after 11 seconds Jan 5 16:37:17 quark spamd[2049]: dns: timeout for sorbs-lastexternal,sorbs after 11 seconds Jan 5 16:37:17 quark spamd[2049]: dns: timeout for NO_DNS_FOR_FROM after 11 seconds Jan 5 16:37:17 quark spamd[2049]: dns: timeout for rfci_envfrom after 11 seconds Jan 5 16:37:17 quark spamd[2049]: dns: timeout for whois,whois-lastexternal after 11 seconds Jan 5 16:37:17 quark spamd[2049]: dns: timeout for iadb-firsttrusted after 11 seconds Jan 5 16:37:17 quark spamd[2049]: dns: timeout for NO_DNS_FOR_FROM after 11 seconds Jan 5 16:37:17 quark spamd[2049]: dns: timeout for bsp-firsttrusted after 11 seconds Jan 5 16:37:17 quark spamd[2035]: dns: timeout for iadb-firsttrusted after 11 seconds Jan 5 16:37:17 quark spamd[2035]: dns: timeout for rfci_envfrom after 11 seconds Jan 5 16:37:17 quark spamd[2035]: dns: timeout for whois,whois-lastexternal after 11 seconds Jan 5 16:37:17 quark spamd[2035]: dns: timeout for sorbs-lastexternal,sorbs after 11 seconds Jan 5 16:37:17 quark spamd[2035]: dns: timeout for NO_DNS_FOR_FROM after 11 seconds Jan 5 16:37:17 quark spamd[2035]: dns: timeout for spamcop after 11 seconds Jan 5 16:37:17 quark spamd[2035]: dns: timeout for bsp-firsttrusted after 11 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for iadb-firsttrusted after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for rfci_envfrom after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for ahbl after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for sorbs-lastexternal,sorbs after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for sorbs after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for whois after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for NO_DNS_FOR_FROM after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for whois,whois-lastexternal after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for NO_DNS_FOR_FROM after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for spamcop after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for bsp-firsttrusted after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for spamcop after 13 seconds Jan 5 16:37:18 quark spamd[2048]: dns: timeout for bsp-untrusted after 13 seconds Jan 5 16:37:19 quark spamd[2053]: dns: timeout for NO_DNS_FOR_FROM after 8 seconds Jan 5 16:37:19 quark spamd[2053]: dns: timeout for sorbs-lastexternal,sorbs after 8 seconds Jan 5 16:37:19 quark spamd[2053]: dns: timeout for whois,whois-lastexternal after 8 seconds Jan 5 16:37:19 quark spamd[2053]: dns: timeout for NO_DNS_FOR_FROM after 8 seconds Jan 5 16:37:19 quark spamd[2036]: dns: timeout for rfci_envfrom after 7 seconds -- I wish the Real World would just stop hassling me! -Matchbox 20, Real World, off the album Yourself or Someone Like You Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
remove known-postmaster entries from AWL?
Hey all, The subject line says it all. I've got a lot of users complaining about bounce spam, and while there's an 0.1 scoring Vbounce ruleset, I notice that more often than not postmaster scores sometimes a whopping .5, and at other times -17. Is there any way to simply say dont do this for unqualified addresses? (or postmaster addresses) -Dan -- If you aren't going to try something, then we might as well just be friends. We can't have that now, can we? -SK Dan Mahoney, December 9, 1998 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: remove known-postmaster entries from AWL?
On Fri, 5 Jan 2007, Matt Kettler wrote: Dan Mahoney, System Admin wrote: Hey all, The subject line says it all. I've got a lot of users complaining about bounce spam, and while there's an 0.1 scoring Vbounce ruleset, I notice that more often than not postmaster scores sometimes a whopping .5, and at other times -17. Is there any way to simply say dont do this for unqualified addresses? (or postmaster addresses) -Dan Why should the AWL even matter here? Because it's matching, with bizarrely sporadic results. You do realize the AWL isn't a whitelist, right? Why is it hitting, then? If it has the power to influence the score hamward (on a non-real and possibly forged email address), I'd prefer it not to apply. -Dan -- Happy, Sad, Happy, Sad, Happy, Sad, Happy, Intruiged! I've never been so in touch with my emotions! -AndrAIa as Hexadecimal, Reboot Episode 3.2.3 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: DNS timeouts on almost all queries
On Fri, 5 Jan 2007, Matt Kettler wrote: Dan Mahoney, System Admin wrote: Any idea what could be causing the following? DNS is against localhost, net::dns is 0.59 I'm seeing a ton of this in my ddebug log: Jan 5 16:37:14 quark spamd[2031]: dns: timeout for sorbs-lastexternal,sorbs after 11 seconds Try dig @localhost www.spamassassin.org ie: is the localhost DNS server working properly? Yes, it is. I'm not seeing any major errors in /var/log/messages, and I'm seeing some rules match on this. It would be quasi-helpful if the error logged what lookup was actually timing out (i.e. 1.1.168.192.someblacklist.org or whatever) so that one could try and diagnose this stuff with tcpdump or querylogs. It's not doing it right now -- it appears to be something that only happens when the system is under reasonably high load, but I *did* verify that lookups were working when I was getting these messages. -Dan -- She's been getting attacked by these leeches, they're leaving these marks all over her neck. You gotta keep her out of those woods. If one more leech gets her, she's gonna get a smack. -Someone's Mother, December 18th, 1998 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: DNS timeouts on almost all queries
On Fri, 5 Jan 2007, Matt Kettler wrote: Dan Mahoney, System Admin wrote: ie: is the localhost DNS server working properly? Yes, it is. I'm not seeing any major errors in /var/log/messages, and I'm seeing some rules match on this. It would be quasi-helpful if the error logged what lookup was actually timing out (i.e. 1.1.168.192.someblacklist.org or whatever) so that one could try and diagnose this stuff with tcpdump or querylogs. It's not doing it right now -- it appears to be something that only happens when the system is under reasonably high load, but I *did* verify that lookups were working when I was getting these messages. Hmm, have you tried hitting the local named with queryperf (it's a DNS mass-querry load-test.) Haven't yet. Was also going to try turning on the querylog (I turn it on via rndc but I'm not getting output anywhere) What named are you using? 9.3.1, which will be upgraded as soon as my ports tree syncs. -Dan -- Pika Pika Pika! -Pikachu, of Pokemon fame. Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: [sa-list] Re: Way to skip scanning per-user?
One of my users just started getting slammed. This one user, out of 400+ is getting a dictionary attack that's overwhelming all my spamd process slots. Doing this on the spamd side would make simply stopping this really simple -- even programmatically (i.e. automatically). Manually, even with the best .procmailrc in the world I don't have a way. Just my 0.02 -Dan -- We are basically...'Bandwidth Pimps'...Hrmmm...But that's cool man! You see these gold chains? It's all good! -Ali Dhoon 03/03/2003, 7PM Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: SPF is hopelessly broken and must die!
On Thu, 14 Dec 2006, Magnus Holmgren wrote: On Thursday 14 December 2006 01:37, Marc Perkel wrote: How do you deal with people forwarding email from another domain when using SPF? *If* you intend to reject mail based on hard SPF failures, then you *must* allow for exceptions for forwarded mail. Mail can only be forwarded from specific hosts, so while it might be tricky it's definitely possible to define such exception in a meaningful way. Demanding that forwarding between arbitrary hosts must simply work (without SRS, DKIM or some other mechanism) is to say that everyone must always trust the envelope sender and mail header like 20 years ago. That is what is really broken. Heh, ironically, everytime I post to [EMAIL PROTECTED], I get a DKIM failure report (but the mail still goes through to the list). Clearly whatever mailing list software they're using is NOT dkim-aware. -Dan -- Happy, Sad, Happy, Sad, Happy, Sad, Happy, Intruiged! I've never been so in touch with my emotions! -AndrAIa as Hexadecimal, Reboot Episode 3.2.3 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Way to skip scanning per-user?
Hey all, I'm looking for an easy way to override ALL scanning (NOT scoring) for a specific user. This is NOT the same as just setting required_score to 1000 -- basically what I want instead is some special way that SA will say nope, not even testing and short circuit. This shouldn't be a difficult feature to implement at all -- I'd imagine about three lines of code :) There are several uses for this, either when a user is using some alternate engine (so why eat CPU on the scanning system?), or under the situation that you have a user who has SUCH a volume of spam that it's under constant attack and you just want to opt them out of the system for diagnostic purposes. Any ideas on how to do this? -Dan -- Long live little fat girls! -Recent Taco Bell Ad Slogan, Literally Translated. (Viva Gorditas) Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Way to skip scanning per-user?
On Thu, 14 Dec 2006, Theo Van Dinter wrote: On Thu, Dec 14, 2006 at 11:59:26AM -0500, Dan Mahoney, System Admin wrote: I'm looking for an easy way to override ALL scanning (NOT scoring) for a specific user. Don't send mails for that user to SA. At the moment, that's a hack in the system-wide procmailrc that I don't know how to do, since the only thing procmail knows about userspace is dropprivs=yes, and there's no translation for an easy way to equate that to email address (i.e. it allows me to do it per *domain* not per user, i.e. [EMAIL PROTECTED], but if a user has two domains, then I'd have to do them each separately). what I want instead is some special way that SA will say nope, not even testing and short circuit. At the moment, you can't do that. This shouldn't be a difficult feature to implement at all -- I'd imagine about three lines of code :) There's code in 3.2 to do it, but it's still the most efficient to just not call SA for mails you don't want scanned (SA will still need to do all the processing to start looking at the mail, until it realizes that the mail is whitelisted or whatever, and then stop processing). Presuming we're looking for the value of the user based on the email address, yes, I understand, but can't you check the value of -u before you even do that? (i.e. at the earliest point) -Dan -- A mother can be an inspiration to her little son, change his thoughts, his mind, his life, just with her gentle hum. -No Doubt, Different People, from Tragic Kingdom Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] RE: Way to skip scanning per-user?
On Thu, 14 Dec 2006, Coffey, Neal wrote:
Dan Mahoney, System Admin wrote:
I'm looking for an easy way to override ALL scanning (NOT scoring)
for a specific user.
This needs to be done in whatever you're using to call SpamAssassin
(postfix, exim, sendmail, etc).
This shouldn't be a difficult feature to implement at all -- I'd
imagine about three lines of code :)
How do you handle messages with multiple recipients? Not to mention
that the envelope to address(s) (who the mail is *actually* delivered
to) don't have to match the headers that SA sees.
I said per-user, not per email address. Spamd knows which local user is
doing the calling before it ever reads the first line of the message.
With spamassassin proper (assuming SQL prefs are in play), check $ or $
-- with spamc/spamd, it's being communicated.
Since SA needs to be called by another program, and that program will be
aware of all of this, that's really the place to do the exemption.
See my previous message. I don't see an easy macro in procmail for the
current effective UID, nor do I know an easy way to say:
if (**my uid is any of these) {
}
else {
call spamassassin
}
Where as a bonus ** is generated dynamically.
If you can supply a snippet of code that does it, I'd love it. If I was
only doing scanning FOR a few select users this might make a bit more
sense, but it makes sense to me that this be a user_prefable item, as
opposed to my users asking me to edit /etc/procmailrc
-Dan
--
SOY BOMB!
-The Chest of the nameless streaker of the 1998 Grammy Awards' Bob Dylan
Performance.
Dan Mahoney
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---
Re: [sa-list] Re: [sa-list] Re: Way to skip scanning per-user?
On Thu, 14 Dec 2006, Theo Van Dinter wrote: On Thu, Dec 14, 2006 at 12:11:11PM -0500, Dan Mahoney, System Admin wrote: At the moment, that's a hack in the system-wide procmailrc that I don't know how to do, since the only thing procmail knows about userspace is dropprivs=yes, and there's no translation for an easy way to equate that to email address (i.e. it allows me to do it per *domain* not per user, i.e. [EMAIL PROTECTED], but if a user has two domains, then I'd have to do them each separately). If you're using procmail, you could look at the X-Original-To (or similar) header to figure out who the mail is going to. Otherwise, you could modify your setup to pass information in to procmail from the MTA. Presuming we're looking for the value of the user based on the email address, yes, I understand, but can't you check the value of -u before you even do that? (i.e. at the earliest point) Ah, there you're talking about spamc/spamd which is a different beasty all together. If you want to skip checks based on how you're calling spamc, then check the value you're going to use for the username and don't call spamc if you don't want the mail scanned. I'm running procmail with dropprivs=yes. There's no easy procmail thing for (getpwnam($)) and I do NOT feel like firing up perl on every message to evaluate that just to figure out if I should fire up the C program that I use so I don't have to fire up perl. I see procmail macros for the email address, and for the _TO thing, but NOTHING that just gives you the goddamned login. I don't need -u on spamc, spamc just picks up that username and runs with it. If I'm running spamc as danm, spamd grabs danm's prefs. When I said -u, I was asking how spamd would recognize the implied value of -u, not the actual command line flag. If that makes sense? -Dan -- It would be bad. -Egon Spengler, Ghostbusters Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Way to skip scanning per-user?
On Thu, 14 Dec 2006, Theo Van Dinter wrote: As an aside, part of this is why I had asked for (a while back) a way to specify the domain portion of the -u argument, i.e. so it could be done per-calling server (i.e. it is assumed that if shell server A and shell server B, each with a distinct user-base are sharing a spamd machine, then their user bases will have prefnames derived from the hostnames of A and B.) -- regardless of the email address used. i.e. localusername @ suffix (where the suffix is supplied to spamc in some global config file, and the localusername is automatic). Knowing how to do this (get the current username) in procmail (without firing up perl or even SED -- I could call a binary like whoami but that's a bit less universal) would also make THIS mostly unnecessary. Again, this is not at all based on email address (except in the case of emails like mine, where my address accurately reflects the FQDN of the calling server -- but then I've always been the exception rather than the rule), but on UID and HOSTNAME. The servers in question have 400 uids each, two hostnames, and potentially MILLIONS of email addresses, especially in a dictionary attack, where the user has a catch-all account. Which does it make sense to modify stats by? -- I am a professional drinker, and I know that that was NOT Jose Cuervo! Well, what was it then? I think it was some mixture of Rubbing Alcohol, and Desenex(TM) Foot Powder, because my feet feel okay, and my back doesn't hurt, but my stomach is killing me! -Dan Mahoney, Costa Rica, August 12th, 1994 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [Devel-spam] SA 3.1.7 children hang but don't die
On Wed, 18 Oct 2006, George R. Kasica wrote: I'm having the same issue with 3.1.7 under FreeBSD 5.4 -- all patches applied to gocr/giftext. -Dan On Wed, 18 Oct 2006 13:20:06 -0500, you wrote: - Original Message - From: Daryl C. W. O'Shea [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: Sandy S [EMAIL PROTECTED]; Chris Lear [EMAIL PROTECTED]; users@spamassassin.apache.org; [EMAIL PROTECTED] Sent: Wednesday, October 18, 2006 1:09 PM Subject: Re: SA 3.1.7 children hang but don't die George R. Kasica wrote: I've dropped back to 3.1.5 last evening about 2200 CDT and no problems since. I'm also running FuzzyOCR 2.3b here and did not see the problem until I got to 3.1.7 I'll cc this to the FuzzyOCR list and see if anyone there is seeing this If someone(s) can definitively confirm whether this problem only happens under 3.1.6/3.1.7 and not 3.1.5 or earlier, please make sure we hear about it. IIRC, it's possible that the fix for bug 5081 (3.1.6) could be affecting this. Daryl Daryl - I switched back to 3.1.5 after my last post, and am sorry to report that I'm still seeing the same issue under 3.1.5. After running a while, the processes in a state of K start building up until I manually kill them. Regretfully (VERY regretfully) turning off FuzzyOCR. Sandy Sandy: I'm NOT Seeing it here with 3.1.5 and FuzzyOCR since 2200 CDT last evening 10/17/06. Normally it would have shown up a couple times since then. FuzzyOCR is still running here no other changes except dropping back to 3.1.5. George ___ Devel-spam mailing list [EMAIL PROTECTED] http://lists.own-hero.net/mailman/listinfo/devel-spam -- She's been getting attacked by these leeches, they're leaving these marks all over her neck. You gotta keep her out of those woods. If one more leech gets her, she's gonna get a smack. -Someone's Mother, December 18th, 1998 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Userprefs via X-Envelope-To header.
Hey all, Has anyone managed to successfully create an easy way to have a custom query look up prefs based on the X-Envelope-To header -- preferably with domain and username via custom query? I just need a few pointers here. -Dan -- Oh, and we just recently got an invoice... Congratulations! -JC and DM, regarding Unpredictable Billing, 8/18/2001 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Spamd keeps getting hung up!
On Fri, 31 Mar 2006, Justin Mason wrote: Hey, if anyone is around RIGHT NOW, I'm getting the issue, it's repeatable, and I can't figure out strace...I'm trying strace -o /home/danm/strace.log -f -ttt /usr/local/bin/spamd -D -u spamd -i -A 72.9.101.130,65.125.237.232,65.125.228.130,127.0.0.1 -q -d -m 40 -r /tmp/spamd.pid -l --min-spare=5 --max-spare=20 but it's only capturing like one line of output to the logfile. I should prolly note that the BSD standard seems to be truss rather than strace, but strace IS in ports and installed oni my spamd box. If you can catch me via instand messenger (gushiDotOrg) or try email, I might be able to help nail this one down, at least as long as this barrage continues, assuming we can get a workable strace it's the same tinsc user again, getting flooded, and I'm now capturing their messages for later analysis (admittedly after spamassassin hits them, but the filter also catches them after I have to KILL spamassassin, which lets us easily see which ones were being processed when it was killed (since they will lack the SA headers) Sorry for the bad punctuation, I'm on satellite. -Dan sounds like a new ticket is in order, alright. btw if *is* load-related, an strace -f -ttt log will show that pretty clearly. --j. Daryl C. W. O'Shea writes: (copying Justin since this has to do with pre-forking) Dan Mahoney, System Admin wrote: On Fri, 10 Mar 2006, Daryl C. W. O'Shea wrote: On 3/10/2006 11:22 AM, Dan Mahoney, System Admin wrote: Okay, I'm still getting these issues. I've corrected every other issue that's plagued us, and the thing still locks up. USUALLY when a user gets some form of dictionary spam. For the users I can identify I've been keeping copies of their stuff. NOTE: This is under a stock 3.1.1, if there are any other patches I should be using from the previous conversations that are NOT in 3.1.1, please let me know, and I'll make sure I have those too. I'm seeing lots of the following: Mar 30 21:52:14 quark spamd[45835]: __alarm__ Mar 30 21:52:14 quark spamd[45835]: __alarm__ Mar 30 21:52:14 quark spamd[45835]: spamd: copy_config timeout (with empty $@), respawning child process after 25 messages at /usr/local/bin/spamd line 982. Mar 30 21:52:16 quark spamd[52479]: __alarm__ Mar 30 21:52:16 quark spamd[52479]: __alarm__ Mar 30 21:52:16 quark spamd[52479]: spamd: copy_config timeout (with empty $@), respawning child process after 9 messages at /usr/local/bin/spamd line 982. This indicates that the patch from bug 4699 is working -- spamd now recognizes that the alarm timed out on copy_config. And also some of this: Mar 30 21:52:31 quark spamd[42292]: syswrite() on closed filehandle GEN88 at /usr/local/lib/perl5/5.8.6/mach/IO/Handle.pm line 451. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: prefork: write of ping failed to 52479 fd=: at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killing failed child 52479 fd= at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killed child 52479 at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 141. Mar 30 21:52:31 quark spamd[42292]: syswrite() on closed filehandle GEN70 at /usr/local/lib/perl5/5.8.6/mach/IO/Handle.pm line 451. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: prefork: write of ping failed to 45835 fd=: at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killing failed child 45835 fd= at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killed child 45835 at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 141. This indicates that the child is exiting, but SpamdForkScaling doesn't know about it until a ping fails 150 seconds later, so a new child isn't spawned for a long time after one of them commits suicide. Example at or around Mar 30 01:48:16 in this file: http://www.gushi.org/maillog33106-0.txt And another similar lockup at Mar 30 21:49:50 -- SAME USER, go figure. I don't have
Re: Spamd keeps getting hung up!
On Sun, 2 Apr 2006, Daryl C. W. O'Shea wrote: Dan Mahoney, System Admin wrote: On Fri, 31 Mar 2006, Daryl C. W. O'Shea wrote: Dan Mahoney, System Admin wrote: Mar 30 21:52:14 quark spamd[45835]: __alarm__ Mar 30 21:52:14 quark spamd[45835]: __alarm__ Mar 30 21:52:14 quark spamd[45835]: spamd: copy_config timeout (with empty $@), respawning child process after 25 messages at /usr/local/bin/spamd line 982. Mar 30 21:52:16 quark spamd[52479]: __alarm__ Mar 30 21:52:16 quark spamd[52479]: __alarm__ Mar 30 21:52:16 quark spamd[52479]: spamd: copy_config timeout (with empty $@), respawning child process after 9 messages at /usr/local/bin/spamd line 982. This indicates that the patch from bug 4699 is working -- spamd now recognizes that the alarm timed out on copy_config. I'm still thinking that you should increase this alarm timeout value -- to something like 600 seconds, or remove it entirely -- if you're going to be running under high load. Mar 30 21:52:31 quark spamd[42292]: syswrite() on closed filehandle GEN88 at /usr/local/lib/perl5/5.8.6/mach/IO/Handle.pm line 451. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: prefork: write of ping failed to 52479 fd=: at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killing failed child 52479 fd= at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killed child 52479 at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 141. This indicates that the child is exiting, but SpamdForkScaling doesn't know about it until a ping fails 150 seconds later, so a new child isn't spawned for a long time after one of them commits suicide. As noted in bug 4852, this appears to only be cosmetic. spamd is continuing to spawn new children as necessary and is continuing to process mail successfully. It does so slowly, but with a load average of over 12 (and sometimes over 15+), I'm not surprised. Oddly enough, one of our other machines on our network (which also runs spamd) also seems to die around the same time. I'm concerned about IT as well, but less than this one. Still, snagging logs there is probably not a bad idea. Any idea what sort of load averages you've got when this starts to happen? It looks like it starts off with a couple children timing out, then you become short on children, mail starts stacking up, and it snowballs from there. I know somewhere in those logs it started rejecting mail on load average 12. A simple one-liner in spamd to echo the load into the logs could be useful (I don't need a patch, but telling me what to put and where to put it could be useful). Alternatively I could just do something with logger(1), echo(1), uptime(1) and cron. As far as I can tell from the log, it appears that the load average was near or above 12 the entire time that the time outs were occurring. After the restart of spamd at Mar 30 14:15:46, everything was fine until another burst of mail around Mar 30 21:10:04 at which point the load average is reported to be 12. At the same time, spamd starts spawning more children, which isn't a great thing to be doing with a load average of 12 or so. As this happens, processing slows down, and more children are spawned. This clears up and we're back to the min number of children around Mar 30 21:13:38. Another burst happens around Mar 30 21:49:56, again with a load average around 12. A lot more children are spawned this time, and things really slow down. copy_config timeouts start happening again, but mail is still being processed and children are exiting and spawning appropriately. This continues to the end of the log. Somewhere along the lines last night I also lost connection to AIM (which runs from that netblock) so it's quite possibly network congestion related. Even so, if I theoretically had 30 seconds of latency 6 hours ago, spamd should theoretically NOT still be hanging now... It doesn't look like spamd ever hangs. It continues to process mail, albeit slowly due to load. This load could be being caused, and sustained, due to the machine hitting swap and thrashing. Do you really have enough memory for 40 spamd children? I don't think I'd try it with less than 2.0GB of RAM dedicated to spamd use. With an MTA, SQL, and whatever else running, I'd probably want 3GB+ of RAM in the machine. This really looks similar to what happens to Sendmail when a machine hits a load average of over 900
Re: Spamd keeps getting hung up!
On Fri, 10 Mar 2006, Daryl C. W. O'Shea wrote: On 3/10/2006 11:22 AM, Dan Mahoney, System Admin wrote: I of course have no idea what to make of this output. Pointers? Each line is one file descriptor. So it doesn't appear that it's using an insane number of them. Next time spamd hangs up, you might want to do this check though. I'm outta ideas... I don't know if Justin will have any ideas without a full strace of a problem spamd parent and children (which could be difficult in getting with the amount of messages processed by your system). Okay, I'm still getting these issues. I've corrected every other issue that's plagued us, and the thing still locks up. USUALLY when a user gets some form of dictionary spam. For the users I can identify I've been keeping copies of their stuff. NOTE: This is under a stock 3.1.1, if there are any other patches I should be using from the previous conversations that are NOT in 3.1.1, please let me know, and I'll make sure I have those too. I'm seeing lots of the following: Mar 30 21:52:14 quark spamd[45835]: __alarm__ Mar 30 21:52:14 quark spamd[45835]: __alarm__ Mar 30 21:52:14 quark spamd[45835]: spamd: copy_config timeout (with empty $@), respawning child process after 25 messages at /usr/local/bin/spamd line 982. Mar 30 21:52:16 quark spamd[52479]: __alarm__ Mar 30 21:52:16 quark spamd[52479]: __alarm__ Mar 30 21:52:16 quark spamd[52479]: spamd: copy_config timeout (with empty $@), respawning child process after 9 messages at /usr/local/bin/spamd line 982. And also some of this: Mar 30 21:52:31 quark spamd[42292]: syswrite() on closed filehandle GEN88 at /usr/local/lib/perl5/5.8.6/mach/IO/Handle.pm line 451. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: prefork: write of ping failed to 52479 fd=: at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killing failed child 52479 fd= at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killed child 52479 at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 141. Mar 30 21:52:31 quark spamd[42292]: syswrite() on closed filehandle GEN70 at /usr/local/lib/perl5/5.8.6/mach/IO/Handle.pm line 451. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: prefork: write of ping failed to 45835 fd=: at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killing failed child 45835 fd= at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killed child 45835 at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 141. Example at or around Mar 30 01:48:16 in this file: http://www.gushi.org/maillog33106-0.txt And another similar lockup at Mar 30 21:49:50 -- SAME USER, go figure. I don't have archived copies of this user's mail -- yet. I've set up archiving for them, and we have everything from now forward, but I'm convinced there's SOMETHING in the spam they're getting that causes a lockup. -Dan -- I love you forever eternally. -Connaian Expression Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Spamd keeps getting hung up!
On Fri, 31 Mar 2006, Daryl C. W. O'Shea wrote: (copying Justin since this has to do with pre-forking) Dan Mahoney, System Admin wrote: On Fri, 10 Mar 2006, Daryl C. W. O'Shea wrote: On 3/10/2006 11:22 AM, Dan Mahoney, System Admin wrote: Okay, I'm still getting these issues. I've corrected every other issue that's plagued us, and the thing still locks up. USUALLY when a user gets some form of dictionary spam. For the users I can identify I've been keeping copies of their stuff. NOTE: This is under a stock 3.1.1, if there are any other patches I should be using from the previous conversations that are NOT in 3.1.1, please let me know, and I'll make sure I have those too. I'm seeing lots of the following: Mar 30 21:52:14 quark spamd[45835]: __alarm__ Mar 30 21:52:14 quark spamd[45835]: __alarm__ Mar 30 21:52:14 quark spamd[45835]: spamd: copy_config timeout (with empty $@), respawning child process after 25 messages at /usr/local/bin/spamd line 982. Mar 30 21:52:16 quark spamd[52479]: __alarm__ Mar 30 21:52:16 quark spamd[52479]: __alarm__ Mar 30 21:52:16 quark spamd[52479]: spamd: copy_config timeout (with empty $@), respawning child process after 9 messages at /usr/local/bin/spamd line 982. This indicates that the patch from bug 4699 is working -- spamd now recognizes that the alarm timed out on copy_config. And also some of this: Mar 30 21:52:31 quark spamd[42292]: syswrite() on closed filehandle GEN88 at /usr/local/lib/perl5/5.8.6/mach/IO/Handle.pm line 451. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: prefork: write of ping failed to 52479 fd=: at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killing failed child 52479 fd= at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killed child 52479 at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 141. Mar 30 21:52:31 quark spamd[42292]: syswrite() on closed filehandle GEN70 at /usr/local/lib/perl5/5.8.6/mach/IO/Handle.pm line 451. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: prefork: write of ping failed to 45835 fd=: at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 330. Mar 30 21:52:31 quark spamd[42292]: Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killing failed child 45835 fd= at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 127. Mar 30 21:52:31 quark spamd[42292]: prefork: killed child 45835 at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/SpamdForkScaling.pm line 141. This indicates that the child is exiting, but SpamdForkScaling doesn't know about it until a ping fails 150 seconds later, so a new child isn't spawned for a long time after one of them commits suicide. Example at or around Mar 30 01:48:16 in this file: http://www.gushi.org/maillog33106-0.txt And another similar lockup at Mar 30 21:49:50 -- SAME USER, go figure. I don't have archived copies of this user's mail -- yet. I've set up archiving for them, and we have everything from now forward, but I'm convinced there's SOMETHING in the spam they're getting that causes a lockup. I think it's actually load related... spamd is timing out the copy_config sooner than it's really taking under high load. If you were to change the alarm value from 10 to 100 or so, around spamd line 949 this may go away. Oddly enough, one of our other machines on our network (which also runs spamd) also seems to die around the same time. I'm concerned about IT as well, but less than this one. Still, snagging logs there is probably not a bad idea. Any idea what sort of load averages you've got when this starts to happen? It looks like it starts off with a couple children timing out, then you become short on children, mail starts stacking up, and it snowballs from there. I know somewhere in those logs it started rejecting mail on load average 12. A simple one-liner in spamd to echo the load into the logs could be useful (I don't need a patch, but telling me what to put and where to put it could be useful). Alternatively I could just do something with logger(1), echo(1), uptime(1
Re: Spamd keeps getting hung up!
On Fri, 10 Mar 2006, Daryl C. W. O'Shea wrote: One of my usual users is getting hit again, and it's locking up the system right now. I'm modifying the system procmailrc to get me copies of all messages, so we can trace this down. I'm absolutely convinced there's a certain type of spam doing this. -Dan On 3/10/2006 11:22 AM, Dan Mahoney, System Admin wrote: I of course have no idea what to make of this output. Pointers? Each line is one file descriptor. So it doesn't appear that it's using an insane number of them. Next time spamd hangs up, you might want to do this check though. I'm outta ideas... I don't know if Justin will have any ideas without a full strace of a problem spamd parent and children (which could be difficult in getting with the amount of messages processed by your system). BTW, I believe there's a bug open on this mentioning the __alarm__s appearing in the maillog. I can't remember what bug number it is at the moment though. Daryl -- She's been getting attacked by these leeches, they're leaving these marks all over her neck. You gotta keep her out of those woods. If one more leech gets her, she's gonna get a smack. -Someone's Mother, December 18th, 1998 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Spamd keeps getting hung up!
On Fri, 10 Mar 2006, Dan Mahoney, System Admin wrote: Okay. SpamAssassin 3.1.1 -- same problems. Uninitialized values, bad match strings, and the ever popular spamd uses all its children and the spam keeps flowing. http://www.gushi.org/maillogAGAIN.txt (warning, 20 megs) The user in question this time would be cww...I believe it was around 2:17 AM. Any other ideas would be GREATLY appreciated. If you want a shell so you can run an strace or can give me a good line that I can use to capture all the messages (a debug module would be great for this)... -Dan -- She's been getting attacked by these leeches, they're leaving these marks all over her neck. You gotta keep her out of those woods. If one more leech gets her, she's gonna get a smack. -Someone's Mother, December 18th, 1998 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Spamd keeps getting hung up!
On Fri, 10 Mar 2006, Daryl C. W. O'Shea wrote: On 10/03/06 12:50 AM, Dan Mahoney, System Admin wrote: As I'm not a C programmer, I don't know what the relation is between a kqueue and an FD -- but could it be related? kqueues use FDs, so they are related. If that original dccifd process (PID 49681) from this morning is still running I'd run this against it to see if it's using a large number of FDs: fstat -vp 49681 First one was run just after your email arrived. Second, just a few minutes ago (8 hour difference). quark# fstat -vp 49681 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W dcc dccifd 49681 root / 2 drwxr-xr-x 512 r dcc dccifd 49681 wd /usr 1389624 drwxrwxr-x 512 r dcc dccifd 49681 text /usr 1389631 -r-xr-xr-x 466050 r dcc dccifd 496810 /dev 22 crw-rw-rw-null rw dcc dccifd 496811 /dev 22 crw-rw-rw-null rw dcc dccifd 496812 /dev 22 crw-rw-rw-null rw dcc dccifd 496813* local stream c29c7460 dcc dccifd 496814* internet dgram udp c2603bf4 dcc dccifd 496815 /usr 1389679 -rw---7564 rw dcc dccifd 496816* local dgram c29c79d8 - c2619dac dcc dccifd 496817* internet dgram udp c2603a8c dcc dccifd 496818* internet dgram udp c34cf5a0 dcc dccifd 496819* internet dgram udp c305e000 dcc dccifd 49681 10* internet dgram udp c2981e10 dcc dccifd 49681 11* internet dgram udp c2981438 dcc dccifd 49681 12 /usr 1389664 -rw-r--r-- 44508 rw dcc dccifd 49681 13* internet dgram udp c2603b40 dcc dccifd 49681 14* internet dgram udp c298121c dcc dccifd 49681 17 /usr 1389664 -rw-r--r-- 44508 rw quark# fstat -vp 49681 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W dcc dccifd 49681 root / 2 drwxr-xr-x 512 r dcc dccifd 49681 wd /usr 1389624 drwxrwxr-x 512 r dcc dccifd 49681 text /usr 1389631 -r-xr-xr-x 466050 r dcc dccifd 496810 /dev 22 crw-rw-rw-null rw dcc dccifd 496811 /dev 22 crw-rw-rw-null rw dcc dccifd 496812 /dev 22 crw-rw-rw-null rw dcc dccifd 496813* local stream c29c7460 dcc dccifd 496814* internet dgram udp c2603bf4 dcc dccifd 496815 /usr 1389679 -rw---7564 rw dcc dccifd 496816* local dgram c29c79d8 - c2619dac dcc dccifd 496817* internet dgram udp c2f3ba8c dcc dccifd 496818* internet dgram udp c2603d5c dcc dccifd 496819* internet dgram udp c305e000 dcc dccifd 49681 10* internet dgram udp c2981438 dcc dccifd 49681 11* internet dgram udp c2603ca8 dcc dccifd 49681 12 /usr 1389664 -rw-r--r-- 44508 rw dcc dccifd 49681 13* internet dgram udp c2603b40 dcc dccifd 49681 14* internet dgram udp c298121c dcc dccifd 49681 17 /usr 1389664 -rw-r--r-- 44508 rw quark# fstat -vp 49680 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W dcc dccifd 49680 root / 2 drwxr-xr-x 512 r dcc dccifd 49680 wd /usr 1389624 drwxrwxr-x 512 r dcc dccifd 49680 text /usr 1389631 -r-xr-xr-x 466050 r dcc dccifd 496800 /dev 22 crw-rw-rw-null rw dcc dccifd 496801 /dev 22 crw-rw-rw-null rw dcc dccifd 496802 /dev 22 crw-rw-rw-null rw dcc dccifd 496804* internet dgram udp c2981bf4 quark# I of course have no idea what to make of this output. Pointers? -Dan -- A single death is a tragedy. A million deaths is a statistic. -Josef Stalin, As quoted on the cover to Savatage's Dead Winter Dead Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Spamd keeps getting hung up!
On Fri, 10 Mar 2006, Daryl C. W. O'Shea wrote: On 3/10/2006 11:22 AM, Dan Mahoney, System Admin wrote: I of course have no idea what to make of this output. Pointers? Each line is one file descriptor. So it doesn't appear that it's using an insane number of them. Next time spamd hangs up, you might want to do this check though. I'm outta ideas... I don't know if Justin will have any ideas without a full strace of a problem spamd parent and children (which could be difficult in getting with the amount of messages processed by your system). True, although because I know which user getting spammed usually triggers it, reproducing this may be possible. Hell, give me an allow line on your spamd and I'll send them there :) BTW, I believe there's a bug open on this mentioning the __alarm__s appearing in the maillog. I can't remember what bug number it is at the moment though. Is there a way to tell dcc to just time out after X number of seconds and let the query die off in the background? -Dan -- I am a professional drinker, and I know that that was NOT Jose Cuervo! Well, what was it then? I think it was some mixture of Rubbing Alcohol, and Desenex(TM) Foot Powder, because my feet feel okay, and my back doesn't hurt, but my stomach is killing me! -Dan Mahoney, Costa Rica, August 12th, 1994 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
More Errors (Was Spamd Keeps Getting Hung Up).
First, a note: I wrote this last night but forgot to send. Spamd gave me a lockup YET AGAIN last night, that will be responded to separately. Anyway, I was pretty happy at the number of things I had cleared out, and issues I had resolved. I'm using basically what amounts to a reinstall of SA 3.1 with the new patches proposed in my last report -- the goal here is to eliminate all the other issues and see if we can isolate the core cause of the lockup. However, fringe issues keep coming up... Thusfar, here's the continuing list: 1) This old friend. Mar 9 01:38:18 quark spamd[61724]: Use of uninitialized value in pattern match (m//) at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Conf/Parser.pm line 547, GEN954 line 2. Mar 9 01:38:18 quark spamd[61724]: Use of uninitialized value in pattern match (m//) at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Conf/Parser.pm line 547, GEN954 line 2. I've also noticed that it will only be one child that gets hit with this, which will log the error over and over again, while other children will be fine. We've addressed it, but from the bug reports, we can't isolate it -- shell is always an open offer. 2) Right after that: Mar 9 01:38:18 quark spamd[61724]: spamd: processing message [EMAIL PROTECTED] for tinsc:58 Mar 9 01:38:18 quark spamd[61724]: Argument a isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument f isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument eb isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument a isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument a isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument c isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument ca isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument a isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Character in 'C' format wrapped in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument 21:45:33 isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument a isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument f isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument eb isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument a isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument a isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument c isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument ca isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument a isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Character in 'C' format wrapped in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument 21:45:33 isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument a isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument f isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument eb isn't
Re: [sa-list] Re: More Errors (Was Spamd Keeps Getting Hung Up).
On Thu, 9 Mar 2006, Daryl C. W. O'Shea wrote: On 09/03/06 02:29 PM, Dan Mahoney, System Admin wrote: Thusfar, here's the continuing list: 1) This old friend. Mar 9 01:38:18 quark spamd[61724]: Use of uninitialized value in pattern match (m//) at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Conf/Parser.pm line 547, GEN954 line 2. Mar 9 01:38:18 quark spamd[61724]: Use of uninitialized value in pattern match (m//) at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Conf/Parser.pm line 547, GEN954 line 2. I've also noticed that it will only be one child that gets hit with this, which will log the error over and over again, while other children will be fine. We've addressed it, but from the bug reports, we can't isolate it -- shell is always an open offer. This doesn't follow a logged __alarm__, does it? Wait on my subsequent post. Coming soon to a list near you. 2) Right after that: Mar 9 01:38:18 quark spamd[61724]: spamd: processing message [EMAIL PROTECTED] for tinsc:58 Mar 9 01:38:18 quark spamd[61724]: Argument a isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument f isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. Mar 9 01:38:18 quark spamd[61724]: Argument eb isn't numeric in pack at /usr/local/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/Util.pm line 711, GEN954 line 48. You can ignore these, they are harmless. They're also fixed in 3.1.1. If you want to get rid of them now, the patch from bug 4780 should eliminate them. Additionally add the patch from bug 4485 to make sure they don't happen. Or... install the current 3.1.1 branch. This is still kinda a production system, but at this point I'm considering it strongly. Could you give me a rough rough date as to when 3.1.1 will be out? (i.e. days, weeks, months) -Dan -- Hey, call me anything you like. I'm Dan to my friends, gushi to my close friends, 'hey, you' to my girlfriend, 'mrrow?' to my cat, and 'why the hell is the router on fire?' to my job. -Dan Mahoney 12/2/02 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Spamd keeps getting hung up!
Hey all. Had another hangup last night. My full log is at http://www.gushi.org/maillog2.txt (but only for the next 24 or so...don't need to leave that out in the open). Scroll down to about Mar 9 07:13:00 to see what happens. I'm noting a few things. First, almost all of the locked up messages are caused by emails going to either user madge or user tinsc (there is no santa claus, heh). I looked at their .procmailrc -- it does NOT appear that I can get those messages for analysis, but I've modified it and let them know. Something specific to those messages is causing SA (or something it calls) to lock up HARD. [Begin Rant] This would be why I keep asking for a status display -- so I could tell what test these things are hanging on, at a glance, because from IBIBIBBIBIBBIB I really have no idea of what's been busy how long doing what, how many messages that child has served, what its average load is, which B is working on which message, or what their memory/CPU usages are. If it comes down to someone being paid to write the module, I'm at this point willing to pay a couple hundred bucks to have someone knowledgable in SA internals do it. I've made the offer before -- I'm an OKAY coder, but don't have a clue as to this codebase. Some kind of a signal handler that would cause the child to dump its message to stdout would be good too. [/Rant] Note all the similar messageIDs: [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] and so on. The __alarm__ messages (some WITH ignore, some not) start up at around Mar 9 07:14:54 They continued until I got into the office today at 2PM. Also, what's the best way to get a snapshot of 3.1.1? -Dan -- Hate fedora with a white hot burning passion right now though ... damn thing is Linux-XP(tm) -Bill Nolan 2/24/04 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Spamd keeps getting hung up!
On Thu, 9 Mar 2006, Daryl C. W. O'Shea wrote: On 09/03/06 03:09 PM, Dan Mahoney, System Admin wrote: Off topic, a bit, but is this harmless? Mar 9 19:18:52 quark spamd[32106]: netset: cannot include 216.89.180.7/32 as it has already been included Mar 9 19:18:52 quark spamd[32106]: netset: cannot include 65.125.228.128/27 as it has already been included Mar 9 19:18:52 quark spamd[32106]: netset: cannot include 216.89.180.7/32 as it has already been included -Dan -- You recreate the stars in the sky with cows? -Furrball, March 7 2005, on Katamari Damacy Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Spamd keeps getting hung up!
On Thu, 9 Mar 2006, Daryl C. W. O'Shea wrote: On 09/03/06 07:03 PM, Dan Mahoney, System Admin wrote: On Thu, 9 Mar 2006, Daryl C. W. O'Shea wrote: On 09/03/06 03:09 PM, Dan Mahoney, System Admin wrote: Off topic, a bit, but is this harmless? Mar 9 19:18:52 quark spamd[32106]: netset: cannot include 216.89.180.7/32 as it has already been included Mar 9 19:18:52 quark spamd[32106]: netset: cannot include 65.125.228.128/27 as it has already been included Mar 9 19:18:52 quark spamd[32106]: netset: cannot include 216.89.180.7/32 as it has already been included It's harmless. It's something I've added to help people get their trust paths configured correctly (especially when they use the new exclusion syntax). Since it is new though, if you don't mind checking to make sure that you really did specify the same /32s three times and the /27 twice, I'd appreciate it. If you'd like you can send me a copy of your trusted_networks and internal_networks config lines and I'll check it. Aah, this might be it...listed in both SQL -AND- the local config files. I'll delete the file versions (unless you think that's a bad idear). Never was really clear as to what the difference was with local and trusted. quark# grep trusted * local.cf:# trusted_networks 212.17.35. local.cf:trusted_networks 65.125.228.128/27 local.cf:trusted_networks 216.89.180.7 local.cf.sample:# trusted_networks 212.17.35. quark# quark# grep internal * 70_sare_html.cf:describe SARE_HTML_IMG_2ATstrange internal image link local.cf:internal_networks 216.89.180.7 quark# mysql -u root spamassassin Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 933125 to server version: 4.1.11 Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql select * from userpref where preference LIKE %network%; +--+---+---++ | username | preference| value | prefid | +--+---+---++ | @GLOBAL | trusted_networks | 216.89.180.7 |239 | | @GLOBAL | trusted_networks | 65.125.228.128/27 |240 | | @GLOBAL | internal_networks | 216.89.180.7 |241 | +--+---+---++ 3 rows in set (0.02 sec) mysql -Dan -- It's like GTA, except you pay for it, and you're allowed to use the car. -Josh, on Zipcar on-demand car-rental, 3/20/05 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: [sa-list] Re: Spamd keeps getting hung up!
On Thu, 9 Mar 2006, Daryl C. W. O'Shea wrote: On 09/03/06 07:35 PM, Dan Mahoney, System Admin wrote: On Thu, 9 Mar 2006, Daryl C. W. O'Shea wrote: On 09/03/06 07:03 PM, Dan Mahoney, System Admin wrote: On Thu, 9 Mar 2006, Daryl C. W. O'Shea wrote: On 09/03/06 03:09 PM, Dan Mahoney, System Admin wrote: Off topic, a bit, but is this harmless? Mar 9 19:18:52 quark spamd[32106]: netset: cannot include 216.89.180.7/32 as it has already been included Mar 9 19:18:52 quark spamd[32106]: netset: cannot include 65.125.228.128/27 as it has already been included Mar 9 19:18:52 quark spamd[32106]: netset: cannot include 216.89.180.7/32 as it has already been included It's harmless. It's something I've added to help people get their trust paths configured correctly (especially when they use the new exclusion syntax). Since it is new though, if you don't mind checking to make sure that you really did specify the same /32s three times and the /27 twice, I'd appreciate it. If you'd like you can send me a copy of your trusted_networks and internal_networks config lines and I'll check it. Aah, this might be it...listed in both SQL -AND- the local config files. I'll delete the file versions (unless you think that's a bad idear). Yeah, the lint warnings are correct. These weren't in --lint, they were in spamd output. Lint wouldn't have reported on both. I'd stick with the file based config for this since SQL isn't used when running spamassassin... it's only used for spamd/spamc. Yes, this is the spamd machine. -Dan trusted | internal - any IP you control Yes (maybe) any IP you trust not to forge headers (optional) Yes No your MSA (important to get this one right) Yes No your MTAs that aren't MSAs Yes Yes your MXesYes Yes relays between your border MXes and SA machine Yes Yes your SA machine Yes Yes If your MSA is the same logical server as your MTA/MX then you need to have it in both trusted internal networks and then: - do not define internal networks (just let SA copy trusted to internal) - only add IPs that you control to trusted_networks - have your roaming users use SMTP auth or POP-before-SMTP - your MTA must place auth tokens in its headers for SMTP auth sessions - you must use the POPAuth plugin to support POP-before-SMTP sessions I think this list is complete for 99% of users. Someone may want to add it to the wiki. I'll do it later tonight if nobody else has. Yay two hour commute. -Dan -- Don't think of it as beer, think of it as a flavored motor oil. -Jeremiah Kristal, on Guinness 3/29/05, 9:52 AM Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Spamd keeps getting hung up!
On Fri, 10 Mar 2006, Daryl C. W. O'Shea wrote: On 09/03/06 03:09 PM, Dan Mahoney, System Admin wrote: Hey all. Had another hangup last night. My full log is at http://www.gushi.org/maillog2.txt (but only for the next 24 or so...don't need to leave that out in the open). Scroll down to about Mar 9 07:13:00 to see what happens. I'm noting a few things. First, almost all of the locked up messages are caused by emails going to either user madge or user tinsc (there is no santa claus, heh). I looked at their .procmailrc -- it does NOT appear that I can get those messages for analysis, but I've modified it and let them know. Something specific to those messages is causing SA (or something it calls) to lock up HARD. The dccifd timeouts concern me, since dccifd should be independent of SpamAssassin. Hrmmm... How would I best check free file descriptors? Also, you might want to see this, posted by the author of DCC. http://www.rhyolite.com/pipermail/dcc/2006/003070.html -Dan dccifd[49681]: continue not asking DCC 49 seconds after failure Is it possible that the machine ran out of file descriptors, or memory? Daryl -- Man, this is such a trip -Dan Mahoney, October 25, 1997 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: Spamd keeps getting hung up!
On Fri, 10 Mar 2006, Daryl C. W. O'Shea wrote: On 10/03/06 12:12 AM, Dan Mahoney, System Admin wrote: On Fri, 10 Mar 2006, Daryl C. W. O'Shea wrote: On 09/03/06 03:09 PM, Dan Mahoney, System Admin wrote: Hey all. Had another hangup last night. My full log is at http://www.gushi.org/maillog2.txt (but only for the next 24 or so...don't need to leave that out in the open). Scroll down to about Mar 9 07:13:00 to see what happens. I'm noting a few things. First, almost all of the locked up messages are caused by emails going to either user madge or user tinsc (there is no santa claus, heh). I looked at their .procmailrc -- it does NOT appear that I can get those messages for analysis, but I've modified it and let them know. Something specific to those messages is causing SA (or something it calls) to lock up HARD. The dccifd timeouts concern me, since dccifd should be independent of SpamAssassin. Hrmmm... How would I best check free file descriptors? cat /proc/sys/fs/file-nr On RedHat systems gives you the # of allocated/in use/max FDs. Also, you might want to see this, posted by the author of DCC. http://www.rhyolite.com/pipermail/dcc/2006/003070.html That's not good. Are you on FreeBSD? Yup. FreeBSD quark.gushi.org 5.4-RELEASE FreeBSD 5.4-RELEASE #0: Sun May 8 10:21:06 UTC 2005 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC i386 As I'm not a C programmer, I don't know what the relation is between a kqueue and an FD -- but could it be related? -Dan -- Ca. Tas. Tro. Phy. -John Smedley, March 28th 1998, 3AM Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
SQL config lines failing to parse
Hey all, I recently tried to turn on SQL on my SA 3.1.0 installation. Apparently user prefs are okay, but the lines that are supposed to turn on the other two modules don't want to load. I copied the config lines almost verbatim from an existing (working) SA 3.1 server, into local.cf: Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_awl_dsnDBI:mysql:spamassassin:localhost Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_awl_sql_username spamassassin Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_awl_sql_password somepass Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_bayes_dsn DBI:mysql:spamassassin:localhost Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_bayes_sql_username spamassassin Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_bayes_sql_password somepass Mar 8 19:10:11 s1 spamd[7532]: config: failed to parse line, skipping: auto_whitelist_factory Mail::SpamAssassin::SQLBasedAddrList -- Oh, and we just recently got an invoice... Congratulations! -JC and DM, regarding Unpredictable Billing, 8/18/2001 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
Re: SQL config lines failing to parse
On Wed, 8 Mar 2006, Matt Kettler wrote: That fixed half of it. Dan Mahoney, System Admin wrote: Hey all, I recently tried to turn on SQL on my SA 3.1.0 installation. Apparently user prefs are okay, but the lines that are supposed to turn on the other two modules don't want to load. I copied the config lines almost verbatim from an existing (working) SA 3.1 server, into local.cf: Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_awl_dsnDBI:mysql:spamassassin:localhost Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_awl_sql_username spamassassin Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_awl_sql_password somepass do you have the following in your v310.pre: loadplugin Mail::SpamAssassin::Plugin::AWL v310.pre Should be installed by default in /etc/mail/spamassassin (or whatever your site_rules dir is) if you installed from source, however many distribution built packages screw up and don't install the .pre files correctly. If it's not there at all, download the file directly from the website: http://spamassassin.apache.org/full/3.1.x/dist/rules/ grab both v310.pre and init.pre as needed. Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_bayes_dsn DBI:mysql:spamassassin:localhost Erm, did you mean bayes_sql_dsn instead of user_bayes_dsn? That, AFAIK, has *never* been a valid option to SA. DOH! I'm an idiot. Thanks. Only thing that confuses me is why this isn't warning in my other system. All good, many MANY thanks. Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_bayes_sql_username spamassassin Mar 8 18:35:11 s1 spamd[4556]: config: failed to parse line, skipping: user_bayes_sql_password somepass Ditto bayes_sql_username and bayes_sql_password. Mar 8 19:10:11 s1 spamd[7532]: config: failed to parse line, skipping: auto_whitelist_factory Mail::SpamAssassin::SQLBasedAddrList Ditto on the loadplugin command. -- Why are you wearing TWO grounding straps? -John Evans, Ezzi Computers August 23, 2001 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
