Re: [qmailtoaster] bayes getting corrupt (revisited)
Citando Jake Vickers [EMAIL PROTECTED]: Eric Shubes wrote: Ok, so here's my situation. I have two apparently identical low-volume servers. One shows the corruption symptoms, the other doesn't (yet). Here is the local.cf (same on both): ok_locales all skip_rbl_checks 0 required_hits 5 report_safe 0 rewrite_header Subject [SPAM] use_pyzor 1 use_auto_whitelist 1 bayes_path /home/vpopmail/.spamassassin/bayes use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 bayes_auto_learn_threshold_spam 7.0 bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_expire 1 The problem seems to manifest itself when tokens begin to become due to expire. # spamassassin -D bayes --lint shows that the problem server has never sync'd, while the one which hasn't failed (yet) shows a sync'd value. It normally syncs when you train it. Here's my local.cf: ok_locales all skip_rbl_checks 1 required_hits 5 report_safe 0 bayes_auto_learn_threshold_spam 8.0 rewrite_header Subject ***SPAM*** use_pyzor 1 use_dcc 1 use_auto_whitelist 1 use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 use_auto_whitelist 1 whitelist_from [EMAIL PROTECTED] score SARE_LOTTO_SPAM2 3.8 score ADVANCE_FEE_2 2.4 score ADVANCE_FEE_2 4.1 score DIET_1 1.2 score MPART_ALT_DIFF_COUNT 2.1 score BAYES_05 0.5 score BAYES_50 5.0 score BAYES_95 9.5 score BAYES_99 10.0 score PYZOR_CHECK 5 score AWL -20 score DCC_CHECK 4.4 score URIBL_JP_SURBL 9.5 score SARE_GIF_STOX 6.5 score SARE_GIF_ATTACH 2.5 Which is pretty close to yours. I have yet to have a problem with the database (15 months since last touched). I do run a script nightly at midnight that checks my spam folder and learns from it, which is also where it would sync. Here's another server's local.cf (this Bayes DB moved from the original server, to this one about 18 months ago; it probably has 28-30 months running): ok_locales all skip_rbl_checks 1 required_hits 5.0 report_safe 0 rewrite_header Subject ***SPAM*** use_pyzor 1 use_bayes 1 bayes_file_mode 0700 #bayes_path /root/.spamassassin/bayes bayes_path /etc/mail/spamassassin/.spamassassin/bayes bayes_auto_learn_threshold_spam 8.0 bayes_auto_expire 1 #use_dcc 1 use_auto_whitelist 1 auto_whitelist_file_mode 0700 #auto_whitelist_path /root/.spamassassin/auto-whitelist auto_whitelist_path /etc/mail/spamassassin/.spamassassin/auto-whitelist # Whitelist section manually added whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] score SARE_LOTTO_SPAM2 3.8 score ADVANCE_FEE_2 2.4 score ADVANCE_FEE_2 4.1 score DIET_1 1.2 score MPART_ALT_DIFF_COUNT 2.1 score BAYES_05 0.5 score BAYES_50 5.0 score BAYES_95 9.5 score BAYES_99 10.0 score PYZOR_CHECK 5 score AWL -20 score URIBL_JP_SURBL 9.5 score SARE_GIF_STOX 6.5 score SARE_GIF_ATTACH 2.5 And here's a snip of the -D --lint: [932] dbg: bayes: DB journal sync: last sync: 1161817266 [932] dbg: bayes: corpus size: nspam = 18150, nham = 16042 [932] dbg: bayes: score = 0.0886822067616296 [932] dbg: bayes: DB expiry: tokens in DB: 147679, Expiry max size: 15, Oldest atime: 1161421975, Newest atime: 1161835713, Last expire: 1161743994, Current time: 1161836551 [932] dbg: bayes: DB journal sync: last sync: 1161817266 Which is also syncing. This one also runs a learning script nightly. Don't know if this helps you any or not - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] hi, with suse I had do change the bayes_file_mode from 0700 to 666 And I've forgotten about these two: -restart spamd BEFORE the sync. -run sync as root and cp bayes settings from /etc/mail/spamassassin to ~./spamassassin just in case about mysql backend, it IS slower and on a very, very busy server you have to setup a separate instance to run bayes 'cause with too much mail the users start to get erros authenticating. And with a webmail sql backend and sql address books... Uff.. Been there, not good :( --- Zsoft - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt (revisited)
Justice London wrote: What is the ownership of the bayes folder/files? vpopmail:vchkpw on all. 700 on directory, 600 on files. Justice London -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 4:30 PM To: qmailtoaster-list Subject: [qmailtoaster] bayes getting corrupt (revisited) Ok, so here's my situation. I have two apparently identical low-volume servers. One shows the corruption symptoms, the other doesn't (yet). Here is the local.cf (same on both): ok_locales all skip_rbl_checks 0 required_hits 5 report_safe 0 rewrite_header Subject [SPAM] use_pyzor 1 use_auto_whitelist 1 bayes_path /home/vpopmail/.spamassassin/bayes use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 bayes_auto_learn_threshold_spam 7.0 bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_expire 1 The problem seems to manifest itself when tokens begin to become due to expire. # spamassassin -D bayes --lint shows that the problem server has never sync'd, while the one which hasn't failed (yet) shows a sync'd value. Does anyone know when, under normal operation, the bayes db is supposed to be sync'd? I would think that after an autolearn=ham or autolearn=spam that this would happen. On one server it appears to do so (and things are fine), but on one where it doesn't (for a long period of time), the expiration process appears to exhibit problems. Some of this observation may be coincidental, but I'm thinking that not syncing might be the root cause of this problem. I've gotta go for now, but will try to follow up on this thread as soon as I can. -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [qmailtoaster] bayes getting corrupt (revisited)
Okay, I had an issue similar to that at one point due to ownership/perms issues. Looks good. Justice -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Thursday, October 26, 2006 12:27 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt (revisited) Justice London wrote: What is the ownership of the bayes folder/files? vpopmail:vchkpw on all. 700 on directory, 600 on files. Justice London -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 4:30 PM To: qmailtoaster-list Subject: [qmailtoaster] bayes getting corrupt (revisited) Ok, so here's my situation. I have two apparently identical low-volume servers. One shows the corruption symptoms, the other doesn't (yet). Here is the local.cf (same on both): ok_locales all skip_rbl_checks 0 required_hits 5 report_safe 0 rewrite_header Subject [SPAM] use_pyzor 1 use_auto_whitelist 1 bayes_path /home/vpopmail/.spamassassin/bayes use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 bayes_auto_learn_threshold_spam 7.0 bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_expire 1 The problem seems to manifest itself when tokens begin to become due to expire. # spamassassin -D bayes --lint shows that the problem server has never sync'd, while the one which hasn't failed (yet) shows a sync'd value. Does anyone know when, under normal operation, the bayes db is supposed to be sync'd? I would think that after an autolearn=ham or autolearn=spam that this would happen. On one server it appears to do so (and things are fine), but on one where it doesn't (for a long period of time), the expiration process appears to exhibit problems. Some of this observation may be coincidental, but I'm thinking that not syncing might be the root cause of this problem. I've gotta go for now, but will try to follow up on this thread as soon as I can. -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt (revisited)
Eric Shubes wrote: Ok, so here's my situation. I have two apparently identical low-volume servers. One shows the corruption symptoms, the other doesn't (yet). Here is the local.cf (same on both): ok_locales all skip_rbl_checks 0 required_hits 5 report_safe 0 rewrite_header Subject [SPAM] use_pyzor 1 use_auto_whitelist 1 bayes_path /home/vpopmail/.spamassassin/bayes use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 bayes_auto_learn_threshold_spam 7.0 bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_expire 1 The problem seems to manifest itself when tokens begin to become due to expire. # spamassassin -D bayes --lint shows that the problem server has never sync'd, while the one which hasn't failed (yet) shows a sync'd value. Does anyone know when, under normal operation, the bayes db is supposed to be sync'd? I would think that after an autolearn=ham or autolearn=spam that this would happen. On one server it appears to do so (and things are fine), but on one where it doesn't (for a long period of time), the expiration process appears to exhibit problems. Some of this observation may be coincidental, but I'm thinking that not syncing might be the root cause of this problem. I've gotta go for now, but will try to follow up on this thread as soon as I can. Ok, I found out what's happening from the SA list: The child is trying to run a Bayes expire, apparently on a large Bayes database that hasn't had a successful expiry run in some time. This attempt to process the Bayes database is probably taking over 300 seconds, and the child is being timed out and killed by something. As a result of being killed, it never finished the Bayes expire processing. So the next child tries to do the same thing, gets timed out and killed, the nex child tries to do the same thing... Run a manual Bayes expire run and it will probably clean up your problems. If this sort of problem starts to reoccur you might consider turning off bayes auto expire and setting up a cron run to do it once a day or so. (Or more often, depending on your mail volume.) Note (to myself), I changed timeoutsmtpd from 1200 (default 20 minutes) to 60 (1 minute) to keep sluggish connections from backing up. Perhaps that wasn't such a good idea. I think this problem happened before I did that though, but I can't be positive. I think that the toaster should have a permanent fix for this. I'm thinking that auto expire should be turned off, and a daily cron job created to handle expirations. The only other way to fix it I can think of would be to increase the timeout window with auto expire on, but I don't like that idea. 20 minutes seems too long of a period of inactivity for an smtp connection. There's also a risk that the sending server will terminate the connection before the expiration process completes. Suggestions? Gotta go away for a while - will check back when I can. -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt
Citando Eric \\\Shubes\\\ [EMAIL PROTECTED]: Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Hi, I found these particular situation where bayes gets currupt -sa-learn --sync run while db is been update by sa -too many simultaneous acesses to the db solutions 've found drop the allowed number of spamd childs and pause the toaster during nightly sync db Zsoft - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt
[EMAIL PROTECTED] wrote: Citando Eric \\\Shubes\\\ [EMAIL PROTECTED]: Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Hi, I found these particular situation where bayes gets currupt -sa-learn --sync run while db is been update by sa -too many simultaneous acesses to the db solutions 've found drop the allowed number of spamd childs and pause the toaster during nightly sync db Zsoft Appreciate the tips, Z. However, the only sa-learn I'm running (that I know of) is autolearn. I'm running standard toaster settings for spam children, and rarely see 2 spawned. This is on both lightly used severs and a heavily used one. Spam configuration is standard toaster, with the exception of SURBLs, and the following local.cf: ok_locales all skip_rbl_checks 0 required_hits 5 report_safe 0 rewrite_header Subject [SPAM] use_pyzor 1 use_auto_whitelist 1 bayes_path /home/vpopmail/.spamassassin/bayes use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 bayes_auto_learn_threshold_spam 7.0 bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_expire 1 I don't know of a nightly sync db that's running. Is there one in the default setup that I'm running and don't know about? Any thoughts appreciated. -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [qmailtoaster] bayes getting corrupt
I posted to this list with the same issue a while back and pretty much the system would sometimes sync properly automatically, sometimes not. I ended up just having to turn off bayes for the time being. Is there any work toward maybe making DSPAM an install option, rather than just spamassassin? I've seen the install instructions for it with qmail-toaster, but it's a bit hackish. Justice London -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 2:29 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt [EMAIL PROTECTED] wrote: Citando Eric \\\Shubes\\\ [EMAIL PROTECTED]: Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Hi, I found these particular situation where bayes gets currupt -sa-learn --sync run while db is been update by sa -too many simultaneous acesses to the db solutions 've found drop the allowed number of spamd childs and pause the toaster during nightly sync db Zsoft Appreciate the tips, Z. However, the only sa-learn I'm running (that I know of) is autolearn. I'm running standard toaster settings for spam children, and rarely see 2 spawned. This is on both lightly used severs and a heavily used one. Spam configuration is standard toaster, with the exception of SURBLs, and the following local.cf: ok_locales all skip_rbl_checks 0 required_hits 5 report_safe 0 rewrite_header Subject [SPAM] use_pyzor 1 use_auto_whitelist 1 bayes_path /home/vpopmail/.spamassassin/bayes use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 bayes_auto_learn_threshold_spam 7.0 bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_expire 1 I don't know of a nightly sync db that's running. Is there one in the default setup that I'm running and don't know about? Any thoughts appreciated. -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt
Would this be a good reason to use the MySQL Bayes storage engine? Q On Tue, 24 Oct 2006 11:45:31 -0700, Eric Shubes wrote: Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt
BTW What is the effect of your corruption? I do sa-learn --sync once per hour and haven't any problem (yet? :-P ) Would this be a good reason to use the MySQL Bayes storage engine? Q On Tue, 24 Oct 2006 11:45:31 -0700, Eric Shubes wrote: Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt
Well it's time to monitoring cpu/mem before the problem gets worse, don't you think? Yeah, I forgot that part all right. The SA process (task) maxes out the cpu for a while, then seems to go idle until the smtp session times out. This creates a backlog that snowballs. Not quite so bad on a small domain, but very severe with higher volumes. Jared Markell wrote: You forgot about the part where things start to hang and your system tasks start to count in numbers.. and for me, at about 350+ tasks, performance degrades VERY badly, resulting in user unknown errors when users try to log in via pop3, or when connecting smtp servers try to deliver a message. In a nutshell, Bayes getting corrupt = many many phone calls Wtf?!? to me :( At any normal given point in time, my server should run at 140~ tasks at one time, so 300+ is bad. :) That's when I started upping my max children to hopefully scan more emails at a time, but that actually worked against me, causing Bayes to get corrupt much faster and thus clog things up much faster. Jared -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 3:25 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt [EMAIL PROTECTED] wrote: BTW What is the effect of your corruption? Message: 2006-10-18 23:18:40.093043500 [30369] warn: bayes: cannot open bayes databases /home/vpopmail/.spamassassin/bayes_* R/W: lock failed: File exists appears repeatedly in spamd log. The message is accepted ok (received in recipient's inbox), but the SMTP session terminates abnormally so the sending server resends the message, resulting in duplicates. That's pretty much all I've determined at this point. Suggestions? I do sa-learn --sync once per hour and haven't any problem (yet? :-P ) Would this be a good reason to use the MySQL Bayes storage engine? Q On Tue, 24 Oct 2006 11:45:31 -0700, Eric Shubes wrote: Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt
For a low volume server, I've simply renamed the /home/vpopmail/.spamassassin folder and let it recreate the bayes database. I hope to have some time to fix things when it happens again (every 2-3 weeks or so). I have saved the old .spamassassin folders though, so I can restore them for testing if/when necessary. I'd hate to try to diagnose this on a high volume server, but I have a low volume one that I could work with on a weekend, but it hasn't exhibited this behavior yet. Hey, I just noticed something. The server that's not having this problem shows # spamassassin -D bayes --lint [5499] dbg: bayes: DB journal sync: last sync: 1161812436 (among other stuff) On the server with the problem, I get: [6091] dbg: bayes: DB journal sync: last sync: 0 It apparently has never sync'd. Perhaps that's the problem. What is supposed to be doing the syncing? When is it supposed to happen? [EMAIL PROTECTED] wrote: Well it's time to monitoring cpu/mem before the problem gets worse, don't you think? Yeah, I forgot that part all right. The SA process (task) maxes out the cpu for a while, then seems to go idle until the smtp session times out. This creates a backlog that snowballs. Not quite so bad on a small domain, but very severe with higher volumes. Jared Markell wrote: You forgot about the part where things start to hang and your system tasks start to count in numbers.. and for me, at about 350+ tasks, performance degrades VERY badly, resulting in user unknown errors when users try to log in via pop3, or when connecting smtp servers try to deliver a message. In a nutshell, Bayes getting corrupt = many many phone calls Wtf?!? to me :( At any normal given point in time, my server should run at 140~ tasks at one time, so 300+ is bad. :) That's when I started upping my max children to hopefully scan more emails at a time, but that actually worked against me, causing Bayes to get corrupt much faster and thus clog things up much faster. Jared -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 3:25 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt [EMAIL PROTECTED] wrote: BTW What is the effect of your corruption? Message: 2006-10-18 23:18:40.093043500 [30369] warn: bayes: cannot open bayes databases /home/vpopmail/.spamassassin/bayes_* R/W: lock failed: File exists appears repeatedly in spamd log. The message is accepted ok (received in recipient's inbox), but the SMTP session terminates abnormally so the sending server resends the message, resulting in duplicates. That's pretty much all I've determined at this point. Suggestions? I do sa-learn --sync once per hour and haven't any problem (yet? :-P ) Would this be a good reason to use the MySQL Bayes storage engine? Q On Tue, 24 Oct 2006 11:45:31 -0700, Eric Shubes wrote: Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' -- -Eric 'shubes' -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt
Message: 2006-10-18 23:18:40.093043500 [30369] warn: bayes: cannot open bayes databases /home/vpopmail/.spamassassin/bayes_* R/W: lock failed: File exists And you can't correct the problem deleting the bayes.lock file and reruning sa-learn --sync ??? Maybe Quinn is correct in that it's time maybe for bayes mysql backend - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [qmailtoaster] bayes getting corrupt
If you have bayes_auto_expire 1 in your local.cf for spamassassin you should be able to force it to do syncs of the database. The problem I have noticed on high-traffic systems is it doesn't help so much for the most part as then the system just keeps trying to force mail through, making the sync die from what I can tell... then you end up with lock files for that basically and spamassassin comes to a screeching halt. Justice London -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 4:03 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt Message: 2006-10-18 23:18:40.093043500 [30369] warn: bayes: cannot open bayes databases /home/vpopmail/.spamassassin/bayes_* R/W: lock failed: File exists And you can't correct the problem deleting the bayes.lock file and reruning sa-learn --sync ??? Maybe Quinn is correct in that it's time maybe for bayes mysql backend - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [qmailtoaster] bayes getting corrupt
It usually syncs whenever you train it, by default.. You have to tell it otherwise when manually teaching it. sa-learn --no-sync ..extra stuff here.. As per dsanchez's comment.. unless you have 24 hours a day free to monitor a single server's usages, I wouldn't really suggest that. ;) I want an email server that I can leave alone and trust that it will stay working for more than at least a week without me baby sitting it. And, I got this server to be able to be that way - if I disable Bayes. Jared -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 4:02 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt For a low volume server, I've simply renamed the /home/vpopmail/.spamassassin folder and let it recreate the bayes database. I hope to have some time to fix things when it happens again (every 2-3 weeks or so). I have saved the old .spamassassin folders though, so I can restore them for testing if/when necessary. I'd hate to try to diagnose this on a high volume server, but I have a low volume one that I could work with on a weekend, but it hasn't exhibited this behavior yet. Hey, I just noticed something. The server that's not having this problem shows # spamassassin -D bayes --lint [5499] dbg: bayes: DB journal sync: last sync: 1161812436 (among other stuff) On the server with the problem, I get: [6091] dbg: bayes: DB journal sync: last sync: 0 It apparently has never sync'd. Perhaps that's the problem. What is supposed to be doing the syncing? When is it supposed to happen? [EMAIL PROTECTED] wrote: Well it's time to monitoring cpu/mem before the problem gets worse, don't you think? Yeah, I forgot that part all right. The SA process (task) maxes out the cpu for a while, then seems to go idle until the smtp session times out. This creates a backlog that snowballs. Not quite so bad on a small domain, but very severe with higher volumes. Jared Markell wrote: You forgot about the part where things start to hang and your system tasks start to count in numbers.. and for me, at about 350+ tasks, performance degrades VERY badly, resulting in user unknown errors when users try to log in via pop3, or when connecting smtp servers try to deliver a message. In a nutshell, Bayes getting corrupt = many many phone calls Wtf?!? to me :( At any normal given point in time, my server should run at 140~ tasks at one time, so 300+ is bad. :) That's when I started upping my max children to hopefully scan more emails at a time, but that actually worked against me, causing Bayes to get corrupt much faster and thus clog things up much faster. Jared -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 3:25 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt [EMAIL PROTECTED] wrote: BTW What is the effect of your corruption? Message: 2006-10-18 23:18:40.093043500 [30369] warn: bayes: cannot open bayes databases /home/vpopmail/.spamassassin/bayes_* R/W: lock failed: File exists appears repeatedly in spamd log. The message is accepted ok (received in recipient's inbox), but the SMTP session terminates abnormally so the sending server resends the message, resulting in duplicates. That's pretty much all I've determined at this point. Suggestions? I do sa-learn --sync once per hour and haven't any problem (yet? :-P ) Would this be a good reason to use the MySQL Bayes storage engine? Q On Tue, 24 Oct 2006 11:45:31 -0700, Eric Shubes wrote: Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' -- -Eric 'shubes' -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED
RE: [qmailtoaster] bayes getting corrupt
I believe you are right that cdb would be faster than Mysql. However, Mysql's differences would be that it can handle a much much higher volume of requests at a time, and also allows updates to the database in-between requests without interrupting anything. Although, that can also depend on how Bayes is implemented into Mysql and if the person(s) who did said implementation are mysql-competent. Jared -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 4:11 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt [EMAIL PROTECTED] wrote: Message: 2006-10-18 23:18:40.093043500 [30369] warn: bayes: cannot open bayes databases /home/vpopmail/.spamassassin/bayes_* R/W: lock failed: File exists And you can't correct the problem deleting the bayes.lock file and reruning sa-learn --sync ??? I don't care to delete the bayes.lock file w/out knowing what it is that isn't unlocking it. I haven't run sa-learn myself at all, so I can hardly rerun what I haven't run to begin with. ;) Is it possible that autolearn isn't sync'ing? Is it supposed to? (That would seem reasonable to me). Maybe Quinn is correct in that it's time maybe for bayes mysql backend I wouldn't go running off in that direction quite yet. My understanding is that cdb is a bit faster than mysql. Besides which, I don't really see any need for it. What would be the justification? -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt
[EMAIL PROTECTED] wrote: Message: 2006-10-18 23:18:40.093043500 [30369] warn: bayes: cannot open bayes databases /home/vpopmail/.spamassassin/bayes_* R/W: lock failed: File exists And you can't correct the problem deleting the bayes.lock file and reruning sa-learn --sync ??? I don't care to delete the bayes.lock file w/out knowing what it is that isn't unlocking it. I haven't run sa-learn myself at all, so I can hardly rerun what I haven't run to begin with. ;) Is it possible that autolearn isn't sync'ing? Is it supposed to? (That would seem reasonable to me). Maybe Quinn is correct in that it's time maybe for bayes mysql backend I wouldn't go running off in that direction quite yet. My understanding is that cdb is a bit faster than mysql. Besides which, I don't really see any need for it. What would be the justification? Users in other forums tell MySQL is much more reliable than BerkelyDB with locks and the like. -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt
Justice London wrote: If you have bayes_auto_expire 1 in your local.cf for spamassassin you should be able to force it to do syncs of the database. I have this setting turned on all right. I'd rather not force it though. I'd rather it sync properly on its own! The problem I have noticed on high-traffic systems is it doesn't help so much for the most part as then the system just keeps trying to force mail through, making the sync die from what I can tell... then you end up with lock files for that basically and spamassassin comes to a screeching halt. Justice London Yeah, we need to find the permanent fix for this. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 4:03 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt Message: 2006-10-18 23:18:40.093043500 [30369] warn: bayes: cannot open bayes databases /home/vpopmail/.spamassassin/bayes_* R/W: lock failed: File exists And you can't correct the problem deleting the bayes.lock file and reruning sa-learn --sync ??? Maybe Quinn is correct in that it's time maybe for bayes mysql backend -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [qmailtoaster] bayes getting corrupt
It usually syncs whenever you train it, by default.. You have to tell it otherwise when manually teaching it. sa-learn --no-sync ..extra stuff here.. As per dsanchez's comment.. unless you have 24 hours a day free to monitor a single server's usages, I wouldn't really suggest that. ;) I want an email server that I can leave alone and trust that it will stay working for more than at least a week without me baby sitting it. And, I got this server to be able to be that way - if I disable Bayes. Have you tried bayes mysql backend http://search.cpan.org/src/FELICITY/Mail-SpamAssassin-3.0.2/sql/README.bayes Jared -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 4:02 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt For a low volume server, I've simply renamed the /home/vpopmail/.spamassassin folder and let it recreate the bayes database. I hope to have some time to fix things when it happens again (every 2-3 weeks or so). I have saved the old .spamassassin folders though, so I can restore them for testing if/when necessary. I'd hate to try to diagnose this on a high volume server, but I have a low volume one that I could work with on a weekend, but it hasn't exhibited this behavior yet. Hey, I just noticed something. The server that's not having this problem shows # spamassassin -D bayes --lint [5499] dbg: bayes: DB journal sync: last sync: 1161812436 (among other stuff) On the server with the problem, I get: [6091] dbg: bayes: DB journal sync: last sync: 0 It apparently has never sync'd. Perhaps that's the problem. What is supposed to be doing the syncing? When is it supposed to happen? [EMAIL PROTECTED] wrote: Well it's time to monitoring cpu/mem before the problem gets worse, don't you think? Yeah, I forgot that part all right. The SA process (task) maxes out the cpu for a while, then seems to go idle until the smtp session times out. This creates a backlog that snowballs. Not quite so bad on a small domain, but very severe with higher volumes. Jared Markell wrote: You forgot about the part where things start to hang and your system tasks start to count in numbers.. and for me, at about 350+ tasks, performance degrades VERY badly, resulting in user unknown errors when users try to log in via pop3, or when connecting smtp servers try to deliver a message. In a nutshell, Bayes getting corrupt = many many phone calls Wtf?!? to me :( At any normal given point in time, my server should run at 140~ tasks at one time, so 300+ is bad. :) That's when I started upping my max children to hopefully scan more emails at a time, but that actually worked against me, causing Bayes to get corrupt much faster and thus clog things up much faster. Jared -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 3:25 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt [EMAIL PROTECTED] wrote: BTW What is the effect of your corruption? Message: 2006-10-18 23:18:40.093043500 [30369] warn: bayes: cannot open bayes databases /home/vpopmail/.spamassassin/bayes_* R/W: lock failed: File exists appears repeatedly in spamd log. The message is accepted ok (received in recipient's inbox), but the SMTP session terminates abnormally so the sending server resends the message, resulting in duplicates. That's pretty much all I've determined at this point. Suggestions? I do sa-learn --sync once per hour and haven't any problem (yet? :-P ) Would this be a good reason to use the MySQL Bayes storage engine? Q On Tue, 24 Oct 2006 11:45:31 -0700, Eric Shubes wrote: Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' -- -Eric 'shubes' -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org
Re: [qmailtoaster] bayes getting corrupt
[EMAIL PROTECTED] wrote: It usually syncs whenever you train it, by default.. You have to tell it otherwise when manually teaching it. sa-learn --no-sync ..extra stuff here.. As per dsanchez's comment.. unless you have 24 hours a day free to monitor a single server's usages, I wouldn't really suggest that. ;) I want an email server that I can leave alone and trust that it will stay working for more than at least a week without me baby sitting it. And, I got this server to be able to be that way - if I disable Bayes. Have you tried bayes mysql backend http://search.cpan.org/src/FELICITY/Mail-SpamAssassin-3.0.2/sql/README.bayes No. I'd rather get the cdb implementation fixed (first). There's no good reason (that I know of) why it shouldn't work properly. Especially with the toaster. Jared -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 4:02 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt For a low volume server, I've simply renamed the /home/vpopmail/.spamassassin folder and let it recreate the bayes database. I hope to have some time to fix things when it happens again (every 2-3 weeks or so). I have saved the old .spamassassin folders though, so I can restore them for testing if/when necessary. I'd hate to try to diagnose this on a high volume server, but I have a low volume one that I could work with on a weekend, but it hasn't exhibited this behavior yet. Hey, I just noticed something. The server that's not having this problem shows # spamassassin -D bayes --lint [5499] dbg: bayes: DB journal sync: last sync: 1161812436 (among other stuff) On the server with the problem, I get: [6091] dbg: bayes: DB journal sync: last sync: 0 It apparently has never sync'd. Perhaps that's the problem. What is supposed to be doing the syncing? When is it supposed to happen? [EMAIL PROTECTED] wrote: Well it's time to monitoring cpu/mem before the problem gets worse, don't you think? Yeah, I forgot that part all right. The SA process (task) maxes out the cpu for a while, then seems to go idle until the smtp session times out. This creates a backlog that snowballs. Not quite so bad on a small domain, but very severe with higher volumes. Jared Markell wrote: You forgot about the part where things start to hang and your system tasks start to count in numbers.. and for me, at about 350+ tasks, performance degrades VERY badly, resulting in user unknown errors when users try to log in via pop3, or when connecting smtp servers try to deliver a message. In a nutshell, Bayes getting corrupt = many many phone calls Wtf?!? to me :( At any normal given point in time, my server should run at 140~ tasks at one time, so 300+ is bad. :) That's when I started upping my max children to hopefully scan more emails at a time, but that actually worked against me, causing Bayes to get corrupt much faster and thus clog things up much faster. Jared -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 3:25 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] bayes getting corrupt [EMAIL PROTECTED] wrote: BTW What is the effect of your corruption? Message: 2006-10-18 23:18:40.093043500 [30369] warn: bayes: cannot open bayes databases /home/vpopmail/.spamassassin/bayes_* R/W: lock failed: File exists appears repeatedly in spamd log. The message is accepted ok (received in recipient's inbox), but the SMTP session terminates abnormally so the sending server resends the message, resulting in duplicates. That's pretty much all I've determined at this point. Suggestions? I do sa-learn --sync once per hour and haven't any problem (yet? :-P ) Would this be a good reason to use the MySQL Bayes storage engine? Q On Tue, 24 Oct 2006 11:45:31 -0700, Eric Shubes wrote: Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' -- -Eric 'shubes' -- -Eric 'shubes' -- -Eric 'shubes
RE: [qmailtoaster] bayes getting corrupt (revisited)
What is the ownership of the bayes folder/files? Justice London -Original Message- From: Eric Shubes [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 25, 2006 4:30 PM To: qmailtoaster-list Subject: [qmailtoaster] bayes getting corrupt (revisited) Ok, so here's my situation. I have two apparently identical low-volume servers. One shows the corruption symptoms, the other doesn't (yet). Here is the local.cf (same on both): ok_locales all skip_rbl_checks 0 required_hits 5 report_safe 0 rewrite_header Subject [SPAM] use_pyzor 1 use_auto_whitelist 1 bayes_path /home/vpopmail/.spamassassin/bayes use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 bayes_auto_learn_threshold_spam 7.0 bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_expire 1 The problem seems to manifest itself when tokens begin to become due to expire. # spamassassin -D bayes --lint shows that the problem server has never sync'd, while the one which hasn't failed (yet) shows a sync'd value. Does anyone know when, under normal operation, the bayes db is supposed to be sync'd? I would think that after an autolearn=ham or autolearn=spam that this would happen. On one server it appears to do so (and things are fine), but on one where it doesn't (for a long period of time), the expiration process appears to exhibit problems. Some of this observation may be coincidental, but I'm thinking that not syncing might be the root cause of this problem. I've gotta go for now, but will try to follow up on this thread as soon as I can. -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt (revisited)
Eric Shubes wrote: Ok, so here's my situation. I have two apparently identical low-volume servers. One shows the corruption symptoms, the other doesn't (yet). Here is the local.cf (same on both): ok_locales all skip_rbl_checks 0 required_hits 5 report_safe 0 rewrite_header Subject [SPAM] use_pyzor 1 use_auto_whitelist 1 bayes_path /home/vpopmail/.spamassassin/bayes use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 bayes_auto_learn_threshold_spam 7.0 bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_expire 1 The problem seems to manifest itself when tokens begin to become due to expire. # spamassassin -D bayes --lint shows that the problem server has never sync'd, while the one which hasn't failed (yet) shows a sync'd value. It normally syncs when you train it. Here's my local.cf: ok_locales all skip_rbl_checks 1 required_hits 5 report_safe 0 bayes_auto_learn_threshold_spam 8.0 rewrite_header Subject ***SPAM*** use_pyzor 1 use_dcc 1 use_auto_whitelist 1 use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 use_auto_whitelist 1 whitelist_from [EMAIL PROTECTED] score SARE_LOTTO_SPAM2 3.8 score ADVANCE_FEE_2 2.4 score ADVANCE_FEE_2 4.1 score DIET_1 1.2 score MPART_ALT_DIFF_COUNT 2.1 score BAYES_05 0.5 score BAYES_50 5.0 score BAYES_95 9.5 score BAYES_99 10.0 score PYZOR_CHECK 5 score AWL -20 score DCC_CHECK 4.4 score URIBL_JP_SURBL 9.5 score SARE_GIF_STOX 6.5 score SARE_GIF_ATTACH 2.5 Which is pretty close to yours. I have yet to have a problem with the database (15 months since last touched). I do run a script nightly at midnight that checks my spam folder and learns from it, which is also where it would sync. Here's another server's local.cf (this Bayes DB moved from the original server, to this one about 18 months ago; it probably has 28-30 months running): ok_locales all skip_rbl_checks 1 required_hits 5.0 report_safe 0 rewrite_header Subject ***SPAM*** use_pyzor 1 use_bayes 1 bayes_file_mode 0700 #bayes_path /root/.spamassassin/bayes bayes_path /etc/mail/spamassassin/.spamassassin/bayes bayes_auto_learn_threshold_spam 8.0 bayes_auto_expire 1 #use_dcc 1 use_auto_whitelist 1 auto_whitelist_file_mode 0700 #auto_whitelist_path /root/.spamassassin/auto-whitelist auto_whitelist_path /etc/mail/spamassassin/.spamassassin/auto-whitelist # Whitelist section manually added whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] whitelist_from [EMAIL PROTECTED] score SARE_LOTTO_SPAM2 3.8 score ADVANCE_FEE_2 2.4 score ADVANCE_FEE_2 4.1 score DIET_1 1.2 score MPART_ALT_DIFF_COUNT 2.1 score BAYES_05 0.5 score BAYES_50 5.0 score BAYES_95 9.5 score BAYES_99 10.0 score PYZOR_CHECK 5 score AWL -20 score URIBL_JP_SURBL 9.5 score SARE_GIF_STOX 6.5 score SARE_GIF_ATTACH 2.5 And here's a snip of the -D --lint: [932] dbg: bayes: DB journal sync: last sync: 1161817266 [932] dbg: bayes: corpus size: nspam = 18150, nham = 16042 [932] dbg: bayes: score = 0.0886822067616296 [932] dbg: bayes: DB expiry: tokens in DB: 147679, Expiry max size: 15, Oldest atime: 1161421975, Newest atime: 1161835713, Last expire: 1161743994, Current time: 1161836551 [932] dbg: bayes: DB journal sync: last sync: 1161817266 Which is also syncing. This one also runs a learning script nightly. Don't know if this helps you any or not - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] bayes getting corrupt
Jared Markell wrote: Does anyone else have the problem of the Bayes database getting corrupt very often? My database has problems quite frequently, I usually have to delete the .spamassassin folder once every 3 days, and the only way to keep the server stable is just to disable Bayes checking all together. However, Bayes helps a lot and I would like to keep it if I could trust it not to corrupt and clog up the email server. Anyone else have these problems and/or have a solution? *Jared * ** I have this problem too. I've been renaming .spamassassin when it starts happening, so I have the old files to do some diagnostics with. I simply haven't gotten around to it, but would like very much to get it fixed. It seems to be ok until the point where tokens are due to expire. I'm guessing that the expiration process has a problem, but don't rightly know. Any cdb gurus here, or does anyone know of a fix in this area? -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
