Re: HUGE bayes DB (non-sitewide) advice?

2005-11-10 Thread email builder
Just a follow-up to my own brain-lapse: If you define a custom user scores query like this: user_scores_sql_custom_querySELECT preference, value FROM spamassassin_settings WHERE username = _USERNAME_ OR username = '!GLOBAL' OR username = CONCAT('@', _DOMAIN_) ORDER BY username

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-09 Thread email builder
Our production database for a large number of emails (but using site wide) is about 40mb. What is your bayes_expiry_max_db_size set to? Do you feel that it has been enough to effectively capture your various user email habits? Default. How can you be running the default value,

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-09 Thread email builder
I guess the relevant point for this thread is that I don't necessarily think that this is the silver bullet as implied. Even if you use a high-availability clustering technology that can mirror writes and reads, you are STILL dealing with the possibility of a database that is just

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-09 Thread Michael Parker
email builder wrote: How can you be running the default value, when the manual says that 15 tokens is only 8MB?? How do you end up with 40MB of data?: bayes_expiry_max_db_size (default: 15) What should be the maximum size of the Bayes tokens database? When expiry occurs, the Bayes

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-09 Thread Michael Parker
Gary W. Smith wrote: Just my $0.02 but if it's in MySQL then you really don't need to expire each one. You can write a custom script that will do this. When you break it down, expire is really just finding those tokens that are beyond the threshold where id=x and time=y. The resultant would

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-09 Thread Michael Parker
email builder wrote: Well, I know there have to be some admins out there who have a lot of users and do not use sitewide bayes.. RIGHT? See original email snippet at bottom. I believe that there are a few running bayes is a similar configuration. It certainly is a tough problem. I

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-09 Thread Gary W. Smith
, November 09, 2005 1:54 AM To: Gary W. Smith; users@spamassassin.apache.org Subject: RE: HUGE bayes DB (non-sitewide) advice? Our production database for a large number of emails (but using site wide) is about 40mb. What is your bayes_expiry_max_db_size set to? Do you feel that it has been

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-09 Thread Gary W. Smith
PROTECTED] Sent: Wednesday, November 09, 2005 1:54 AM To: Gary W. Smith; users@spamassassin.apache.org Subject: RE: HUGE bayes DB (non-sitewide) advice? Our production database for a large number of emails (but using site wide) is about 40mb. What is your bayes_expiry_max_db_size set to? Do

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-09 Thread email builder
Thanks a lot for checking, Gary! --- Gary W. Smith [EMAIL PROTECTED] wrote: You're right, my guy gave me the size of bayes + awl. The real number is 14.5mb. (with an overhead of 3.2mb). Not sure, that's just what phpmyadmin is reporting. I'll check again. I can't remember if the DB is

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-09 Thread email builder
Well, I know there have to be some admins out there who have a lot of users and do not use sitewide bayes.. RIGHT? See original email snippet at bottom. snip * Other ideas: - increase system memory as much as possible - per-domain Bayes instead of per-user???

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-08 Thread Michael Monnerie
On Dienstag, 8. November 2005 03:50 email builder wrote: From what I understand, MySQL cluster design is such that the data nodes keep all the table data in memory, which would not be feasible in a 160GB scenario... No. Cluster means: Take two machines of same config, and mirror them. It's

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-08 Thread Gary W. Smith
Monnerie [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 08, 2005 9:48 AM To: users@spamassassin.apache.org Subject: Re: HUGE bayes DB (non-sitewide) advice? On Dienstag, 8. November 2005 03:38 email builder wrote: In-memory storage: All data stored in each data node is kept in memory

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-08 Thread email builder
In-memory storage: All data stored in each data node is kept in memory on the node's host computer. For each data node in the cluster, you must have available an amount of RAM equal to the size of the database times the number of replicas, This refers to the first line: In-memory

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-08 Thread email builder
Our production database for a large number of emails (but using site wide) is about 40mb. What is your bayes_expiry_max_db_size set to? Do you feel that it has been enough to effectively capture your various user email habits? __ Start your

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-08 Thread Gary W. Smith
Default. Gart -Original Message- From: email builder [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 08, 2005 8:58 PM To: Gary W. Smith; users@spamassassin.apache.org Subject: RE: HUGE bayes DB (non-sitewide) advice? Our production database for a large number of emails (but using

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-08 Thread Gary W. Smith
To: Gary W. Smith; users@spamassassin.apache.org Subject: RE: HUGE bayes DB (non-sitewide) advice? Our production database for a large number of emails (but using site wide) is about 40mb. What is your bayes_expiry_max_db_size set to? Do you feel that it has been enough to effectively

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-08 Thread Dhawal Doshy
email builder wrote: In-memory storage: All data stored in each data node is kept in memory on the node's host computer. For each data node in the cluster, you must have available an amount of RAM equal to the size of the database times the number of replicas, This refers to the first line:

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-07 Thread email builder
Well, I know there have to be some admins out there who have a lot of users and do not use sitewide bayes.. RIGHT? See original email snippet at bottom. I'll start the ball rolling with what few tweaks we've made, although they are not enough; we desperately need more ideas to make this

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-07 Thread Gary W. Smith
an opinion. Gary Wayne Smith -Original Message- From: email builder [mailto:[EMAIL PROTECTED] Sent: Monday, November 07, 2005 10:56 AM To: [EMAIL PROTECTED]; users@spamassassin.apache.org Subject: Re: HUGE bayes DB (non-sitewide) advice? Well, I know there have to be some admins out

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-07 Thread email builder
Well, I know there have to be some admins out there who have a lot of users and do not use sitewide bayes.. RIGHT? See original email snippet at bottom. snip * Other ideas: - increase system memory as much as possible - per-domain Bayes instead of per-user??? This might be

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-07 Thread Gary W. Smith
We run a linux-ha cluster. Works out well. -Original Message- From: email builder [mailto:[EMAIL PROTECTED] Sent: Monday, November 07, 2005 6:51 PM To: users@spamassassin.apache.org Subject: RE: HUGE bayes DB (non-sitewide) advice? From what I understand, MySQL cluster design

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-04 Thread email builder
As a result of this, however, we are currently burdened with an 8GB(! yep, you read it right) bayes database (more than 20K users having mail delivered). Consider using bayes_expiry_max_db_size in conjunction with bayes_auto_expire Using? So you are saying you use non-sitewide

Re: HUGE bayes DB (non-sitewide) advice?

2005-11-04 Thread Michael Monnerie
On Freitag, 4. November 2005 21:04 email builder wrote: *SOMEONE* out there has to be doing something like this, no??? I would be interested in that, too. mfg zmi -- // Michael Monnerie, Ing.BSc --- it-management Michael Monnerie // http://zmi.at Tel: 0660/4156531 Linux

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-03 Thread Matthew.van.Eerde
email builder wrote: As a result of this, however, we are currently burdened with an 8GB(! yep, you read it right) bayes database (more than 20K users having mail delivered). Consider using bayes_expiry_max_db_size in conjunction with bayes_auto_expire -- Matthew.van.Eerde (at) hbinc.com

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-03 Thread email builder
--- [EMAIL PROTECTED] wrote: email builder wrote: As a result of this, however, we are currently burdened with an 8GB(! yep, you read it right) bayes database (more than 20K users having mail delivered). Consider using bayes_expiry_max_db_size in conjunction with bayes_auto_expire

RE: HUGE bayes DB (non-sitewide) advice?

2005-11-03 Thread Matthew.van.Eerde
email builder wrote: As a result of this, however, we are currently burdened with an 8GB(! yep, you read it right) bayes database (more than 20K users having mail delivered). Consider using bayes_expiry_max_db_size in conjunction with bayes_auto_expire Using? So you are saying you