Re: [AMaViS-user] MySQL InnoDB Performance - Ultra Slow Queries

2010-07-06 Thread Nathan M
On Fri, Jul 2, 2010 at 8:41 PM, Rodolfo Gonzalez Gonzalez rgonza...@gnt.cc wrote: On 02/07/10 19:29, Nathan M wrote: The thing that gets me is the Query time being 264 seconds that when watching status sits in status 'Copying to tmp table'.  I know InnoDB You got it. When MySQL goes copying

Re: [AMaViS-user] MySQL InnoDB Performance - Ultra Slow Queries

2010-07-06 Thread Nathan M
On Sat, Jul 3, 2010 at 4:40 AM, Michael Scheidell scheid...@secnap.net wrote: On 7/2/10 8:29 PM, Nathan M wrote: I've got a production amavisd-new array of servers handling roughly 16,000,000 messages per day. Users can toggle quarantining. With the quarantines that are enabled and a 2 week

Re: [AMaViS-user] MySQL InnoDB Performance - Ultra Slow Queries

2010-07-06 Thread Nathan M
On Tue, Jul 6, 2010 at 10:34 AM, Michael Scheidell scheid...@secnap.net wrote: partition_tag is a key, yes use it, and DON'T use 'in' (as: partition_tag in (18,19) or partition_tag between, etc.  some versions of mysql mess optimization up. I tried it out by adding the active (3) partition

[AMaViS-user] MySQL InnoDB Performance - Ultra Slow Queries

2010-07-02 Thread Nathan M
I've got a production amavisd-new array of servers handling roughly 16,000,000 messages per day. Users can toggle quarantining. With the quarantines that are enabled and a 2 week retention we have roughly 12 million message in quarantine at any given time. We send a summary email to subscribers

[AMaViS-user] Tag message, deliver to quarantine mailbox (amavis + postfix)

2010-04-01 Thread Nathan M
Hello, Curious if anybody has accomplished the following and/or can give me any tips on how to accomplish. I would like to instead of amavis delivering quarantine to a SQL database (how we have it now), instead have it perhaps add a header: X-Quarantine: TRUE or something to that effect. Second

Re: [AMaViS-user] Mysql 5.1 + Partitioning Storage Schema / Cleanup

2009-08-13 Thread Nathan M
On Fri, Aug 7, 2009 at 5:46 PM, Nathan Mlocu.li...@gmail.com wrote: I'm open to suggestions on this front from anybody actively using mysql and partitions.  I am using the following config: $sql_partition_tag = sub { my($msginfo)=...@_; iso8601_week($msginfo-rx_time) }; We maintain

[AMaViS-user] Mysql 5.1 + Partitioning Storage Schema / Cleanup

2009-08-07 Thread Nathan M
I'm open to suggestions on this front from anybody actively using mysql and partitions. I am using the following config: $sql_partition_tag = sub { my($msginfo)=...@_; iso8601_week($msginfo-rx_time) }; We maintain quarantine for 3 weeks maximum, therefore my suggested partitioning looks like

Re: [AMaViS-user] Q about partitioning tables and data types

2009-08-03 Thread Nathan M
On Mon, Aug 3, 2009 at 9:22 AM, Luis Daniel Lucio Quirozluis.daniel.lu...@gmail.com wrote: OK, and I've already a database with some emails, if I change $sql_partition_tag = 1, will I mess the DB? $sql_partition_tag = 1; would do little good. If you have the db setup with partition_tag, it's

[AMaViS-user] POLL: Database Hardware vs Daily Messages

2009-08-03 Thread Nathan M
I am curious what amavisd-new users out there have for DB setups for maintaining the storage component of amavis data (maddr, msgrcpt, msgs, quarantine tables). I'll start. Incoming Email: 10,000,000 messages/day Current Hardware: Single Core 2000Mhz Athlon 64 CPU 4GB RAM 2 x Seagate ES 500GB

Re: [AMaViS-user] Database Maintenance - Timeout for Quarantine

2009-08-02 Thread Nathan M
On Thu, Jul 30, 2009 at 9:44 PM, Nathan Mlocu.li...@gmail.com wrote:    'ins_quar' =      'INSERT INTO quarantine (partition_tagBREAKME, mail_id, chunk_ind, mail_text)'.      ' VALUES (?,?,?,?)', One tiny edit here. In testing this worked fine; however, the above change does no good when

Re: [AMaViS-user] Q about partitioning tables and data types

2009-08-01 Thread Nathan M
On Fri, Jul 31, 2009 at 7:59 PM, Luis Daniel Lucio Quirozluis.daniel.lu...@gmail.com wrote: A stupid question, what advantages do I get if I use partition_tag with amavisd+mysql? Faster database cleanups. On a high traffic mail server, cleanups can begin to take days instead of minutes.

Re: [AMaViS-user] MySQL server has gone away

2009-07-31 Thread Nathan M
On Fri, Jul 31, 2009 at 4:26 AM, Sahil Tandonsa...@tandon.net wrote: I've seen discussion of this before with respect to mysql's wait_timeout variable, but my question is a little different. If I restart amavisd-new and mysql, mail flows fine.  This is a test machine so if I try to send

Re: [AMaViS-user] Database Maintenance - Timeout for Quarantine

2009-07-30 Thread Nathan M
On Thu, Jul 30, 2009 at 11:47 AM, Michael Scheidellscheid...@secnap.net wrote: 50gb is a lot of quarantine storage. you should probably schedule the cleanup weekly, or monthly, early in the am. if you want, pause (stop) amavisd while it is happening, or add table write locks first. I

Re: [AMaViS-user] Database Maintenance - Timeout for Quarantine

2009-07-30 Thread Nathan M
On Thu, Jul 30, 2009 at 12:07 PM, Michael Scheidellscheid...@secnap.net wrote: role it in.  let amavisd add partition_tag values (for now) without doing the mysql thing. in (14 days?)  how long is your retention period?) in 14 days, anything without a partition tag is useless, then just

Re: [AMaViS-user] Database Maintenance - Timeout for Quarantine

2009-07-30 Thread Nathan M
On Mon, Jul 27, 2009 at 3:58 PM, Nathan Mlocu.li...@gmail.com wrote: Hello, We have roughly a 50GB quarantine table on a MySQL InnoDB database. Running amavisd-new-2.6.2.  I want to do some maintenance on the quarantine table; however, am concerned it is going to take many many hours to

[AMaViS-user] Database Maintenance - Timeout for Quarantine

2009-07-27 Thread Nathan M
Hello, We have roughly a 50GB quarantine table on a MySQL InnoDB database. Running amavisd-new-2.6.2. I want to do some maintenance on the quarantine table; however, am concerned it is going to take many many hours to complete. My goal is to not stop email delivery; however, I do not mind