Re: [vchkpw] Suggestions for improving performance

2004-02-29 Thread Yuri Nosyrev
28.02.2004, 18:23, Erwin Hoffmann wrote:
 2. To avoid processing of mail for nonexisting accounts you might want to
 use my RECIPIENTS extension for qmail-smtpd. To support virtutal domains
 you have to compile it without the locals restriction. 

about spamcontrol, RECIPIENT extension and vpopmail:

when I compiled spamcontrol with locals=no, it seems to me qmail-smtpd
checks not only domains in locals, virtualdomains, rcpthosts, but EVERY
outgoing email address, doesn't it? 
well, sometimes it can help us to selective relaying to external email
world, but usually wrong...

so, for full vpopmail support you have to compile spamcontrol with
local=yes, but change line 38 (spamcontrol-2.2.9) at recipients.c 
from:
  flaglocals = control_readfile(locals,control/locals,0);
to
  flaglocals = control_readfile(locals,control/rcpthosts,0);

since usually no one domain in vpopmail configuration is really local

but, it might be wrong if we are secondary MX for some domain!!! 
in this case it MUST another file, which is 
ls -l /home/vpopmail/domains  /var/qmail/control/domains)

-- 
-
Best regards,
Yuri Nosyrevmailto: nua at slavel.ru



Re: [vchkpw] Suggestions for improving performance

2004-02-28 Thread Erwin Hoffmann
Hi Jeff,

At 13:53 27.02.04 -0500, Jeff Koch wrote:

Does anybody have any suggestion for improving the performance of a 
mailserver runnning qmail/vpopmail/qmailadmin/qmail-scanner/spamassassin? 
We're using a 2.4Ghz P4 with 1GB RAM and a 40GB SCSI drive. The operating 
system is a basic RH8.0 install with the ext3 journalling file system. 
We're handling about 50K messages/day and seem to exhausting the 
capabilities of the server. During peak periods CPU idle time reaches 0% 
and load averages will exceed 10.0

We've seen some recommendations ( 
http://people.redhat.com/alikins/system_tuning.html , for example) for 
changing hard disk and file system parameters mostly having to do with 
read/write caching and cache sizes and minumum delays before writing to 
disk but we have been afraid to try any of these for fear of making things 
worse.

We would welcome any suggestions or URL's that you could point to - 
particularly with respect to performance tuning that will work with qmail.

As been discussed, probably you are suffering from mails to non-existing
accounts and double-bounces.

Suggestions:

1. To get rid of the double-bounces, use Russel's doublebouncetrim patch.

2. To avoid processing of mail for nonexisting accounts you might want to
use my RECIPIENTS extension for qmail-smtpd. To support virtutal domains
you have to compile it without the locals restriction. In order to
support a domain-based adding and removal of users you have to do some
(trivial) coding. However, unlike John Levine's badrcptto (or even mine)
the Users/Recipients Database can be kept autonomous for each domain.

It would be a good start, to analyse your traffic situation in the first
place. You can use my newanalyse package to do that.

My SPAMCONTORL includes all necessary patches as well.

See:

http://www.fehcom.de/qmail.html

Here, you get all sources.

regards.
--eh.

Dr. Erwin Hoffmann | FEHCom | http://www.fehcom.de/
Wiener Weg 8, 50858 Cologne | T: +49 221 484 4923 | F: ...24


Re: [vchkpw] Suggestions for improving performance

2004-02-27 Thread Jeremy Kitchen
On Fri, 2004-02-27 at 12:53, Jeff Koch wrote:
 Does anybody have any suggestion for improving the performance of a 
 mailserver runnning qmail/vpopmail/qmailadmin/qmail-scanner/spamassassin? 

are you using spamd and having qmail-scanner call spamc?  or is it
calling spamassassin.  if the latter, change it to spamd/spamc model,
much more efficient.

 We're using a 2.4Ghz P4 with 1GB RAM and a 40GB SCSI drive. The operating 
 system is a basic RH8.0 install with the ext3 journalling file system. 
 We're handling about 50K messages/day and seem to exhausting the 
 capabilities of the server. During peak periods CPU idle time reaches 0% 
 and load averages will exceed 10.0

with ext3 you have to be careful about catchall accounts, and accounts
that nobody checks that get a lot of email.  If there are say, 25000
messages in an account and it gets 100 messages per hour, that will
bring your server performance down quite a bit.

of course, using reiserfs on the mail store will fix that problem.  I've
had over 150k emails in one account with reiserfs, and the system wasn't
even being phased as I pumped more mail at it (was load testing for a
customer)

 We would welcome any suggestions or URL's that you could point to - 
 particularly with respect to performance tuning that will work with qmail.

what is the approximate queue load, and what is your conf-split value?

do you have big-{ext-,}todo?

-Jeremy

-- 
Jeremy Kitchen
Systems Administrator
[EMAIL PROTECTED]
Kitchen @ #qmail on EFNet - Join the party!
.
Inter7 Internet Technologies, Inc.
www.inter7.com
866.528.3530 toll free
847.492.0470 int'l
847.492.0632 fax
GNUPG key ID: 93BDD6CE



Re: [vchkpw] Suggestions for improving performance

2004-02-27 Thread Jeff Koch
Hi Jeremy:

Thanks for replying.

We're using spamc in a maildrop filter called for each pop account (via 
qmailadmin's spam enable setting). User SA prefs are maintained by. MySQL

Good suggestion on the catch-alls - we'll do a scan for accounts with many 
emails. We do have some with 10 to 15,000 emails. ReiserFS was not an 
option on RH8.0 unfortunately.

We've got 28,000 emails in the queue - not preprocessed is zero.
- concurrencyincoming = 30
- debugging is turned on for qmail-scan (seems to be the only way to 
generate a log of all messages processed) but the log is closed and zipped 
each night.
- smtpd log shows we're using about 15 of the 30 incoming connections.
- softlimit in /var/qmail/supervise/qmail-smtpd/run is 30MB
- vpopmail is configured for MySQL auth with all domains in one table.
- we use mfcheck and the checkuser patch (mysql) to drop questionable 
connections.

I don't know what you mean by 'conf-split' or big-{ext-}.

At 02:01 PM 2/27/2004, you wrote:
On Fri, 2004-02-27 at 12:53, Jeff Koch wrote:
 Does anybody have any suggestion for improving the performance of a
 mailserver runnning qmail/vpopmail/qmailadmin/qmail-scanner/spamassassin?
are you using spamd and having qmail-scanner call spamc?  or is it
calling spamassassin.  if the latter, change it to spamd/spamc model,
much more efficient.
 We're using a 2.4Ghz P4 with 1GB RAM and a 40GB SCSI drive. The operating
 system is a basic RH8.0 install with the ext3 journalling file system.
 We're handling about 50K messages/day and seem to exhausting the
 capabilities of the server. During peak periods CPU idle time reaches 0%
 and load averages will exceed 10.0
with ext3 you have to be careful about catchall accounts, and accounts
that nobody checks that get a lot of email.  If there are say, 25000
messages in an account and it gets 100 messages per hour, that will
bring your server performance down quite a bit.
of course, using reiserfs on the mail store will fix that problem.  I've
had over 150k emails in one account with reiserfs, and the system wasn't
even being phased as I pumped more mail at it (was load testing for a
customer)
 We would welcome any suggestions or URL's that you could point to -
 particularly with respect to performance tuning that will work with qmail.
what is the approximate queue load, and what is your conf-split value?

do you have big-{ext-,}todo?

-Jeremy

--
Jeremy Kitchen
Systems Administrator
[EMAIL PROTECTED]
Kitchen @ #qmail on EFNet - Join the party!
.
Inter7 Internet Technologies, Inc.
www.inter7.com
866.528.3530 toll free
847.492.0470 int'l
847.492.0632 fax
GNUPG key ID: 93BDD6CE
Best Regards,

Jeff Koch 




Re: [vchkpw] Suggestions for improving performance

2004-02-27 Thread Doug Clements
28000 mails in the queue and you're using the chkuser patch? My guess is
that over 27000 of those are spam double-bounces waiting to be delivered. I
might suggest setting up another machine to be a smart smtp host to handle
outgoing mail. Trying to resend all that mail takes non-trivial resources.
You might also consider lowering your queuelifetime, which will
double-bounce things quicker that will likely just sit in the queue until
they expire.

--Doug

Jeff Koch wrote:
 Hi Jeremy:

 Thanks for replying.

 We're using spamc in a maildrop filter called for each pop account
 (via qmailadmin's spam enable setting). User SA prefs are maintained
 by. MySQL

 Good suggestion on the catch-alls - we'll do a scan for accounts with
 many emails. We do have some with 10 to 15,000 emails. ReiserFS was
 not an option on RH8.0 unfortunately.

 We've got 28,000 emails in the queue - not preprocessed is zero.
 - concurrencyincoming = 30
 - debugging is turned on for qmail-scan (seems to be the only way to
 generate a log of all messages processed) but the log is closed and
 zipped each night.
 - smtpd log shows we're using about 15 of the 30 incoming connections.
 - softlimit in /var/qmail/supervise/qmail-smtpd/run is 30MB
 - vpopmail is configured for MySQL auth with all domains in one table.
 - we use mfcheck and the checkuser patch (mysql) to drop questionable
 connections.

 I don't know what you mean by 'conf-split' or big-{ext-}.


 At 02:01 PM 2/27/2004, you wrote:
 On Fri, 2004-02-27 at 12:53, Jeff Koch wrote:
 Does anybody have any suggestion for improving the performance of a
 mailserver runnning
 qmail/vpopmail/qmailadmin/qmail-scanner/spamassassin?

 are you using spamd and having qmail-scanner call spamc?  or is it
 calling spamassassin.  if the latter, change it to spamd/spamc model,
 much more efficient.

 We're using a 2.4Ghz P4 with 1GB RAM and a 40GB SCSI drive. The
 operating system is a basic RH8.0 install with the ext3 journalling
 file system. We're handling about 50K messages/day and seem to
 exhausting the capabilities of the server. During peak periods CPU
 idle time reaches 0% and load averages will exceed 10.0

 with ext3 you have to be careful about catchall accounts, and
 accounts that nobody checks that get a lot of email.  If there are
 say, 25000 messages in an account and it gets 100 messages per hour,
 that will bring your server performance down quite a bit.

 of course, using reiserfs on the mail store will fix that problem.
 I've had over 150k emails in one account with reiserfs, and the
 system wasn't even being phased as I pumped more mail at it (was
 load testing for a customer)

 We would welcome any suggestions or URL's that you could point to -
 particularly with respect to performance tuning that will work with
 qmail.

 what is the approximate queue load, and what is your conf-split
 value?

 do you have big-{ext-,}todo?

 -Jeremy

 --
 Jeremy Kitchen
 Systems Administrator
 [EMAIL PROTECTED]
 Kitchen @ #qmail on EFNet - Join the party!
 .
 Inter7 Internet Technologies, Inc.
 www.inter7.com
 866.528.3530 toll free
 847.492.0470 int'l
 847.492.0632 fax
 GNUPG key ID: 93BDD6CE

 Best Regards,

 Jeff Koch



[vchkpw] Suggestions for improving performance

2004-02-27 Thread Jeff Koch
Does anybody have any suggestion for improving the performance of a 
mailserver runnning qmail/vpopmail/qmailadmin/qmail-scanner/spamassassin? 
We're using a 2.4Ghz P4 with 1GB RAM and a 40GB SCSI drive. The operating 
system is a basic RH8.0 install with the ext3 journalling file system. 
We're handling about 50K messages/day and seem to exhausting the 
capabilities of the server. During peak periods CPU idle time reaches 0% 
and load averages will exceed 10.0

We've seen some recommendations ( 
http://people.redhat.com/alikins/system_tuning.html , for example) for 
changing hard disk and file system parameters mostly having to do with 
read/write caching and cache sizes and minumum delays before writing to 
disk but we have been afraid to try any of these for fear of making things 
worse.

We would welcome any suggestions or URL's that you could point to - 
particularly with respect to performance tuning that will work with qmail.

Thanks



Best Regards,

Jeff Koch