Paul Guglielmino <[EMAIL PROTECTED]> wrote:
>
> Thursday morning we did a test mailing of 100,000 people. It's still
> going alomst 24 hours later. According to other people's results, this
> is terrible.
I'm not an expert on the internals of qmail, but I see one problem...
As a first test we didn't change our scripts at all and
> used the sendmail wrapper program. The scripts finished quickly but we
> still have a ton of messages sitting in the queue.
>
> My first step was to up the number in concurrencyremote to 100 and
> restart qmail. However I still never got more than 16 or 17
> qmail-remote's running at one time. I changed conf-spawn to 250 and
> concurrencyremote to 240 but still the number of qmail-remote's never
> got above 40. I then changed the concurrencylocal to 120 and the
> timeoutremote to 60 but still no help.
>
> My last step was to change the conf-split from 23 to 200. After that I
> started getting syslog errors like:
>
> Jul 14 09:48:31 charon qmail: 963582511.269118 warning: unable to stat
> mess/14/476827
When you change conf-split, you can't expect the new binaries to be able to
use the queue from the old binaries. It changes the assumptions that qmail
uses about which subdirectories particular files in the queue will be in.
The general method to change conf-split on a running machine is something
like the following:
-change conf-split
-compile qmail and install to a DIFFERENT location than the original.
Let it run in parallel with the old version (different queue, different
control directory, etc).
-wait until the queue of the original qmail instance is empty.
-compile qmail again with the larger conf-split, installing to the directory
of the ORIGINAL qmail installation.
-let the queue of the 2nd/alternate installation empty
-delete the 2nd/alternate qmail instance.
> And nothing was going faster so I changed it back. I also have
> sigalrm-ed qmail-send hoping to get the queue moving faster but that
> didn't seem to help.
>
> Here's part of the output from qmail-showctl:
>
> qmail home directory: /var/qmail.
> user-ext delimiter: -.
> paternalism (in decimal): 2.
> silent concurrency limit: 125.
This can be changed in the source, I believe -- but it can depend on your
OS.
> subdirectory split: 23.
> user ids: 502, 503, 504, 0, 505, 506, 507, 508.
> group ids: 501, 502.
> concurrencylocal: Local concurrency is 120.
> concurrencyremote: Remote concurrency is 123.
> databytes: (Default.) SMTP DATA limit is 0 bytes.
> qmqpservers: (Default.) No QMQP servers.
> queuelifetime: (Default.) Message lifetime in the queue is 604800
> seconds.
> smtproutes: (Default.) No artificial SMTP routes.
> timeoutconnect: (Default.) SMTP client connection timeout is 60 seconds.
> timeoutremote: SMTP client data timeout is 60 seconds.
> timeoutsmtpd: (Default.) SMTP server data timeout is 1200 seconds.
>
> So I guess my question is "What's going on???" :-) It seems that other
> people have had this problem but the answer has always been "restart
> qmail" which I did... We still have some messages in the queue to be
> preprocessed if that matters.
What is your hardware? What does `mailq` show (it'll be big)? Are most of
the messages remaining in the queue just bounces and deferrals?
If it's not just bounces and deferrals, you're hitting a bottleneck somewhere.
Unfortunately, you've given us no information to help determine what that
bottleneck is (network bandwidth, disk bandwidth, linear search time,
CPU, logging).
What are you using for logging, anyway? If its syslog, that could be your
problem right there.
Charles
--
-----------------------------------------------------------------------
Charles Cazabon <[EMAIL PROTECTED]>
GPL'ed software available at: http://www.qcc.sk.ca/~charlesc/software/
Any opinions expressed are just that -- my opinions.
-----------------------------------------------------------------------
