Wietse: I know who you are -- big fan of yours ever since tcpwrappers :-)
The only reason I wrote in is that this queueing of mail issue appeared fairly recently, and other times when I have had to collect large volumes of mail from this client my system has been fine. Below I have pasted some logs that show long delivery times (sitting in the queue for a day) but the last delay (delivery delay?) is in the upper-teens. Just trying to reconcile why this is: if it's something on my end that is fine and I can work it out, but maybe it is an unforeseen issue with Postfix and large volumes of piped mail. The system itself is a Nehalem Xserve, so it seems to have plenty of horsepower to handle the load. (anonymized a bit in hostname and email address) Jan 14 15:45:43 server001 postfix/local[24570]: 26E371265ED: to=<receiver_em...@receiver_ip>, relay=local, delay=81352, delays=0.59/81343/0/7.7, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:43 server001 postfix/local[22292]: AFAF31265EA: to=<receiver_em...@receiver_ip>, relay=local, delay=81353, delays=0.5/81344/0/8.2, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:45 server001 postfix/local[23365]: B811A1265E3: to=<receiver_em...@receiver_ip>, relay=local, delay=81355, delays=0.34/81341/0/13, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:45 server001 postfix/local[23852]: E5F061265EC: to=<receiver_em...@receiver_ip>, relay=local, delay=81354, delays=0.41/81344/0/10, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:46 server001 postfix/local[23580]: F15EA1265D5: to=<receiver_em...@receiver_ip>, relay=local, delay=81358, delays=0/81339/0/19, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:46 server001 postfix/local[22196]: 3121C1265E5: to=<receiver_em...@receiver_ip>, relay=local, delay=81356, delays=0.42/81342/0/14, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:46 server001 postfix/local[25810]: E55181265EB: to=<receiver_em...@receiver_ip>, relay=local, delay=81355, delays=0.19/81344/0/11, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:46 server001 postfix/local[25651]: 8918B1265E2: to=<receiver_em...@receiver_ip>, relay=local, delay=81356, delays=0/81341/0/15, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:46 server001 postfix/local[24774]: B6F851265D2: to=<receiver_em...@receiver_ip>, relay=local, delay=81358, delays=0/81337/0/21, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:47 server001 postfix/local[25378]: 26EDF1265EE: to=<receiver_em...@receiver_ip>, relay=local, delay=81355, delays=0.57/81343/0/11, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:48 server001 postfix/local[23204]: DC9B11265E4: to=<receiver_em...@receiver_ip>, relay=local, delay=81358, delays=0.32/81342/0/16, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:49 server001 postfix/local[24247]: 80DB71265E0: to=<receiver_em...@receiver_ip>, relay=local, delay=81359, delays=0.21/81341/0/18, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:49 server001 postfix/local[25905]: 840B11265FC: to=<receiver_em...@receiver_ip>, relay=local, delay=81355, delays=0.35/81348/0/7, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:49 server001 postfix/local[24394]: 302EA1265DD: to=<receiver_em...@receiver_ip>, relay=local, delay=81360, delays=0.12/81339/0/21, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Jan 14 15:45:50 server001 postfix/local[23554]: 4B37E1265DF: to=<receiver_em...@receiver_ip>, relay=local, delay=81360, delays=0.09/81340/0/20, dsn=2.0.0, status=sent (delivered to command: /usr/bin/db_capture) Warmest Regards, Wendy On Thu, Jan 14, 2010 at 12:34 PM, Wietse Venema <wie...@porcupine.org> wrote: > Wendigo Thompson: >> Victor: >> >> I wanted to use Postfix as a delivery mechanism so that I would be >> able to queue and hold mail during database outages and other >> maintenance -- a job that it does very well. Is there really no >> advice available on how to increase the speed of Postfix's local mail >> delivery? > > I'm the guy who wrote Postfix. > > With your numbers, 300k messsages/day and 3ms/delivery, the whole > day's volume should be done in about 900s. Since you see deliveries > draining into the weekend, your numbers don't seem to add up. > > In the Postfix logs you can see how long it took to deliver mail. > Here is an example of a pipe alias that handles all my own email: > > Jan 14 00:08:51 spike postfix/local[98331]: 9F53A1F3EA5: > to=<xxx>, relay=local, delay=0.41, > delays=0.37/0.02/0/0.03, dsn=2.0.0, status=sent (delivered > to command: yyy) > > Here, it took 0.03 seconds to deliver a small message for "xxx" to > a small program "yyy". In addition it took 0.37s+0.02=0.39s to > receive the message and schedule it for delivery. This message > arrived via a slow connection. More information about this is at > http://www.postfix.org/postconf.5.html#delay_logging_resolution_limit > > Now, If you want to deliver hundreds of thousands each day to a > pipe alias, then executing a whole program each time is expensive, > but it is possible if your machine has enough horse power. > > So, we have two options: > > a) The "red" option: to waste more resources, increase concurrency, > by increasing local_destination_concurrency_limit (default: 2). > > b) The "green" option: instead of wasting CPU cycles with a local(8) > pipe alias that executes and terminates one program per delivery, > use transport_maps to direct mail to a persistent SMTP or LMTP > process that keeps the database connection open. This is the > option that Victor suggested. > > Wietse >
