On 8/23/2012 7:29 PM, Wietse Venema wrote: > Mike Mitchell: >> The first issue I know is being reported is seemingly a throttling >> of submissions to the server--we can send about 50, then we're >> made to wait almost exactly 6 seconds, then we can send about 50 >> more. > > This is a built-in safety feature. > > If a client were allowed to flood Postfix with messages, then Postfix > would deliver ZERO MAIL until the client stops sending. Many people > would find that unacceptable.
> To avoid this, the Postfix SMTP server inserts delays when the mail > inflow exceeds the mail outflow. This is not a stiff coupling, it > allows for slippage, as common sense would require. Thus to achieve the high msg transfer rates you mention you must use parallelism. One method is to bind multiple IP addresses to the NIC of the submitting host and make multiple SMTP connections in a round robin fashion, along with controlling submission rate in your application. For instance, if you bind and connect from 20 IP addresses and send 10 msgs/sec down each connection, that's 12,000 msgs/minute. Ten msgs/sec is a low enough rate that smtpd should not apply the brakes on any of the connections. You could probably do 20/sec and cut the total time in half. You'll need to experiment. Now, this is strictly the submission rate into Postfix. If it can't deliver the messages at the same or similar rate you'll still have some issues to work out on the delivery side. Assuming Postfix can do 20/sec without putting on the brakes, that's 400 submissions/sec aggregate. That's 400 msgs written into the queue and 400 read from the queue during delivery, for a bare minimum 800 IOPS at the storage. To achieve this with sufficient headroom, you'll need 8x 15K SAS drives in RAID10. And you'll need a quality RAID controller with 512KB to 1MB of BBWC, such as an LSI. Or use the same controller with two enterprise class SAS SSDs in RAID1. I advise against using software RAID for this workload. You'll simply find yourself spending countless hours trying to optimize it for this workload, and not achieving success, for the savings of a $700 RAID card and battery. Get the card and spend your time tuning the submission application. And DO NOT USE PARITY RAID for a high volume Postfix queue. Use only RAID1/10. Parity RAID = mega FAIL. Now you need a queue filesystem capable of the parallelism in this workload. If your server is Linux then you should use XFS, as it will yield far superior performance to EXT3/4 and JFS. If your server runs *BSD, Solaris, AIX, or other, you'll need consult others. -- Stan
