On Tue, Sep 20, 2011 at 12:08:29PM -0400, Derek Knapp wrote: > Is there any kind of option for the minimum number of queue runners? > > I have a shell script that adds emails to the queue (using > /usr/sbin/exim -bS -odq) but every time I check the server there are > thousands of emails in the queue waiting to be sent. > > I have seen many posts online about using queue_only option, but non > really say how they handle starting up the queue runners.. if I > change my shell script to remove the -odg, the emails are sent out > much faster than 1st queuing them, but I figured adding them to the > exim queue 1st would allow exim to send them in batches (for example > if there are 100 emails going to hotmail, it could send them all at > once) > > Is there a proper way to handle the starting of queue runners? > should I just create an init script that runs exim -q1m 100 times > every time the server starts?
When an Exim queue runner process starts, it begins working through the queue linearly, either in random order or based on message received order, depending on if you've set the "queue_run_in_order" option. The problem is that if a queue runner hits a message to be delivered remotely, and the remote server isn't responding quickly, it may take up to 5 minutes for it to time-out, defer the delivery, and then move on to the next message. If you have a lot of messages on the queue, and a lot of remote hosts that are slow to respond, it could literally take hours or even days for a single queue runner to move through the queue. To offset this, you set "queue_run_max" to the maximum number of simultaneous queue runners you want to ever have running at once. This value doesn't start up queue runners, or guarantee that many will be running. It's just a limit on the maximum allowed. Exim locks messages during delivery so that two queue runners dont process the same message at once. If you have powerful enough hardware, you can set queue_run_max to be fairly high. Queue runners are started automatically, one at a time, every so many minutes as specified in the "-qXXm" command line option. If you have it set to something like "-q15m", a new queue runner will only start once every 15 minutes. If you are queueing all your email, this may be way too high a value. If your hardware is powerful enough, and you want the queue runners to ramp up quickly, you could set it to something even as low as "-q30s" or "-q20s". Be careful though! It all depends on how quickly you want queue runners to ramp up to the maximum and what your hardware is capable of. Couple of other things, in no particular order: 1. If you restart Exim with a sighup restart signal, the existing queue runners keep running, but Exim resets it's internal queue count to zero. That means you'll have twice the number of maximum queue runners running at some point. That can quickly kill a server if you send a restart signal several times in a row. I've gotten in the habit of killing Exim completely to avoid that. 2. If you frequently have a large queue due to slow-responding hosts, I would recommend creating a separate instance of Exim, perhaps on a different virtual IP address or port number, and have a router in your main configuration that forwards messages over to this "fallback" instance of Exim if a message can't be delivered quickly. This keeps your main queue lean so that properly-responding mail hosts get deliveries fast and slower-responding mail hosts get put into a cess-pool, so to speak. If you need example routers to do this, let me know. 3. My personal preference is to not automatically queue everything, but to set the timeout values on my main remote transport very low. I have the smtp connection timeouts and command timeouts set to 40 seconds (40 works well because many hosts delay up to 30 seconds before responding). If a remote server responds quickly, the message gets delivered immediately. Otherwise, it goes to the fallback instance of Exim. This makes for very fast deliveries to 95% of the mail servers, and then a separate queue for everyone else. Why make 95% of deliveries suffer because of the 5% of ill-behaving hosts? Hope this helps some. -- Dean Brooks [email protected] -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
