Maybe I've abused my root privileges today, but it's only for the good of the humanity and any failed experiments can be easily reversed.
Anyway. I couldn't see any reason that Mailman should hand Exim a separate copy for every 100 recipients - in fact, why split the recipient list at all when Exim can theoretically handle an unlimited number? - so I switched off that chunking. The advantages are that a little bit of space and internal memory bandwidth are saved when messages aren't duplicated. The mail queue also becomes less cluttered and queue runs go faster since Exim processes one message at a time during a queue run, but does many parallel deliveries if a message has multiple recipients. The drawback is that if delivery for one recipient is very slow, it blocks retrying of all the other recipients. And then I noticed and realised one thing. When Exim got a single message copy to deliver to all the 1477 active, regular subscribers of exim-users, it took a very long time, 4-7 minutes, before the first remote connection was opened. The reason is, of course, that Exim routes all addresses first, one address at a time, and over 1000 serial DNS lookups mean a lot of waiting. So, to get to the point, would it be at all possible to parallelise the DNS lookups (and is it worth it)? Remote routers usually don't generate new addresses. The exception is when the "self" option kicks in. A router can also pass in that case or decline after DNS lookup if ignore_target_hosts removes all the hosts found. Idea: Introduce a new yield and/or a new address_item list, addr_resolving. Add a function host_find_bydns_async, which will initiate DNS queries using an asynchronous resolver library[1] and take care of the results provided via the callback interface (since there are multiple levels of abstraction, I think multiple levels of callbacks are needed). Routers can call this function and return, putting the address on the addr_resolving list. When there are no more addresses to route (addr_new == NULL), deliver_message() will run the select() loop needed to process the pending DNS queries. Either the callback associated with host_find_bydns_async or a callback in each router will put the results somewhere in each addr_item structure. When there are no more pending DNS queries, addr_resolving can be moved to addr_new, and routing can continue. The process repeats until all addresses have been routed. Sounds easy, right? :-) The other option I could think of involves threading... [1] After a quick survey I suggest libares from ftp://athena-dist.mit.edu/pub/ATHENA/ares, because a) it's API resembles the standard resolver (res_*) library's, and b) it's in Debian :-). -- Magnus Holmgren [EMAIL PROTECTED] (No Cc of list mail needed, thanks)
pgpF9xXOtaIVK.pgp
Description: PGP signature
-- ## List details at http://www.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
