On Mon, May 13, 2013 at 05:18:05PM -0400, Wietse Venema wrote:
> The qmgr(8) concurrency scheduler limits the concurrency per nexthop.
> That does not change when prescreen is inserted between qmgr(8) and
> smtp(8) processes.
>
> For each nexthop:
>
> number of qmgr-prescreen connections + number of qmgr-smtp connections
> <= concurrency limit
Patrick does raise a valid new concern about the prescreen design.
Suppose that all ~100 smtp(8) delivery agents are busy, and that
prescreen is willing to accept ~500 simultaneous qmgr(8) delivery
requests in the expectation that for many of these DNS lookups and
or initial connections will incur high latency.
Suppose that instead all 500 DNS lookups and initial connections
complete quickly, giving us 500 parallel connections to some set
of remote servers. In the mean time the 100 currently busy SMTP
deliveries are taking their time.
We now have a problem, since we we've lots of connections we can't
immediately start using. And by the time we've capacity to use
them the remote servers may well drop the idle connections.
So while doing pre-emptive DNS lookups is quite safe, doing
pre-emptive connections is more risky. A similar issue exists
in principle with postscreen, in that more connections might be
all accepted for "pass OLD" than backend smtpd(8) processes to
serve them, leaving clients "stranded" for some time.
The impedance mistmatch is less severe with postscreen since so
much mail is spam, and because clients are generally more willing
to wait out delays than servers.
So the prescreen design may not pan out. And my contention is that
in any case it is a bit pricey it terms of implementation cost to
benefit.
If we limit prescreen to initial DNS lookups, the cost to implement
gets much lower, and much of the initial latency is avoided for
dead sites with broken DNS, so that could be of some use, and we
don't tie up remote resources by prefetching DNS results. So a
DNS-only pre-screen could be added, still not sure it is worth it.
We'd need lots of data on how much of the latency of dead destinations
is DNS latency vs. connection timeout latency.
--
Viktor.
