On 25.6.2020 19:15, Wietse Venema wrote:
This came up when people asked for a way to prevent the degenerate
case that I described: non-bulk mail getting stuck behind bulk mail.
As long as the incoming queue is not entirely in FIFO order, then
the active queue will contain a mix of messages from different
senders, and the scheduler can interleave their deliveries.
I see. Thanks for being so patient with me. So it's not that the active
queue as such is being congested with single sender, it's just that the
mail from other senders sits behind those mails for too long. That's good.
What about the following solution: introducing per-sender windows,
similar to per-destination windows?
In little more details: nqmgr keeps the list of jobs in FIFO order.
(There is some preemption logic which allows some mail to move towards
the front of the list under some conditions, but fortunately we can
completely ignore that here). The selection of next entry for delivery
is simple in essence, simply traverse this list and return entry of the
first job which is not limited by per-destination delivery windows.
If we added another condition which might prevent the job from being
selected, in this case a per-sender limit, the traversal would simply
continue and next job on the list would be considered.
Implementation wise, I don't think it's very difficult to implement. For
performance reasons, the current job is cached to prevent the traversal
from the start every time and there is some bookkeeping related to
keeping track of per-destination limits and lifting them efficiently,
but in case of job sender it should be much simpler. Job can have many
destinations but only one sender, after all.
This solution would limit how many deliveries can single sender have in
progress, allowing other senders to slip by in their FIFO order, even if
they wouldn't preempt the bulk mail on the job list as such.
The limit could be dynamic in this case, too. We can come up with
whatever formula we want. If there is just single sender in the whole
system, he can have all delivery agents, if there are many senders, you
would obviously want to scale it down to just a portion of that.
If you are worried that there would be several bulk senders which would
hog all delivery agents anyway (their windows together would be more
than number of delivery agents), we could add one level of indirection,
and basically assign the senders to the classes you proposed, and keep
track of the limits per class rather than individual sender.
Patrik
