Patrik Rak:
[ Charset ISO-8859-2 converted... ]
> On 25.6.2020 19:15, Wietse Venema wrote:
>
> > This came up when people asked for a way to prevent the degenerate
> > case that I described: non-bulk mail getting stuck behind bulk mail.
> >
> > As long as the incoming queue is not entirely in FIFO order, then
> > the active queue will contain a mix of messages from different
> > senders, and the scheduler can interleave their deliveries.
>
> I see. Thanks for being so patient with me. So it's not that the active
> queue as such is being congested with single sender, it's just that the
> mail from other senders sits behind those mails for too long. That's good.
>
> What about the following solution: introducing per-sender windows,
> similar to per-destination windows?
>
> In little more details: nqmgr keeps the list of jobs in FIFO order.
> (There is some preemption logic which allows some mail to move towards
> the front of the list under some conditions, but fortunately we can
> completely ignore that here). The selection of next entry for delivery
> is simple in essence, simply traverse this list and return entry of the
> first job which is not limited by per-destination delivery windows.
> If we added another condition which might prevent the job from being
> selected, in this case a per-sender limit, the traversal would simply
> continue and next job on the list would be considered.
I believe that is worse than 'classes' or sender round robin:
You artifcially bound the amount of delivery resources for a sender,
even in the absence of any other email mail in the queue that
competes for those resources.
The result is that Postfix holds mail in the queue longer than
necessary, resulting in unnecessary active queue congestion and
slowing down all other deliveries that might arrive.
Classes or sender round-robin will give the sender all resources
in the absence of other mail, so Postfix clears the queue as quickly
as possible while still respection destination concurrency limits.
It's a liveness/utilization isssue. No delivery agent should be
idle when it could deliver mail that satisfies concurrency limits.
Wietse
> Implementation wise, I don't think it's very difficult to implement. For
> performance reasons, the current job is cached to prevent the traversal
> from the start every time and there is some bookkeeping related to
> keeping track of per-destination limits and lifting them efficiently,
> but in case of job sender it should be much simpler. Job can have many
> destinations but only one sender, after all.
>
> This solution would limit how many deliveries can single sender have in
> progress, allowing other senders to slip by in their FIFO order, even if
> they wouldn't preempt the bulk mail on the job list as such.
>
> The limit could be dynamic in this case, too. We can come up with
> whatever formula we want. If there is just single sender in the whole
> system, he can have all delivery agents, if there are many senders, you
> would obviously want to scale it down to just a portion of that.
>
> If you are worried that there would be several bulk senders which would
> hog all delivery agents anyway (their windows together would be more
> than number of delivery agents), we could add one level of indirection,
> and basically assign the senders to the classes you proposed, and keep
> track of the limits per class rather than individual sender.
>
> Patrik
>
