Patrik Rak: > > > On 24 Jun 2020, at 01:29, Wietse Venema <wie...@porcupine.org> wrote: > > > > ?Problem: nqmgr's fairness breaks down for single-recipient bulk mail > > You describe the solutions but maybe you could describe the problem first?
My text was a bit disorganized, but the gist is that nqmgr uses a clever approach to interleave (bulk mail) multi-recipient deliveries with (non-bulk) single-recipient deliveries. This allows the non-bulk mail to trickle out *during* bulk delivery, instead of being stuck *after* the bulk delivery. I am making a simplification as nqmgr will also interleave workloads with other sizes, not just single-recipient and multi-recipient. When the bulk mail is in the form of single-recipient messages, then nqmgr cannot distinguish between bulk mail and non-bulk mail, and these deliveries are no longer interleaved. Deliveries to a specific destination become sequential. Again, I am making a simplification, this time that all email is single-recipient. The idea then is to have three FIFO classes with delivery requests that are round-robin interleaved by design: one class for senders with, say, 1-20 pending delivery requests, one class for senders with 20-400, and one class for senders with more than 400 pending delivery requests. The scheduler has up to 20,000 delivery requests in memory. For a given transport and destination, if all three classes have work then round-robin interleaving gives each class gets 1/3 of the delivery slots, 1/2 if there are only two classes with work, all slots if there is only one class witth work, and zero if there is no work. If implemented in oqmgr, it's brain-dead round-robin with FIFO. What could possibly go wrong? :-) Well it does not interleave deliveries within the same class. But, with single-recipient messages, that wasn't happening anyway, so we are not making things worse. > > Alternative 2: multiple mostly independent scheduling classes > > ... > > Other benefits: we can choose the class based on other properties, > > for example message size, or both size and number of pending delivery > > requests per sender, or something else. It seems unlikely that we > > will need more than three classes: small, medium, large. > > Ah, the classes. I remember we have talked about something like > this in the past, many years ago. I have spent several weeks of > that summer figuring out the proper way of doing this, but when > it came to implementation, I have decided I won?t push this forward > and put the whole thing on hold. > > I don?t remember any of it now but maybe I wrote the most important > ideas somewhere, so I can have a look. I remember we discussed classes, but I don't think that I was able to reduce the concept to a combination of really brain-dead simple concepts. I think it would work (in oqmgr), I just need to find time to do this. Wietse