On Mon, Apr 15, 2013 at 01:20:58PM +0200, Giorgio Luchi wrote:
> How can we implement round-robin by sender ip/authenticated user
> and to preserve the memory constraint too?
>
> - "sender" is the sender's ip address or the authenticated user
> name (i.e. "80.93.143.50" or "giorgio.luchi")
> - "rrsender_message_limit" is the max number of messages in the
> sender queue (i.e. "3")
> - "rrsender_queue_limit" is the max number of senders actually
> loaded in RAM (i.e. "6000")
>
> [...]
>
> I hope this can help in understanding and in finding a solution
> to what we need.
Your specific proposal is not viable. The scheduler (queue manager
processing the active queue) works with a bounded subset of messages
and message recipients. Think of these as people who are in the
airport terminal whose ticket class determines which line they may
join, with some lines getting better service than others.
The capacity of the capacity of the roads leading to the terminal
is finite. A first-class ticket holder cannot take advantage of
preferrential treatment at the terminal if he is stuck behind
thousands of economy ticket holders on a congested highway.
The highway is analogous to the Postfix incoming and deferred queues
and finite capacity of smtpd(8) processes to take in new mail.
You must abandon all hope of trying to make the highway more "fair"
than FIFO, this is not possible.
What is possible is tweaking the algorithm at the terminal. It is
in principle possible to adjust the scheduler algorithm for selecting
the next message to send (out of those already in the active queue).
Right now the scheduler is able to interleave delivery of later
arriving small messages with progressive delivery of larger (mailing
list) messages, so that a single newsletter delivery does not
substantially delay the delivery of all subsequent mail.
If the same newsletter were to be injected into the queue as as a
burst of individual messages the scheduler would not be able to
apply Patrick Raq's nqmgr FIFO preemption logic.
One could conceivable enhance the algorithm to support a notion
of job groups, where the pre-emption logic works at two different
levels.
- A job group can be preempted by a later job group after the
first job group consumes enough delivery slots.
- A job within a group can be preempted by a later job within that
group after the first job consumes enough delivery slots.
It would then be up to the input stage to tag queue files with job
group identifiers. This could be done a policy service that returns
a new access(5) action to set the job group id.
The effect is to logically "re-assemble" a multi-message bulk
mailing from a single source (be it by IP, SASL username, sender
address, ...) into a single logical message which is subject to
the scheduler preemption algorithm.
This is perhaps a fruitful direction, though an ISP would likely
get more "bang for the buck" by rate limiting input with a policy
service. The outlined design would be of greater utility in large
corporate and some hosted email marketing scenarios.
If anyone were to design an build such a thing on contract, it
would probably be Patrick Raq (if he's available). Otherwise,
nobody else comes to mind, you'd have to hire someone who's capable
of understanding Patrick's design in detail, and extending it
correctly.
The result would be unlikely to be incorporated into the mainstream
Postfix release unless it were exceedingly well documented and
implemented with care. Most likely you'd be stuck maintaining it
as a private patch indefinitely.
So the best approach is perhaps to find some other MTA that does
what you want if one exists. Otherwise, try to leverage the existing
feature set to approximate what you want, rate limiting input is
my best suggestion.
--
Viktor.
