Re: Prioritising outgoing mail
On Mon, Mar 02, 2009 at 10:44:21PM +0800, Wouter van Marle wrote: Hi list, From me a question that seems to be asked now and then here, but I could not find any answers even on whether this is possible in the first place. I would like to be able to prioritise outgoing e-mail so they do not get stuck in the queue. This as I now and then send out a large number of e-mails with attachments, and that saturates my connection for a prolonged time. It doesn't matter that those mails get out slower, as long as they get out eventually I'm happy. Use a custom transport for these messages with a low concurrency limit, or use traffic shaping in the TCP stack to limit the bandwidth per SMTP connection. -- Viktor. Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. To unsubscribe from the postfix-users list, visit http://www.postfix.org/lists.html or click the link below: mailto:majord...@postfix.org?body=unsubscribe%20postfix-users If my response solves your problem, the best way to thank me is to not send an it worked, thanks follow-up. If you must respond, please put It worked, thanks in the Subject so I can delete these quickly.
Re: Prioritising outgoing mail
On 2 Mar 09, at 23:09, Victor Duchovni wrote: On Mon, Mar 02, 2009 at 10:44:21PM +0800, Wouter van Marle wrote: Hi list, From me a question that seems to be asked now and then here, but I could not find any answers even on whether this is possible in the first place. I would like to be able to prioritise outgoing e-mail so they do not get stuck in the queue. This as I now and then send out a large number of e-mails with attachments, and that saturates my connection for a prolonged time. It doesn't matter that those mails get out slower, as long as they get out eventually I'm happy. Use a custom transport for these messages with a low concurrency limit, You mean like installing sendmail or so in parallel to postfix and then have sendmail send out the lower-priority mails? or use traffic shaping in the TCP stack to limit the bandwidth per SMTP connection. And how would that get certain mails out with priority? It sounds to me like this would slow down the overall process. I have up to 100 smtp processes running at a time, but as long as new mails end up at the back of the queue still no progress there. They have to come first. Wouter. -- Viktor. Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. To unsubscribe from the postfix-users list, visit http://www.postfix.org/lists.html or click the link below: mailto:majord...@postfix.org?body=unsubscribe%20postfix-users If my response solves your problem, the best way to thank me is to not send an it worked, thanks follow-up. If you must respond, please put It worked, thanks in the Subject so I can delete these quickly.
Re: Prioritising outgoing mail
Wouter van Marle: On 2 Mar 09, at 23:09, Victor Duchovni wrote: On Mon, Mar 02, 2009 at 10:44:21PM +0800, Wouter van Marle wrote: Hi list, From me a question that seems to be asked now and then here, but I could not find any answers even on whether this is possible in the first place. I would like to be able to prioritise outgoing e-mail so they do not get stuck in the queue. This as I now and then send out a large number of e-mails with attachments, and that saturates my connection for a prolonged time. It doesn't matter that those mails get out slower, as long as they get out eventually I'm happy. Use a custom transport for these messages with a low concurrency limit, You mean like installing sendmail or so in parallel to postfix and then have sendmail send out the lower-priority mails? No, use a POSTFIX transport map. or use traffic shaping in the TCP stack to limit the bandwidth per SMTP connection. And how would that get certain mails out with priority? It sounds to me With the concurrency limit (see above), low priority mail can use up only a limited portion of the bandwidth. Wietse
Re: Prioritising outgoing mail
Hi all, Would it be possible to add an extension to the user's address, e.g. user+s...@example.com, that would be mapped through a separate transport (e.g. the slow: as suggested in the man page), and be rewritten by trivial-rewrite to u...@example.com before being sent out. An option like this would do the job for me. It would allow me to easily maintain my maillist, rewriting addresses on the fly when creating the mails, and allowing regular mails to be handled with priority. And any ideas on how/where such a slow: transport (with a limited number of smtp daemons to be started) would be configured? I can't find anything about this in the man pages. Except that it is possible. Wouter. On Tue, 2009-03-03 at 11:25 +0800, Wouter van Marle wrote: On Mon, 2009-03-02 at 11:18 -0500, Victor Duchovni wrote: On Mon, Mar 02, 2009 at 11:59:31PM +0800, Wouter van Marle wrote: Use a custom transport for these messages with a low concurrency limit, You mean like installing sendmail or so in parallel to postfix and then have sendmail send out the lower-priority mails? No I mean a Postfix transport, as in transport(5) and master(5). The problem of a transport map (I have just read the man page, which as usual is highly technical so I am not sure whether I fully understand the purpose and working of transport maps) is that there is a huge overlap between receivers of the low-priority mail list and regular e-mail receivers. Most of the regular e-mail receivers also receive this mail list. Many of my mail list receivers are on common domains like gmail.com and yahoo.com, thus this would slow down all other mails to those domains as well, even if they are not part of the mail list. Setting it per recipient is not a good idea because of maintenance issues (keeping mail list and transport map in sync), and because of the regular mails that may be sent to those recipients while a mail list run is in progress. The idea of using a slow: transport as suggested in the transport(5) man page (without elaborating unfortunately...) to limit the number of smtp deamons that sounds like the way to go to me. Then I can reserve 80 deamons for the mail list, or maybe 50, enough to saturate my uplink - still allowing regular mails to have an smtp available to be handled immediately. This appears to me a way to let the other mails jump the queue and be processed with priority. That there is little bandwidth available is not too much of an issue, then it might take a minute instead of seconds to send out, still a major improvement of the hours it may take in the current situation. or use traffic shaping in the TCP stack to limit the bandwidth per SMTP connection. And how would that get certain mails out with priority? It sounds to me like this would slow down the overall process. I have up to 100 smtp processes running at a time, but as long as new mails end up at the back of the queue still no progress there. They have to come first. It would not, but you won't saturate the entire link with any given email, leaving enough room for other traffic. If you can limit the concurrency of this particular message, then you'll have some bandwidth left over for other messages. I don't like that idea very much: when I have only a few mails to send out, I want them to go with the maximum speed possible. I have 2 Mbit available, so with 100 smtp connections could limit it to say 20 kbit per smtp process. But that would leave the rest of my bandwidth idle when there are less than 100 active smtp connections, which is the case like 90% of the time. Wouter.
Re: Prioritising outgoing mail
On Tue, Mar 03, 2009 at 11:25:55AM +0800, Wouter van Marle wrote: On Mon, 2009-03-02 at 11:18 -0500, Victor Duchovni wrote: On Mon, Mar 02, 2009 at 11:59:31PM +0800, Wouter van Marle wrote: Use a custom transport for these messages with a low concurrency limit, You mean like installing sendmail or so in parallel to postfix and then have sendmail send out the lower-priority mails? No I mean a Postfix transport, as in transport(5) and master(5). The problem of a transport map (I have just read the man page, which as usual is highly technical so I am not sure whether I fully understand the purpose and working of transport maps) is that there is a huge overlap between receivers of the low-priority mail list and regular e-mail receivers. Most of the regular e-mail receivers also receive this mail list. You may need to do sender-dependent routing for this sender, and inject the mail into a different queue, or get the originating system to do this directly. It would not, but you won't saturate the entire link with any given email, leaving enough room for other traffic. If you can limit the concurrency of this particular message, then you'll have some bandwidth left over for other messages. I don't like that idea very much: when I have only a few mails to send out, I want them to go with the maximum speed possible. I have 2 Mbit available, so with 100 smtp connections could limit it to say 20 kbit per smtp process. But that would leave the rest of my bandwidth idle when there are less than 100 active smtp connections, which is the case like 90% of the time. Does limiting bandwidth for small messages signicantly impact delivery latency? Also who said you should divide the bandwidth 100-fold? You give the slow transport 5 parallel threads, and up to half the bandwidth, so each channel gets 10% of the bandwidth. -- Viktor. Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. To unsubscribe from the postfix-users list, visit http://www.postfix.org/lists.html or click the link below: mailto:majord...@postfix.org?body=unsubscribe%20postfix-users If my response solves your problem, the best way to thank me is to not send an it worked, thanks follow-up. If you must respond, please put It worked, thanks in the Subject so I can delete these quickly.
