Re: postfix-queue taking a LONG time on large mails
> Did you specify FLAG_MASK_EXTERNAL as argument to queue/postfix-queue ? I believe this will only affect the further processing of the mail, e.g. use of virtual domain/alias tables. As it does not work properly without that flag anyway, yes, it was set. In the meanwhile I did some further investigation. It appears that quite a good part of the delay is happening internally in qpsmtpd. As I said, with my alternative queuing plugin a big mail now only took about 16 seconds. I then increased logging to see which plugin runs when and the queue plugin is only called after 14 of these 16 seconds. In the steps before queuing qpsmtpd does some copying/processing and it processes the mail line by line. I suspect this to be the reason of the delay. I'm not sure if this can be changed easily. I would guess that other MTAs do this processing already while receiving the mail (data-phase before data post), therefore not having to do the processing in full afterwards. Oh, and the queing plugin will again process the mail line by line, adding further delay. This is true for all queue plugins. This can surely not be changed easily without deep modifications inside qpsmtpd and the way we process a mail. With my alternative queing plugin the delay is now more or less acceptable but I'm still not satisfied. I think more research in finding the source of the long delay is warranted as even processing 10 MB line by line shall not be that slow on a modern machine (this is a quadcore Opteron). Regards, Michael -- It's an insane world, but i'm proud to be a part of it. -- Bill Hicks
RE: postfix-queue taking a LONG time on large mails
Hello, Did you specify FLAG_MASK_EXTERNAL as argument to queue/postfix-queue ? Does it change something ? I've had problems with the plugin until I added this parameter. Sydney -Message d'origine- De : Michael Holzt [mailto:[email protected]] Envoyé : jeudi 29 novembre 2012 01:09 À : [email protected] Objet : Re: postfix-queue taking a LONG time on large mails > Anyone know what the reason for this might be? Is this a general problem > of the postfix-queue plugin? The postfix-queue plugin uses a socket to the cleanup daemon. I'm not sure if the connection over the socket is slow (the cleanup daemon might be slow reading data from it) or if there is some additional processing which takes time. An alternative path would be to drop the mail into the postdrop queue and the notify the pickup-daemon. This would have the advantage that qpsmtpd would not need to wait for an answer. A disavantage would be that if something goes wrong after putting the mail in the queue (e.g. pickup-daemon not accepting it), it would not reflect on the SMTP return code. Another (minor) disavantage would be that doing it this way, Postfix would insert another Received-Line (but who cares). I have just written a very stupid queue plugin which uses Mail::Postfix::Postdrop to inject the mail. Interestingly enough it seems that the code from our Postfix-queue Plugin was copied from there (we e.g. have a comment which still talks about speaking to the pickup daemon). I tested my plugin and while it still took 16 seconds to queue the mail (10 MB attachemnet) , this is certainly much better than the more than two minutes of the current queue plugin. Therefore I suggest that we change either the current plugin (which for reasons unknown to me uses a library which was made directly part of qpsmtpd) or add a second one (the internal library would need to be slightly modified). Regards Michael
Re: postfix-queue taking a LONG time on large mails
> Anyone know what the reason for this might be? Is this a general problem > of the postfix-queue plugin? The postfix-queue plugin uses a socket to the cleanup daemon. I'm not sure if the connection over the socket is slow (the cleanup daemon might be slow reading data from it) or if there is some additional processing which takes time. An alternative path would be to drop the mail into the postdrop queue and the notify the pickup-daemon. This would have the advantage that qpsmtpd would not need to wait for an answer. A disavantage would be that if something goes wrong after putting the mail in the queue (e.g. pickup-daemon not accepting it), it would not reflect on the SMTP return code. Another (minor) disavantage would be that doing it this way, Postfix would insert another Received-Line (but who cares). I have just written a very stupid queue plugin which uses Mail::Postfix::Postdrop to inject the mail. Interestingly enough it seems that the code from our Postfix-queue Plugin was copied from there (we e.g. have a comment which still talks about speaking to the pickup daemon). I tested my plugin and while it still took 16 seconds to queue the mail (10 MB attachemnet) , this is certainly much better than the more than two minutes of the current queue plugin. Therefore I suggest that we change either the current plugin (which for reasons unknown to me uses a library which was made directly part of qpsmtpd) or add a second one (the internal library would need to be slightly modified). Regards Michael
