Are there any plans in the future to make it multi-threaded at all? I presume in a way postfix piping the mail to dbmail-smtp would actually be quicker (though less preferred).
I did consider load balancing lmtp just for the sake of clearing down the queue as at times it almost seemed like the amount of mail coming in out weighed the speed at which it was inserting particularly with a large backlog that we had. James Greig -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Paul J Stevens Sent: 15 June 2014 15:06 To: DBMail mailinglist Subject: Re: [Dbmail] lmtpd performance dbmail-lmtpd being single threaded, 1 or 2 per second per process is quite normal. short of making lmtpd multi-threaded: If you need more throughput, you could spin up more lmtpd process on different ports and have postfix load-balance over them using something like haproxy. Never done that myself, but it should work. On 13-06-14 16:30, James Greig wrote: > Hi, > > > > Today we had a few issues which caused a back log of mail (around > 16000 emails to queue up on postfix servers). When the issue seemed > to be finally sorted lmtpd struggled to process the queue inserting > around 1 or 2 emails per second. Am I being impatient and this is > normal? I seem to recall dbmail 2.2 being able to churn through > emails and insert them into mysql a lot quicker. > > > > Just to confirm dbmail pop3d and imapd are all running (and busy) > updating, reading etc suitably quickly. We run the system on ZFS with > SSD write and SSD read caches and mysql is tuned. There are no slow > queries and show processlist never has anything in it. It seems like > lmtpd is just handling only 1 at a time i.e. not multi threaded? > > > > Jun 13 15:28:44 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [19] mailbox [82] message > [15047635] size [25880] is inserted > > Jun 13 15:28:45 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [4651] mailbox [20107] > message [15047637] size [2751] is inserted > > Jun 13 15:28:46 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [4870] mailbox [21135] > message [15047641] size [22597] is inserted > > Jun 13 15:28:47 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [2353] mailbox [10460] > message [15047643] size [90509] is inserted > > Jun 13 15:28:47 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [2289] mailbox [10227] > message [15047645] size [10100] is inserted > > Jun 13 15:28:48 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [2847] mailbox [12396] > message [15047647] size [23445] is inserted > > Jun 13 15:28:49 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [3194] mailbox [13819] > message [15047649] size [81818] is inserted > > Jun 13 15:28:49 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [4872] mailbox [21140] > message [15047651] size [61540] is inserted > > Jun 13 15:28:50 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [824] mailbox [3504] message > [15047653] size [14261] is inserted > > Jun 13 15:28:54 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [4559] mailbox [19593] > message [15047655] size [31050] is inserted > > Jun 13 15:28:54 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [3520] mailbox [15205] > message [15047657] size [30910] is inserted > > Jun 13 15:28:55 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [6505] mailbox [28584] > message [15047659] size [7388] is inserted > > Jun 13 15:28:55 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [6137] mailbox [26902] > message [15047661] size [123777] is inserted > > Jun 13 15:28:56 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [1158] mailbox [4976] message > [15047663] size [142518] is inserted > > Jun 13 15:28:57 mailhost-4a dbmail/lmtpd[4814]: Notice:[message] > sort_deliver_to_mailbox(+2260): useridnr [3098] mailbox [13445] > message [15047665] size [2357] is inserted > > > > Any thoughts welcome, maybe I'm missing something it just seems a bit > slow that's all and I think I'll be here a while getting 16k+ emails > into the db for customers. > > > > James G > > > > _______________________________________________ > DBmail mailing list > [email protected] > http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail > -- ________________________________________________________________ Paul J Stevens pjstevns @ gmail, twitter, github, linkedin www.nfg.nl/[email protected]/+31.85.877.99.97 _______________________________________________ DBmail mailing list [email protected] http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail _______________________________________________ DBmail mailing list [email protected] http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
