Viktor Dukhovni <[email protected]> (Mi 03 Feb 2016 09:20:49 CET): > On Wed, Feb 03, 2016 at 08:33:06AM +0100, Heiko Schlittermann wrote: > > > Does anybody know anything about benchmarking an MTA? > > What do we count as performance? > > > > e.g.: > > visible by the user: time, a message spent in the queue > > visible by the admin: (spooled messages)/time the MTA can send > > visible by the sending mta: messages/time the MTA can accept¹ > > > > Any other suggestions? > > > > Would anybody be willing to share performance stats? > > (In a first step: submit the results from a tailored eximstats output?) > > Meaningful numbers are very difficult to come by because performance > is substantially limited by how much anti-spam/anti-virus scanning > one enables inbound, and which RBLs one queries, ... Or what kind > of rate limits one's outbound traffic is subjected to by the major > mailbox providers (and where most of one's email is sent).
I'd like to see a performance that is solely bound to resources we can't influence. But as - e.g. - Exim's queue runners step on each other's feet, it't not the fault of the remote site. > A lot can also depend on proper tuning. The available resources > of an MTA can range by 1, 2 or more orders of magnitude in RAM > capacity, disk IOPS and network bandwidth. Some tuning may be > required to adjust the MTA to utilize the available resources. Same as above. Of course, lots of RAM accelerate the processing, lowering the risk of locking problems (given a constant number of queue runners). > So performance is extremely variable from site to site for the same > software. My impression is that Exim is chosen more for flexibility > of built-in content inspection, and not so much performance. Exim as a MTA framework :) > would, for example, expect Postfix (when not throttled on the > receiving end) to out-perform Exim on raw throughput, but definitely > not on flexibility of ACLs and content inspection. Lego vs. Playmobil :) Yes, I agree, Exim is not famous for it performance, compared with Postfix. But I'm looking for a better definition of the term "performance", what do people consider as the number to compare. > As for what to measure, I've usually focused on latency first > because the MTAs I managed were handling people-to-people mail > primarily, rather than bulk newsletters and the like. That matches the "queue time" (optionally logged as QT=…), the time a given message spent in the queue. I agree, it's the most user visible factor. > Maintaining throughput under load is another important metric, some > MTAs (notably Sendmail) tended to over-throttle under load and do > very poorly when stressed. -- Heiko
signature.asc
Description: Digital signature
-- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
