While benchmarking a SMTP content filter, using smtp-source as a traffic generator and smtp-sink as sink, the message transfer rates were much worse than expected (about 100 seconds, instead of just a few seconds for 1000 messages).
It turned out the problem is in a TCP session over a loopback interface between a content filter and smtp-sink. When pipelining is used and all the MAIL FROM, RCPT TO, and DATA arrive in one packet, the smtp-sink responds in two packets: the first is a "250 2.1.0 Ok" response to a MAIL FROM command, and the second packet carries a response to the rest: "250 2.1.5 Ok\r\n354 End data with...\r\n". The trouble is that there is a 0.1 second delay between the two response packets. The second packet is only sent by smtp-sink after receiving an ACK to the first, and that only happens after 0.1 seconds due to a delayed ACK setting of a system (FreeBSD 9.0, net.inet.tcp.delayed_ack=1). The workarounds are: - disable net.inet.tcp.delayed_ack globally or: - disable pipelining announcement in smtp-sink (option -p) or: - ignore pipelining announcement in a content filter The true solution seems to be to either disable Nagle in smtp-sink (TCP_NODELAY), or to send all the SMTP responses in one go. Seems the postfix itself does not suffer from this problem, only the smtp-sink. A tcpdump packet capture is available at: http://www.ijs.si/~mark/tmp/0.log.gz Mark
