On Sat, 2003-09-06 at 22:34, Russell Coker wrote: > On Sat, 6 Sep 2003 00:08, Eric Sproul wrote: > > until the entire message has been received and processed, the receiving > > MTA is not responsible for the message. In fact, I think this is > > RFC-specified. Why then, if the receiver isn't responsible, would it > > want to spend disk I/O queuing a message that may end up being rejected > > or may fail to come completely in? > > The incidence of messages that fail part way through is quite low. Expecially > in communications between big servers (which corresponds to a large portion > of the non-spam traffic). Optimising for the common case makes sense to me. >
I should think, though, that using a milter that will reject a message based on the DATA content as it is streaming in would increase the likelihood of such occurrences. For instance, a virus-scanning milter will reject the message as soon as it sees a signature, causing the conversation to be aborted. During a large outbreak, a non-trivial percentage of SMTP traffic may be viruses, so there is still a benefit to buffering in RAM vs. queueing to disk. However, I do agree that asynchronous writes and ext3 unlink-before-commit would mitigate the I/O hit. > > As many other ISP admins know, a large percentage of customers > > are the psychotic kind, prone to POPing their multi-MB mailboxes every > > $%^&[EMAIL PROTECTED] minute, and leaving all the messages on the server. This > > puts a > > non-trivial strain on even a fairly hefty dual-x86 box with H/W RAID5 > > and 2GB of RAM. > > I have not noticed that. I have only noticed a very small portion of users > doing that. With 1,000,000 users the number of psychotic POP users is small > enough that you can deal with them individually. > > Maybe customers of Dutch ISPs are smarter than those of whichever country you > are in. I'm in the US, but let's not start a flame thread over the collective intelligence of our respective populations... ;^P > Why not change your POP server to instead of rejecting the connections to put > gratuitous delays. So if the time since the last connection is < 5 minutes > then make every operation take an extra 18 seconds (some pop servers have 20 > second time outs). That would delay a minimal POP session by 72 seconds > which better than halve the load. I agree, but the political problem remains. Whether I outright reject or impose delays (which will probably generate timeout errors on the client), it generates calls from confused people. But, rather than spend the time explaining the situation, the company would rather the calls not come in at all. Ah, the joy of engineering. > > I did some more figuring on our mail volume and found that even though > > each of our 4 mail routers processes 11-12 messages/second (each message > > requires up to 20 LDAP lookups and a milter for spam filtering), I see > > A caching LDAP proxy would be good for this situation. Converting 20 LDAP > lookups over the network to a single LDAP lookup and 19 accesses to a local > cache daemon should provide some significant benefits. Yes, we run local slapd daemons on all the mailrouters, updated from a central master. We see very good performance from this setup, as noted above. Eric -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
