I think you've lost track of the topic, Len (certainly abetted by the mixture of several sub-topics in the old thread).
Let me recap: - Terry thought he might have seen MS drop TCP/IP connections in midstream once the DATA had exceeded a fixed size, causing a retry instead of a DSN. He noted that this was a working hypothesis only. - You said this sounded fine to you, as it saved bandwidth. - Scott and I reminded you that without a proper close to the connection, it does not save on aggregate bandwidth, but is actually completely wasteful over time, as the RFCs require a retry. - Mike felt that the supposed practice was outdated in either case, as the ESMTP SIZE property should be used. - You noted that someone attempting to send an oversize message for malicious purposes would misstate its SIZE. - I showed that the RFCs make it clear that "SMTP-like" data sent out of sequence from receiver to sender has no reasonable expectation of being buffered, let alone impacting the delivery status of the message. - Terry reposted with a log snippet that suggested to you that MS was actually interrupting the DATA stream to send a 552, *then* dropping the connection--yet that this, too, was apparently causing a retry instead of a DSN. - You said this, too, sounded fine to you, as everyone would be happy since an SMTP-like sequence of bytes was being sent. - I reminded you that the only reason we're discussing this at all is that some mailservers are not happy! To the RFC-compliant sender, the fact that the request/reply sequence was not completed successfully is a clear signal that the message should be retried. That's fine and dandy for all...until one side punishes the other for adherence to the RFCs. - You stated that known standards used to construct MTAs are secondary to defending against attacks, and--implicitly--that sending MTAs must be able to interpret undocumented behavior, on the fly, in precisely the fashion intended by the receiver, or suffer the consequences. You did not present an explanation of how remote MTAs are to interpret behavior not governed by open standards or even proprietary documentation. - I noted that other defense vectors, such as the socket, firewall, and router level, were much more efficient at blocking such attacks, with an identical chance of sender retries (that is, 100%) but less I/O toll on the receiving host and no RFC argument to speak of--that is, such defenses do not expect the sending MTA to send a DSN, as they take into account the RFC-mandated retries. - Mike had a few other thoughts on SMTP and ESMTP. As you can see, NO ONE is saying you can't use extremely large and/or recurrent oversize messages as cause to blackhole a remote sender. NO ONE is saying that oversize messages are not a potential attack vector, in addition to being common in normal traffic, and so should be treated with special sensitivity and monitoring. NO ONE is saying that proper self-defense may not entail dropping connections in midstream due to real-time metering of message characteristics. What I HAVE said is that, just as we don't expect senders not to retry after a dropped connection, you CANNOT expect senders to behave as if a series of bytes that happens to have *similarities* to RFC-mandated SMTP replies, but is sent out of sequence, was sent as part of a normal SMTP conversation. That's my point. That's it. And, both theoretically and practically, beyond debate. > The MX's I run monitor all activity and if some jerk sends me 20 > attempts of 5 Mb each, he's on the radar and the ip gets blocked. I think you're off-topic here, since I don't believe that your own servers implement the functionality/expectations being argued. Moreover, your tactics are not exactly clear from your post, but I think you mean one of these scenarios: Scenario 1: He sends you a message with SIZE 20000000, you take in 5 MB, then drop the connection--either by resetting the socket outright, or sending some meaningless bytes across the wire, then resetting the socket. You log the IP. You have told the sender that the convo didn't finish correctly. You have *instructed* the sender to retry. So the message stays in his queue and hits you 19 more times. You then block the IP. Reaction 1: That's justification for blocking the IP? Sounds ridiculously draconian. Scenario 2: He sends you a message with understated SIZE 4000000, you take in 5 MB and note that there's more data on the way, then drop the connection--either by resetting the socket outright, or sending some meaningless bytes across the wire, then resetting the socket. You log the IP. You have told the sender that the convo didn't finish correctly. You have *instructed* the sender to retry. So the message stays in his queue and hits you 19 more times. You then block the IP. Reaction 2: As above. Scenario 3: You 552 the oversize message after full receipt, so the server should not retry and the human should learn from his mistake. You then get 19 NEW oversize messages from the same IP address, 552ing each one and blackholing after the last. Reaction 3: This sounds great to me. It's strict without being absurd. *And it has nothing to do with the argument at hand.* > If I "RFC-correctly" allow him to send one 100 Mb msg just so I can > have the polite opportunity to 5xx him, the abuse count is under the > radar. Only if your "abuse count" is maintained in a backward fashion. You want an abuse counter that takes more variables into account, e.g. you want to meter the probability that a given sender is on the attack? Here are some tests to start, with varying weights: %CONNECTION_PROTO = %SMTP And %RECEIVED_MESSAGE_SIZE > %MAX_MESSAGE_SIZE %CONNECTION_PROTO = %SMTP And %RECEIVED_MESSAGE_SIZE >= (2 * %MAX_MESSAGE_SIZE) %CONNECTION_PROTO = %ESMTP And %MAILFROM_ANNOUNCED_SIZE = %NULL And %RECEIVED_MESSAGE_SIZE > %MAX_MESSAGE_SIZE %CONNECTION_PROTO = %ESMTP And %MAILFROM_ANNOUNCED_SIZE = %NULL And %RECEIVED_MESSAGE_SIZE >= (2 * %MAX_MESSAGE_SIZE) %CONNECTION_PROTO = %ESMTP And %MAILFROM_ANNOUNCED_SIZE <> %NULL And %RECEIVED_MESSAGE_SIZE > %MAILFROM_ANNOUNCED_SIZE %CONNECTION_PROTO = %ESMTP And %MAILFROM_ANNOUNCED_SIZE <> %NULL And %RECEIVED_MESSAGE_SIZE > %MAILFROM_ANNOUNCED_SIZE And %RECEIVED_MESSAGE_SIZE > %MAX_MESSAGE_SIZE %CONNECTION_PROTO = %ESMTP And %MAILFROM_ANNOUNCED_SIZE <> %NULL And %RECEIVED_MESSAGE_SIZE > %MAILFROM_ANNOUNCED_SIZE And %RECEIVED_MESSAGE_SIZE >= (2 * %MAX_MESSAGE_SIZE) Log deeply and you have a sensible system. Log as shallowly as you sound like you are right now, and you're breaking an RFC-compliant sender just for retrying; if you short-circuit the DSN, you've guaranteed that you will blackhole someone for one tiny offense. The only way you're going to get innocent servers, and innocent but educatable users, out of your radar is by 552ing the first attempt and trending subsequent messages, eventually blackholing. The thresholds could be proportional: 20 tries of 5 MB, 10 tries of 10 MB, 2 tries of 50 MB, just one 552ed try of 100 MB, as a general example. If you blackhole some senders during transmission--a threshold of zero--that could happen at > 100 MB. Of course, all of this metering is in addition to other checks and weights. > Eating 100 Mb incoming msgs that are 95 Mb bigger than I allow just > because it's required by RFC is plain silly and unreasonable. "Having my RFC-compliant product require others only to be the same is unreasonable." I'm not just baiting you when I say you sound like Microsoft. This really is a dangerous position for someone who's sounded off on them so often. For most of the rest of us, it wouldn't seem so aberrant. Did they pay you off secretly? > You guys sound like Internet mail is some kind of > tea-party-on-the-croquet-lawn. I'm sorry that you think we're too soft on defense. A unilateralist expectation that others will play by your unpublished rules is not a "cool solution." I think that lack of support for a proprietary, undocumented function on a publicly referenced server should not break an application that implements all standard functions. > I'm sure every major ISP has similar tactics in place, and Internet > is not suffering from it. I did a small amount of research on this. The test was to see if a connection over the EHLO SIZE, and without a MAILFROM SIZE, would be broken in midstream. Earthlink: 10 MB EHLO SIZE, 20 MB message sent, normal 552, DSN returned. ATT: 10 MB EHLO SIZE, 20 MB message sent, normal 552, DSN returned. MSN: 3 MB EHLO SIZE, 10 MB sent, connection broken after ".", message requeued (properly). Yahoo: 10 MB EHLO SIZE, 20 MB sent, connection broken after ".", message requeued (properly). This sample shows that these major ISPs are not doing anything brilliant, and that IMail reacts properly to their replies; if you know of any MTAs that would respond differently to the same replies, please post their logs. It's notable that MSN and Yahoo are not blacklisting simply for retrying as required by the RFCs, as you say you do (the MSN and Yahoo messages were left in the queue overnight until finally returned to the sender, and I was able to send again in the morning). Why they circumvent the DSN is beyond me. Perhaps to punish the sender's mail server more, and to make the sender wait agonizingly long for a reply? Yet without real 552 in the logs, one must rely on trial-and-error to find out why a message failed! To me, this is further proof that these two giants are giant jokes. > The kind of RFC "violation" I'm talking about doesn't break anybody > else's mail server or screw up Internet. Sure, *the first time the message is sent*, such tactics don't break anybody's mail server. But if the fact that the message is sent 20 more times, *as required by the RFCs*, ends up getting the sending IP blacklisted, you have ABSOLUTELY broken others' mail servers and played very, very unfair. > BTW, you RFC-huggers would love IMGate/postfix's new Sender Address > Verification, that rejects mail from a [EMAIL PROTECTED] that, in > real-time during the SMTP session, won't accept bounces. HUGE > amounts of crap blocked. I have no quibble with that. If you 5xx a RCPT, you should not expect a retry. If you were to break the connection instead, you should expect a retry. Q.E.D. Please read this post carefully before responding. -Sandy To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/ Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/
