At 05:11 PM 6/20/2003 -0400, you wrote:
Just saw this on /. and thought that you all might be interested...
http://projects.puremagic.com/greylisting/
I've just published a paper on a new and unique spam blocking method called "Greylisting". The
We have comparable mechanisms designed into the AI of Message Sniffer... When we thought about concepts like this: delaying then allowing messages... We discovered a dark side. Spammers will likely adopt a policy of automatically resending their spam after a delay - even if it was not rejected (because they won't wait to see if the message got through).
It is likely that as methodologies like this become more widely deployed spammers will evolve to a "double barrelled" approach of delivery where each message is shot-gunned two or more times to each address as a matter of course. While this requires significantly more bandwidth from the spammers it is clear that the costs of bandwidth are not a sufficient deterrent - in part because more and more spammers are not paying for ANY of that bandwidth (jeem & variants of all "compromised third party" mechanisms).
The failure in thinkng behind this "greylisting" approach is that it holds the expectation that the spammer will be forced to follow RFCs and resend their messages in response to temporary failures according to those protocols. In fact, spammers already do not pay any attention to the protocols in the first place, and since automatically re-broadcasting their messages satisfies the "appearance" of a retry in the current protocol there is in fact very little additional cost-in-complexity for the spammer.
None the less, the delay-before-accept methodology is somewhat effective and is therefore inevitable (I think) as an escalation in the battle continues. (some similar and more lightweight implementations are already being used and experimented with) Saddly delay-before-accept will bring with it a profound increase in the bandwidth used by spammers.
While it seems on the surface that forcing spammers to retry their delivery will complicate their lives and increase their costs, the reality is that there is little additional cost for the spammers who already use mechanisms that virtually ignore standard protocols. Their solution is simple: send all messages again after an hour or so... and maybe again after that.
Another unfortunate variant of this response is the "radio rotation" methodology whereby the spammer delivers to their spambot (some compromised or dedicated hardware) a message to be delivered to the current lists. All messages at the spambot are repeatedly transmitted in rotation (like songs being played at a commercial radio station) until the message is removed from the "playlist". The spambot simply retransmits each message in it's queue repeatedly. As new messages are uploaded to the spambot, older messages are removed from the playlist to make room.
It's scary I know, but it's on it's way if it's not already in place (there is some evidence that this is already in wide deployment with some spammers).
_M
Pete McNeil (madscientist)
President, MicroNeil Research Corporation
Chief SortMonster, www.SortMonster.com
PS: I've considered a similar protocol that would force the required complexity on the spammers but since it would require broad deployment to be effective the methodology has been shelved (at least for now). Here is a short description of the IRRQ protocol. (IRRQ = Intelligent Retry Request).
IRRQ adjusts the SMTP protocol by enforcing a lightweight authentication (automated challenge) for new senders to an MTA. IRRQ is still in consideration for later deployment as part of a COT protocol suite in SortMonster. (COT = Circle Of Trust).
1. A new sender (SMTP SENDER + SOURCE MTA) attempts to deliver a message.
2. The new sender is not in the local COT reference so their message is initially rejected with a temporary failure. However the temporary failure messages is modified to include an authentication code (in the form of a special email address) for future retries of this message. The authentication code is a secure one-time-pad.
<- 451 Please try again later using <[EMAIL PROTECTED]>
3. The sending MTA stores the authentication code with the message to be retried. The receiving MTA stores the authentication message and envelope information with an expiration time and a guard time.
4. The receiving MTA may perform other operations to automatically or manually white-list the new sender. For example, in a COT model the peers in the local COT might be queried for a rating of the sender or an acceptance policy. As a result any sending MTAs that are not implementing the protocol can be accommodated after the delay dependent upon the policies of the receiving MTA and the services (such as COT, RBLs, etc.) at it's disposal.
5. The sending MTA retries the message after a given guard time (established by policies, but typically between 15 and 120 minutes). In this retry the authentication code is added to the envelope and sent as the first recipient.
-> RCPT TO: <[EMAIL PROTECTED]>
6. The receiving MTA recognizes the authentication code and retrieves it's stored copy of the envelope for verification. The sending MTA completes the envelope and if it matches the stored version the message is accepted normally.
7. The authentication message is retired and may be stored for a period in order to detect hack attempts. A particular authentication code can only be used once legitimately.
NOTE 1: Sending MTAs that do not implement IRRQ are not effected by the adjustments to the protocol and may still be accepted based on policy decisions evaluated in step 4.
NOTE 2: Attempts to hack IRRQ by sending falsified authentication codes, reusing codes, or altering the envelope associated with the code will result in strong negative ratings within a COT.
NOTE 3: This methodology strongly biases the mail system against spammers by forcing legitimate senders to properly implement the retry protocol. Spammers typically use systems which transmit messages (on the fly) without regard to bounces or other response messages. Simple re-transmission counters to the IRRQ protocol will not be effective.
