Re: [Mailman-Developers] [Greg Stark gsstark@mit.edu] Re: Bounce removal parameters default values
J C Lawrence [EMAIL PROTECTED] writes: On 17 Jun 2004 14:36:39 -0400 Greg Stark [EMAIL PROTECTED] wrote: In the absence of VERP this is far more difficult than it at first seems. The simple question becomes, even in the presence of a customised test message, how do you recognise a bounce for that particular message vs others? Well of course that's what VERP was invented for after all. -- greg ___ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org
Re: [Mailman-Developers] [Greg Stark gsstark@mit.edu] Re: Bounce removal parameters default values
Brad Knowles [EMAIL PROTECTED] writes: At 2:36 PM -0400 2004-06-17, Greg Stark wrote: Virus scans are only one type of bounce that could cause someone to be unsubscribed spuriously. For example, most mail servers have a maximum message size for example. Consider the security implications: all I have to do to mass unsubscribe many people--even everyone--on a list is send a message over 50k. Everyone using old versions of sendmail will be unsubscribed. A larger message will unsubscribe anyone using most modern MTAs. Nor do the tests that require multiple bounces protect anything; I just have to send my attack a few times quickly. That's one whacky line-wrap algorithm your MUA uses. 50k?!? Where are you getting this number? Maximum message size on most MTAs is usually a default of something like 1-10MB, or even unlimited. In more than ten years of specializing in running mail systems, I don't think I have *once* seen an MTA that was default configured to a maximum message size of just 50k. Well I said what I meant, old version of sendmail. 50k was indeed the standard maximum size for sendmail installs prior to MIME attachments and html-mail and all these new-fangled gadgets. I'm subscribed to plenty of mailing lists where attachments and html mail are severely discouraged so even today it wouldn't be out of place to refuse mail from these lists over 50k. This is basically what Mailman is now doing. From the Mailman-2.1.5/NEWS file: - The bounce processor has been redesigned so that now when an address's bounce score reaches the threshold, that address will be sent a probe message. Only if the probe bounces will the address be disabled. The score is reset to zero when the probe is sent. Also, bounce events are now kept in an event file instead of in memory. This should help contain the bloat of the BounceRunner. Hum. Well, then, uh, great! :) Thanks. Now I just have to figure out why I'm still being dropped. I guess some of these mailing lists are old versions of mailman. This is version skew issue is going to be a big issue going forward. Users of mailing lists and ultimately any network service are at the mercy of the admins of every list they're on to upgrade. -- greg ___ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org
[Mailman-Developers] [Greg Stark gsstark@mit.edu] Re: Bounce removal parameters default values
So the problem I described last January and again mentioned last September is still happening to me, and now to a lot more people. It will only become more and more prevalent as viruses become more common and sites that filter them become more common. Perhaps I should restate the problem more simply. Mailman is committing the basic sin of network security -- receiving data from the network and trusting it for purposes other than as opaque data. It is using messages posted to the list -- the content and format of which it does not control -- to detect bouncing email addresses. Because of this it cannot tell if the bounces it's receiving are caused by a broken email address or caused by some particularity of the posted message. Virus scans are only one type of bounce that could cause someone to be unsubscribed spuriously. For example, most mail servers have a maximum message size for example. Consider the security implications: all I have to do to mass unsubscribe many people--even everyone--on a list is send a message over 50k. Everyone using old versions of sendmail will be unsubscribed. A larger message will unsubscribe anyone using most modern MTAs. Nor do the tests that require multiple bounces protect anything; I just have to send my attack a few times quickly. Really Mailman should simply not trust outside data for any purpose. It should treat the bounces received from mailing list messages purely as hints. It should then send its *own* message with content not subject to any control from outside to the user. Only if that known inoffensive message bounces should it consider removing the user. This is really a DOS security issue, though the worst case attack is unsubscribing many users of a list. That it gets triggered normally even when not specifically under attack only makes the problem apparent. ---BeginMessage--- The problem I described in January is still happening. I find the current bounce processing of mailman to be inadequate. Something more like the bounce processing of ezmlm is needed. I should not be removed from a mailing list purely on the basis of bounces of uncontrolled messages. The messages that bounced could have been spam or outlook worms or whatever. Before removing a subscriber mailman should send a message with known content testing the address. Only if such a message bounces should a user be dropped. As it is I'm being removed from mailing lists whenever a new Outlook worm appears and sends multiple messages in a row. Or a new spammer discovers a list I'm on and sends multiple messages in a row to the list. It's especially bad on low-volume lists where it's quite possible for spam or Outlook worm messages to be the only messages for days. Greg Stark [EMAIL PROTECTED] writes: I find I am being removed from mailman mailing lists left and right. I believe the default values for the bounce removal should be reconsidered. It's possible that you haven't had many users in my situation and so haven't really had a chance to tune these parameters on the low end yet. But they clearly aren't working for me at a few sites. My particular situation is that my site has seen fit to filter viruses by refusing delivery. This causes a bounce from the remote MTA every time someone sends me an Outlook virus. Why my site administrators felt this was necessary is a question for another day, it's not like I use Outlook or like my spam filters wouldn't have thrown these messages away anyways, but whatever. The net result is that some small fraction of messages to me bounce and list management software notices this. The only reason I became aware of the problem was because ezmlm also does this type of processing but it sends a warning message before removing users. It only removes you if the warning message itself bounces. In fact it sends two such warning messages and only removes the user if *both* bounce. This provides the user with a chance to react to the first message and fix the problem -- if they ever see the message. I could beg for a similar feature in mailman, but I'm not sure it's necessary. But I am sure it's necessary to tune the bounce processing parameters. The relatively few bounces I'm generating shouldn't be causing me to get removed when all the real messages are being delivered fine. It seems the legitimate messages that are correctly delivered should reset the count of bounces to 0. Reading the source it seems it has to see DEFAULT_MAX_POSTS_BETWEEN_BOUNCES such legitimate posts between messages. I'm fairly convinced this parameter should always be 0. If any successful delivery occurs the user should never be removed due to bounces. What I don't understand is how DEFAULT_MAX_POSTS_BETWEEN_BOUNCES relates to the parameters I see in the admin. None of the parameters in the admin corresponds to this. How is it calculated? -- greg ___ Mailman-Developers mailing list [EMAIL
[Mailman-Developers] Happy Mailman Reminder Day
I forgot to wish all a happy Mailman Reminder Day yesterday. Do these reminders not bother everyone else? Getting a few dozen of these once a month used to really piss me off. I long since sent them to my spam folder but just on principle whoever decided these should be enabled by default ought to be first against the wall when the revolution comes. Subject: Topics Topics: lists.linux.it mailing list memberships reminder lists.vipul.net mailing list memberships reminder mail.kde.org mailing list memberships reminder lists.berlios.de mailing list memberships reminder gnucash.org mailing list memberships reminder lists.bestpractical.com mailing list memberships reminder python.org mailing list memberships reminder www-124.ibm.com mailing list memberships reminder redhat.com mailing list memberships reminder lists.gnucash.org mailing list memberships reminder ethereal.com mailing list memberships reminder postgis.refractions.net mailing list memberships reminder ---BeginMessage--- This is a reminder, sent out once a month, about your lists.linux.it mailing list memberships. It includes your subscription info and how to use it to change it or unsubscribe from a list. You can visit the URLs to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. In addition to the URL interfaces, you can also use email to make such changes. For more info, send a message to the '-request' address of the list (for example, [EMAIL PROTECTED]) containing just the word 'help' in the message body, and an email message will be sent to you with instructions. If you have questions, problems, comments, etc, send them to [EMAIL PROTECTED] Thanks! Passwords for [EMAIL PROTECTED]: List Password // URL [EMAIL PROTECTED] [EMAIL PROTECTED] http://lists.linux.it/options/barcode/gsstark%40mit.edu ---End Message--- ---BeginMessage--- This is a reminder, sent out once a month, about your lists.vipul.net mailing list memberships. It includes your subscription info and how to use it to change it or unsubscribe from a list. You can visit the URLs to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. In addition to the URL interfaces, you can also use email to make such changes. For more info, send a message to the '-request' address of the list (for example, [EMAIL PROTECTED]) containing just the word 'help' in the message body, and an email message will be sent to you with instructions. If you have questions, problems, comments, etc, send them to [EMAIL PROTECTED] Thanks! Passwords for [EMAIL PROTECTED]: List Password // URL [EMAIL PROTECTED] izicir http://lists.vipul.net/mailman/options/ricochet-devel/gsstark%40mit.edu ---End Message--- ---BeginMessage--- This is a reminder, sent out once a month, about your mail.kde.org mailing list memberships. It includes your subscription info and how to use it to change it or unsubscribe from a list. You can visit the URLs to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. In addition to the URL interfaces, you can also use email to make such changes. For more info, send a message to the '-request' address of the list (for example, [EMAIL PROTECTED]) containing just the word 'help' in the message body, and an email message will be sent to you with instructions. If you have questions, problems, comments, etc, send them to [EMAIL PROTECTED] Thanks! Passwords for [EMAIL PROTECTED]: List Password // URL [EMAIL PROTECTED] password https://mail.kde.org/mailman/options/kde-devel/gsstark%40mit.edu ---End Message--- ---BeginMessage--- This is a reminder, sent out once a month, about your lists.berlios.de mailing list memberships. It includes your subscription info and how to use it to change it or unsubscribe from a list. You can visit the URLs to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. In addition to the URL interfaces, you can also use email to make such changes. For more info, send a message to the '-request' address of the list (for example, [EMAIL PROTECTED]) containing just the word 'help' in the message body, and an email message will be sent to you with instructions. If you have questions, problems, comments, etc, send them to [EMAIL
[Mailman-Developers] [java-help@gcc.gnu.org] ezmlm warning
As an example of what I mean this is the notice that ezmlm sends. It's really helpful too, it explains that the messages have been bouncing, provides an example of the bounce so the user can make a guess *why* they're bouncing, and provides a link to the archives (actually instructions on retrieving them by e-mail). It also explains that in order to drop you from the list the bounce notification message has to itself bounce. In fact it goes a step further and sends a followup message. It will only remove you if that message bounces too. I think mailman really needs something like this. I haven't been removed from any ezmlm lists even though I periodically get these notifications from lots of them. In fact I wouldn't even know *why* I was being dropped from mailman lists regularly if it weren't for these ezmlm notifications. That's really bad. Mailman drops people without any warning or notification of what it's doing or why. ---BeginMessage--- Hi! This is the ezmlm program. I'm managing the [EMAIL PROTECTED] mailing list. Messages to you from the java mailing list seem to have been bouncing. I've attached a copy of the first bounce message I received. If this message bounces too, I will send you a probe. If the probe bounces, I will remove your address from the java mailing list, without further notice. I've kept a list of which messages from the java mailing list have bounced from your address. Copies of these messages may be in the archive. To retrieve a set of messages 123-145 (a maximum of 100 per request), send an empty message to: [EMAIL PROTECTED] To receive a subject and author list for the last 100 or so messages, send an empty message to: [EMAIL PROTECTED] Here are the message numbers: 15761 15819 16007 --- Enclosed is a copy of the bounce message I received. Return-Path: Received: (qmail 8053 invoked for bounce); 18 Sep 2003 10:04:12 - Date: 18 Sep 2003 10:04:12 - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=1063263222sources.redhat.com1345767 Subject: failure notice Hi. This is the qmail-send program at sources.redhat.com. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. [EMAIL PROTECTED]: 18.7.21.83 failed after I sent the message. Remote host said: 550 5.0.0 Possible virus (badname) ---End Message--- -- greg ___ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers
[Mailman-Developers] Re: Bounce removal parameters default values
Harald Meland [EMAIL PROTECTED] writes: [Greg Stark] Before removing a subscriber mailman should send a message with known content testing the address. Only if such a message bounces should a user be dropped. Uhm... what parts of such a known content message do you think can safely be assumed to still be discernible when Mailman receives the bounce message? You've misunderstood me. I'm not suggesting it should expect the bounce message to have known content. What I'm suggesting is that Mailman should *send* a message with known content itself, and only if that message bounces should it decide the address is invalid. This is what ezmlm does. As much as I dislike ezmlm and qmail for other reasons and like Mailman for other reasons, this is one thing it gets right and Mailman gets wrong. Deciding an address is invalid on the basis of messages posted to the list is bogus. Mailman can't know whether the message posted to the list bounced because the address was invalid, or merely because the content of that particular message triggered a content-based filter. -- greg ___ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers
Re: [Mailman-Developers] Re: Bounce removal parameters default values
Barry Warsaw [EMAIL PROTECTED] writes: On Fri, 2003-09-26 at 11:41, Greg Stark wrote: What I'm suggesting is that Mailman should *send* a message with known content itself, and only if that message bounces should it decide the address is invalid. It seems difficult to test a negative (what? it doesn't bounce after 10 days? I guess it'll never bounce). I prefer Mailman's positive test approach of sending several notices and requiring an explicit confirmation for reinstatement. That sounds great, except I'm subscribed to 183 lists, mostly low volume. Periodically I get interested in some project I put aside long ago, check my mail folder for it and discover I've stopped receiving messages months ago. That sucks. I don't see why using a message with known content is any more of a negative test than basing the decision on list messages with unknown content. If you get a bounce from a probe you disable the recipient. That's as much a positive test as looking for bounces. Deciding an address is invalid on the basis of messages posted to the list is bogus. Mailman can't know whether the message posted to the list bounced because the address was invalid, or merely because the content of that particular message triggered a content-based filter. Bounce messages triggered by content-based filters are evil and must be eradicated. As it turns out in this case the mail server isn't directly generating a bounce, it's returning an SMTP 5xx error. So the previous SMTP server in the chain generates a proper bounce to the envelope address. I've discussed this with the mail admins and their position is that the filters aren't perfect and have caught valid mail in the past. They're worried that simply bitbucketing the mail will cause valid mail to be lost. Causing valid bounces to be sent to the envelope sender means the innocent victim in the forged From header isn't harassed with bounce messages. Instead the list that resent the message gets the bounce and processes it as a failed message -- which in fact is exactly what happened. It just shouldn't extrapolate from those messages to assuming the mailbox itself is down. When SoBig.F came out, we had effective filters in place within a day or so for the specific viruses themselves. What absolutely killed us was all the helpful bounces that the zillions of content filters send when they block such a message. And even if you think /that's/ okay, not putting limits on those block messages is still evil. Most of this only applies to the stupid filters that incorrectly send bounce messages to the From header, and often do it with a valid envelope sender themselves so they're just asking for mail loops. -- greg ___ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers
Re: [Mailman-Developers] Re: Bounce removal parameters default values
Barry Warsaw [EMAIL PROTECTED] writes: - We probe your address for a while, and if we get a bounce, then we disable you and do the normal notifications for reinstatement. I don't understand this one. Why would you have to poll to check for bounces. You handle the bounce as it comes in. - If we /don't/ get a bounce from you for, what? X number of days, what If you don't get a bounce you don't do anything. Do you normally do things when nothing happens? I don't necessarily trust that we can send only one, but OTOH I don't want to fill up your inbox with probe messages. Why would you keep sending probes? You only send probes when the conditions are met which are the current conditions for disabling a user. So what I'm suggesting is: . When you get a bounce from a list message you record it and see if the limits have been reached. If N consecutive previous list messages to the address have bounced covering the minimum time period etc then you send a probe. . When you get a bounce from a probe you always disable the address. You could add a check to see if the probe was within the last few days and check whether any list messages have gotten through since the probe was sent, but all of that is just for paranoia's sake, in case someone flushes an old mail spool after fixing a mailbox. Actually ezmlm takes that one step further. If a probe bounces then it waits some time period and sends another probe. If the second probe bounces then you're dropped. -- greg ___ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers
Re: [Mailman-Developers] Re: Bounce removal parameters default values
My principled side says that an alarming number of sites actually do use content filters and that they are a reality of email life and we should properly handle reality. Content filters are not necessarily evil. It's bouncing to the From header that's evil. If it makes you feel better consider the case of somebody who has their MTA configured to refuse messages over a certain size, like all old installations of sendmail for example. So if for a few days only large kernel patches are send to the mailing list and mailman sees every message to that user bounce. It doesn't mean the address is any worse off than usual, it's just those messages that were refused. -- greg ___ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers
[Mailman-Developers] Re: Bounce removal parameters default values
The problem I described in January is still happening. I find the current bounce processing of mailman to be inadequate. Something more like the bounce processing of ezmlm is needed. I should not be removed from a mailing list purely on the basis of bounces of uncontrolled messages. The messages that bounced could have been spam or outlook worms or whatever. Before removing a subscriber mailman should send a message with known content testing the address. Only if such a message bounces should a user be dropped. As it is I'm being removed from mailing lists whenever a new Outlook worm appears and sends multiple messages in a row. Or a new spammer discovers a list I'm on and sends multiple messages in a row to the list. It's especially bad on low-volume lists where it's quite possible for spam or Outlook worm messages to be the only messages for days. Greg Stark [EMAIL PROTECTED] writes: I find I am being removed from mailman mailing lists left and right. I believe the default values for the bounce removal should be reconsidered. It's possible that you haven't had many users in my situation and so haven't really had a chance to tune these parameters on the low end yet. But they clearly aren't working for me at a few sites. My particular situation is that my site has seen fit to filter viruses by refusing delivery. This causes a bounce from the remote MTA every time someone sends me an Outlook virus. Why my site administrators felt this was necessary is a question for another day, it's not like I use Outlook or like my spam filters wouldn't have thrown these messages away anyways, but whatever. The net result is that some small fraction of messages to me bounce and list management software notices this. The only reason I became aware of the problem was because ezmlm also does this type of processing but it sends a warning message before removing users. It only removes you if the warning message itself bounces. In fact it sends two such warning messages and only removes the user if *both* bounce. This provides the user with a chance to react to the first message and fix the problem -- if they ever see the message. I could beg for a similar feature in mailman, but I'm not sure it's necessary. But I am sure it's necessary to tune the bounce processing parameters. The relatively few bounces I'm generating shouldn't be causing me to get removed when all the real messages are being delivered fine. It seems the legitimate messages that are correctly delivered should reset the count of bounces to 0. Reading the source it seems it has to see DEFAULT_MAX_POSTS_BETWEEN_BOUNCES such legitimate posts between messages. I'm fairly convinced this parameter should always be 0. If any successful delivery occurs the user should never be removed due to bounces. What I don't understand is how DEFAULT_MAX_POSTS_BETWEEN_BOUNCES relates to the parameters I see in the admin. None of the parameters in the admin corresponds to this. How is it calculated? -- greg ___ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers -- greg ___ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers
[Mailman-Developers] Bounce removal parameters default values
I find I am being removed from mailman mailing lists left and right. I believe the default values for the bounce removal should be reconsidered. It's possible that you haven't had many users in my situation and so haven't really had a chance to tune these parameters on the low end yet. But they clearly aren't working for me at a few sites. My particular situation is that my site has seen fit to filter viruses by refusing delivery. This causes a bounce from the remote MTA every time someone sends me an Outlook virus. Why my site administrators felt this was necessary is a question for another day, it's not like I use Outlook or like my spam filters wouldn't have thrown these messages away anyways, but whatever. The net result is that some small fraction of messages to me bounce and list management software notices this. The only reason I became aware of the problem was because ezmlm also does this type of processing but it sends a warning message before removing users. It only removes you if the warning message itself bounces. In fact it sends two such warning messages and only removes the user if *both* bounce. This provides the user with a chance to react to the first message and fix the problem -- if they ever see the message. I could beg for a similar feature in mailman, but I'm not sure it's necessary. But I am sure it's necessary to tune the bounce processing parameters. The relatively few bounces I'm generating shouldn't be causing me to get removed when all the real messages are being delivered fine. It seems the legitimate messages that are correctly delivered should reset the count of bounces to 0. Reading the source it seems it has to see DEFAULT_MAX_POSTS_BETWEEN_BOUNCES such legitimate posts between messages. I'm fairly convinced this parameter should always be 0. If any successful delivery occurs the user should never be removed due to bounces. What I don't understand is how DEFAULT_MAX_POSTS_BETWEEN_BOUNCES relates to the parameters I see in the admin. None of the parameters in the admin corresponds to this. How is it calculated? -- greg ___ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers
