Re: filtering of mailing lists and NATs
I am sorry, if I sound harsh, but I think this chain of mails is becoming more concerning than the amount of spam one receives. Could we put an end to it? James M Galvin wrote: Keith, Your NAT analogy is weak, very weak, at best. It's opening premise is flawed, as is this entire discussion of mail list filtering, because it confuses policy with implementation. The IETF has a policy of openness for all its mailing lists. The problem is most of the argument against filtering defines openness as all messages shall be distributed. This is false. Every IETF mailing list has a charter, a known purpose for its use. It is entirely reasonable and legitimate to reject all submissions that are outside the scope of the charter. If we can not agree on that point this whole discussion is pointless. Implementation is wholly separate from policy, and a primary concern for the list maintainer. A list maintainer needs to figure out how to identify messages that are within scope and ideally would like to automate that process. I would assert they can do this without anyone's approval or guidance. The only issue anyone in the IETF can have with that is if the list maintainer has a skewed sense of within scope or if whatever process they use generates false positives. But you can not know this until after the fact. We do so many things in this organization on the basis of subjective judgement with after decision peer review, (less so now than even just 5 years ago but still) why should this be any different? Mail filtering is not in and of itself a bad thing. It is a tool, a legitimate tool, that when used as part of a larger solution to the problem of maintaining the integrity of a mailing list is extremely valuable. Restricting the posting of messages to subscribers is not bad, it is an excellent choice for the first line defense against off-topic messages. The issue is whether it is the only solution employed. Messages from non-subscribers need to be reviewed to determine if they are within scope. In a worst case this review is done manually but it doesn't need to be. There are a few (I mean less than 5) additional technological criteria that can be applied that will correctly review 95% or more of the non-subscriber messages. This minimizes the manual work. I know this because I do this and have been doing it for years. I have a 100% success rate at keeping spam off mailing lists and no complaints. The total volume of email I deal with far exceeds the needs of all the IETF lists combined. This is not rocket science. Furthermore, I don't see how the occasional 24-48 hour delay in getting an occasional message distributed is bad. So many people have this idealistic view of email immediacy. Have you ever really looked at the Received: lines for messages distributed to the main IETF list? Messages to me typically take about 6 hours to get delivered but I've seen delays as long as 18 hours. And the delay is *not* at my end. Jim On Mon, 21 May 2001, Keith Moore wrote: Date: Mon, 21 May 2001 18:00:02 -0400 From: Keith Moore [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: filtering of mailing lists and NATs it occurs to me that most of the methods that have been proposed for filtering spam from mailing lists have a lot in common with NATs. in both cases, the proponents say (in effect) if it works for me and for my small set of test cases, it must be okay to impose this on everyone. if some legitimate traffic is excluded by my filters, they is of no consequence - they should be willing to jump through whatever hoops that I believe are appropriate. and if people have to abandon practices that they find useful in order to to get around my filters, that is of no consequence either, because they do not need to be doing those things anyway Keith -- The only way to solve a problem is to look at it in the face.
Re: filtering of mailing lists and NATs
On Wed, 23 May 2001, Keith Moore wrote: Every IETF mailing list has a charter, a known purpose for its use. It is entirely reasonable and legitimate to reject all submissions that are outside the scope of the charter. If we can not agree on that point this whole discussion is pointless. I couldn't disagree with you more. It's one thing for a chair to point out that a topic is not within the list's charter, and quite another thing for someone to arbitrarily filter material that he/she doesn't think is within the list's charter. So your point is simply that you want the decisions of what is in scope and what is not to be visible? This is entirely reasonable, in my opinion, since to do otherwise make us vulnerable to censorship. I can think of at least two ways to do this; there are probably others. Is this the only reason that you reject mail filtering, i.e., you are opposed to mail filtering behind closed doors? In any case, you delegate the job to the Chair. I don't see any reason why a Chair could not delegate this job, nor any prohibition against it for that matter, especially if the actions are visible. Implementation is wholly separate from policy, and a primary concern for the list maintainer. A list maintainer needs to figure out how to identify messages that are within scope and ideally would like to automate that process. I would assert they can do this without anyone's approval or guidance. I disagree that it is appropriate for a list maintainer (at least on an IETF list) to determine whether a message is in scope for the list, other than on a *very* coarse level for eliminating obvious spam. The chair and/or the AD have the authority determine whether things are in scope; the list maintainer should only filter things on their explicit authority. And it's not appropriate to filter anybody's input at the source unless they have repeatedly failed to follow the directions of the chair - and this should be only as a last resort. And now you're digging in to the next level of implementation, which is both the competence of and the criteria used by the list maintainer (be it the Chair or some other designee). I suspect you're equating list maintainer with the sysadmin who manages the technology. In that case I largely agree with your assessment above. However, more generally, we are talking about a moderator (not a censor). In that case, the criteria used really does depend on the competence of the moderator, but I really don't see a big issue here. It seems to me we choose a moderator much the same way we choose a Chair of a working group. Mail filtering is not in and of itself a bad thing. It is a tool, a legitimate tool, that when used as part of a larger solution to the problem of maintaining the integrity of a mailing list is extremely valuable. I don't disagree with this statement as a generality. But the way that you suggest that the tool be used would destroy the integrity of the mailing list as a vehicle for open discussion rather than maintaining it. I don't see how. I'm suggesting that filtering can be used to automate some of the process of maintaining the integrity of a mailing list and open discussion. I don't understand how you turned that around. Restricting the posting of messages to subscribers is not bad, it is an excellent choice for the first line defense against off-topic messages. Once again I empatically disagree. Whether a posting comes from a subscriber is completely orthogonal to whether the message is on topic. I agree. To complete my thinking I would add that most IETF lists are pretty good at being self-policing, as far as managing subscribers is concerned. In that context, if a subscriber starts moving outside scope we deal with that pretty well. Thus, it's not that subscribers are always on topic, it's that we know how to deal with subscribers who are off-topic. The issue is whether it is the only solution employed. The issue here is whether it is appropriate at all. Nobody has argued that other solutions could not be considered. As I asked above I will ask again here, is it that you are opposed to mail filtering as a tool or mail filtering behind closed doors? Jim
Re: filtering of mailing lists and NATs
In die Tue, 22 May 2001 20:49:51 -0400 Eric Rosen [EMAIL PROTECTED] scripsit: I do this for the mailing list of the MPLS working group, so I'm aware of what a nuisance it is. But as far as mailing list management goes, it's not nearly as big a nuisance as trying to figure out which of the error messages to owner-mpls are bogus and which are real. (The mailing list has 3000 members and each message to it results in 100 error messages.) mailman seems to have an automated way to put subscribers whose email bounces out of the list, but I must confess I prefer to look at the errors and decide case per case. Luckily, my lists have at most 400 users. ciao, .mau.
RE: filtering of mailing lists and NATs
Can someone remind me what spamming exactly is? From what I see in my inbox I must assume it is something like boring 1000s of ietf subscribers with tens of emails on filtering of mailing lists. Am I right? Leen Mak. -Original Message- From: Maurizio Codogno [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 23, 2001 8:51 To: [EMAIL PROTECTED] Subject: Re: filtering of mailing lists and NATs In die Tue, 22 May 2001 20:49:51 -0400 Eric Rosen [EMAIL PROTECTED] scripsit: I do this for the mailing list of the MPLS working group, so I'm aware of what a nuisance it is. But as far as mailing list management goes, it's not nearly as big a nuisance as trying to figure out which of the error messages to owner-mpls are bogus and which are real. (The mailing list has 3000 members and each message to it results in 100 error messages.) mailman seems to have an automated way to put subscribers whose email bounces out of the list, but I must confess I prefer to look at the errors and decide case per case. Luckily, my lists have at most 400 users. ciao, .mau.
Re: filtering of mailing lists and NATs
Keith, Your NAT analogy is weak, very weak, at best. It's opening premise is flawed, as is this entire discussion of mail list filtering, because it confuses policy with implementation. The IETF has a policy of openness for all its mailing lists. The problem is most of the argument against filtering defines openness as all messages shall be distributed. This is false. Every IETF mailing list has a charter, a known purpose for its use. It is entirely reasonable and legitimate to reject all submissions that are outside the scope of the charter. If we can not agree on that point this whole discussion is pointless. Implementation is wholly separate from policy, and a primary concern for the list maintainer. A list maintainer needs to figure out how to identify messages that are within scope and ideally would like to automate that process. I would assert they can do this without anyone's approval or guidance. The only issue anyone in the IETF can have with that is if the list maintainer has a skewed sense of within scope or if whatever process they use generates false positives. But you can not know this until after the fact. We do so many things in this organization on the basis of subjective judgement with after decision peer review, (less so now than even just 5 years ago but still) why should this be any different? Mail filtering is not in and of itself a bad thing. It is a tool, a legitimate tool, that when used as part of a larger solution to the problem of maintaining the integrity of a mailing list is extremely valuable. Restricting the posting of messages to subscribers is not bad, it is an excellent choice for the first line defense against off-topic messages. The issue is whether it is the only solution employed. Messages from non-subscribers need to be reviewed to determine if they are within scope. In a worst case this review is done manually but it doesn't need to be. There are a few (I mean less than 5) additional technological criteria that can be applied that will correctly review 95% or more of the non-subscriber messages. This minimizes the manual work. I know this because I do this and have been doing it for years. I have a 100% success rate at keeping spam off mailing lists and no complaints. The total volume of email I deal with far exceeds the needs of all the IETF lists combined. This is not rocket science. Furthermore, I don't see how the occasional 24-48 hour delay in getting an occasional message distributed is bad. So many people have this idealistic view of email immediacy. Have you ever really looked at the Received: lines for messages distributed to the main IETF list? Messages to me typically take about 6 hours to get delivered but I've seen delays as long as 18 hours. And the delay is *not* at my end. Jim On Mon, 21 May 2001, Keith Moore wrote: Date: Mon, 21 May 2001 18:00:02 -0400 From: Keith Moore [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: filtering of mailing lists and NATs it occurs to me that most of the methods that have been proposed for filtering spam from mailing lists have a lot in common with NATs. in both cases, the proponents say (in effect) if it works for me and for my small set of test cases, it must be okay to impose this on everyone. if some legitimate traffic is excluded by my filters, they is of no consequence - they should be willing to jump through whatever hoops that I believe are appropriate. and if people have to abandon practices that they find useful in order to to get around my filters, that is of no consequence either, because they do not need to be doing those things anyway Keith
Re: filtering of mailing lists and NATs
At 05:59 AM 5/23/2001, Keith Moore wrote: What about months of work wasted because a WG didn't get the input of those driven away by spam? that's equally as bad as the months of work wasted because the WG didn't get the input of someone driven away by the spam filter, of course. Keith, there are several barriers of entry for people who wish to work on Internet protocols. There are financial barriers, time barriers and most of all, educational barriers. We all have to learn how email lists work (some of us had to learn USENET), just as we all had to learn how to access the Internet. It is incumbent on the participant to move up the learning curve and follow the email list policy even if that includes extra effort. If that policy includes subscription, then you just have to go along with it, onerous or not. The Internet doesn't bend to individuals.
Re: filtering of mailing lists and NATs
Keith, there are several barriers of entry for people who wish to work on Internet protocols. There are financial barriers, time barriers and most of all, educational barriers. We all have to learn how email lists work (some of us had to learn USENET), just as we all had to learn how to access the Internet. It is incumbent on the participant to move up the learning curve and follow the email list policy even if that includes extra effort. If that policy includes subscription, then you just have to go along with it, onerous or not. The Internet doesn't bend to individuals. no disagreement about the general argument - we have to understand how to use our technology.and from time to time we have to learn to use new technologies. but subscribe-before-post is not inherently a feature of mailing lists. we have a choice about whether to impose that extra hump in the learning curve. my argument is that a choice to do so, for the specific case of IETF lists, is a poor one. Keith
Re: filtering of mailing lists and NATs
(maybe the above will fool majordomo into not filtering this message?) On Wed, 23 May 2001, Keith Moore wrote: Every IETF mailing list has a charter, a known purpose for its use. It is entirely reasonable and legitimate to reject all submissions that are outside the scope of the charter. If we can not agree on that point this whole discussion is pointless. I couldn't disagree with you more. It's one thing for a chair to point out that a topic is not within the list's charter, and quite another thing for someone to arbitrarily filter material that he/she doesn't think is within the list's charter. So your point is simply that you want the decisions of what is in scope and what is not to be visible? that's only one aspect of filtering list traffic based on the From address. It's not the only issue of concern. In any case, you delegate the job to the Chair. no, even the Chair should not be filtering messages based on whether he/she thinks they are in scope, for reasons stated previously. correcting people who post off-topic contributions is okay; editing or censoring those contributions is not. the only way to make those contributions (and the Chair's corrective action) sufficiently visible is to allow both to be posted to the list. I suspect you're equating list maintainer with the sysadmin who manages the technology. In that case I largely agree with your assessment above. However, more generally, we are talking about a moderator (not a censor). In that case, the criteria used really does depend on the competence of the moderator, but I really don't see a big issue here. It seems to me we choose a moderator much the same way we choose a Chair of a working group. no, I reject the entire concept of a moderator on an IETF list to do anything other than filter obvious spam. I don't care who is doing the censoring - the chair, the AD, the list maintainer, or someone else. It's inappropriate no matter who does it. Mail filtering is not in and of itself a bad thing. It is a tool, a legitimate tool, that when used as part of a larger solution to the problem of maintaining the integrity of a mailing list is extremely valuable. I don't disagree with this statement as a generality. But the way that you suggest that the tool be used would destroy the integrity of the mailing list as a vehicle for open discussion rather than maintaining it. I don't see how. I'm suggesting that filtering can be used to automate some of the process of maintaining the integrity of a mailing list and open discussion. I don't understand how you turned that around. because a discussion in which some participants' input are censored is not an open discussion. it lacks integrity because it pretends to be an open discussion when in reality it is subject to control and/or censorship, and this has a chilling effect on the discussion and on the result. Restricting the posting of messages to subscribers is not bad, it is an excellent choice for the first line defense against off-topic messages. Once again I empatically disagree. Whether a posting comes from a subscriber is completely orthogonal to whether the message is on topic. I agree. To complete my thinking I would add that most IETF lists are pretty good at being self-policing, as far as managing subscribers is concerned. In that context, if a subscriber starts moving outside scope we deal with that pretty well. Thus, it's not that subscribers are always on topic, it's that we know how to deal with subscribers who are off-topic. and we should use the same mechanism for dealing with non-subscribers are off-topic. whether a contributor is a subscriber is irrelevant. The issue here is whether it is appropriate at all. Nobody has argued that other solutions could not be considered. As I asked above I will ask again here, is it that you are opposed to mail filtering as a tool or mail filtering behind closed doors? mail filtering as a tool for bringing messages to the attention of a moderator is okay, but that moderator's role should be limited to filtering obvious spam. Keith
Re: filtering of mailing lists and NATs
So, here are the choices: 1. Save thousands of people from having to deal with multiple spams per day, at the cost of presenting a minor inconvenience to a few, or 2. Require thousands of people to receive and deal with spam (or to learn all about mail filtering), in order to avoid inconveniencing a few. Easy decision to make. For every bit of whining by the usual suspects, there are thousands of folks that are very happy to have the spam kept out of their mailbox automatically. (Every mailing list manager knows that the whining by Keith and Lloyd is nothing compared to the whining by the list members as they get spammed multiple times per day.) Indeed, this is a lot like the arguments re NAT. There are the thousands of people it helps, vs. the few who are yelling that the sky will fall if it is not stamped out.
Re: filtering of mailing lists and NATs
Date:Mon, 21 May 2001 20:21:10 -0700 From:grenville armitage [EMAIL PROTECTED] Message-ID: [EMAIL PROTECTED] | Most spammers strike me as opportunistic and not overly interested | in special-case-handling a couple of subscribe-to-send lists, Of course, and as long as they can get to the vast majority of their target, it will probably remain that way. But as soon as the spammers need to go to some extra effort to reach their audience, you can be sure they will. Remember, once, they sent from any random invented host name - then everyone started having their mailers reject mail from unknown domains - now all the spam comes from perfectly valid domains, which not only makes the checks for invalid domains a waste of time (the check spends time achieving nothing at all) but also results in all the failed spam (the bounces - and the abuse from people who received it) being dumped on whichever unfortunate site they picked to use as the domain name. A supposed technological fix to a non-technological problem that just made things worse, not better. Now we're having suggested that only known e-mail addresses be allowed to send to certain destinations. Assuming that becomes really popular (rather than just used on a small set of irrelevant lists) how long do you think it will be before the spammer's lists of names contain not only the destination address, but the From: address they should use to send to that address? I mean, how hard do you think it is to stick From: [EMAIL PROTECTED] in the heading of the mail? One more technological fix that won't work. And again, it will make things worse, since then we won't be able to tell easily what is traffic from people we expect to send to the lists, and what is not. This is not a technological problem - it is a social problem. We cannot fix spam by technological means - it has to be fixed by social means. And not only are technological fixes making things actually worse as illustrated above, they're also suggesting to people that perhaps there may be a technological fix that will actually finally solve the problem, which lessens the demand for real social fixes instead. Give up, let the spam through, deluge everyone with it - then the opposition to it will rise quickly enough, and become urgent enough, that the correct kind of remedies can be put in place. kre
Re: filtering of mailing lists and NATs
In your previous mail you wrote: This is not a technological problem - it is a social problem. We cannot fix spam by technological means - it has to be fixed by social means. = thanks for this nice summary about the spam problem! [EMAIL PROTECTED]
Re: filtering of mailing lists and NATs
You wrote: So, here are the choices: 1. Save thousands of people from having to deal with multiple spams per day, at the cost of presenting a minor inconvenience to a few, or 2. Require thousands of people to receive and deal with spam (or to learn all about mail filtering), in order to avoid inconveniencing a few. Another similarity to NATs is that you don't know how many people are behind a single (subscribed) address. For instance, I read your message via a local news server. Of course, this means that any attempt to work out the utility value of a filtering system must fail. I'm perfectly happy to filter messages to this list locally. To be frank, it takes a very small amount of my time. Surely people who want to subscribe to this list are capable of setting up local filters? Regards, -leo
Re: filtering of mailing lists and NATs
Date:Mon, 21 May 2001 20:21:10 -0700 From:grenville armitage [EMAIL PROTECTED] Message-ID: [EMAIL PROTECTED] | Most spammers strike me as opportunistic and not overly interested | in special-case-handling a couple of subscribe-to-send lists, Of course, and as long as they can get to the vast majority of their target, it will probably remain that way. But as soon as the spammers need to go to some extra effort to reach their audience, you can be sure they will. ... Now we're having suggested that only known e-mail addresses be allowed to send to certain destinations. Assuming that becomes really popular (rather than just used on a small set of irrelevant lists) how long do you think it will be before the spammer's lists of names contain not only the destination address, but the From: address they should use to send to that address? A long time, actually. While it is true that spammers will work around anything that seriously impedes the flow of spam, you have not shown that spam sent to lists is at all important to spammers. Every indication I see is that lists are primarily useful to spammers as a source of addresses to send spam to directly, and less as a target for spamming lots of people indirectly. Indeed, most spammers that send to lists seem totally uninterested in the fact that they are sending to a list; it is simply another address they have culled from some sort of scan. And while there have been some isolated reports of subscribe-then-send and send-using-a-subscriber strategies used by spammers, the frequency of their use appears to be way out of porportion to the number of lists that have successfully fended off spam by using various subscriber-only techniques. I mean, how hard do you think it is to stick From: [EMAIL PROTECTED] in the heading of the mail? Actually, maintaining an additional per-list address and keeping that address up to date is pretty difficult. It is much easier -- and quite effective -- to simply prowl for addresses that reach users directly. This is not a technological problem - it is a social problem. We cannot fix spam by technological means - it has to be fixed by social means. In general, I agree with this assessment. But that doesn't mean that some point fixes don't help in some cases. Ned
Re: filtering of mailing lists and NATs
So, here are the choices: 1. Save thousands of people from having to deal with multiple spams per day, at the cost of presenting a minor inconvenience to a few, or 2. Require thousands of people to receive and deal with spam (or to learn all about mail filtering), in order to avoid inconveniencing a few. you have it backwards. all subscribers of the list are 'inconvenienced' if we discourage legitimate contributions from folks who are not willing to jump through arbitrary and time-consuming hoops that we impose on them just because a few people insisted (even in the face of evidence to the contrary) that they knew what was best for everyone else. calling those hoops a 'minor inconvenience' is also misleading. Indeed, this is a lot like the arguments re NAT. There are the thousands of people it helps, vs. the few who are yelling that the sky will fall if it is not stamped out. the people who are helped by NAT are also hurt by NAT. but they might not realize that NAT is the reason that they cannot deploy IP telephony. they'll blame the new application rather than the NAT because they've been brainwashed into thinking that NAT is the right thing to do, and also because the guy who bought the NATs in the first place is not going to admit that he was wrong. Keith
Re: filtering of mailing lists and NATs
however, I have seen a couple of occasions where I believe that a 'moderator' acted inappropriately in filtering messages that came from non-subscribers but were arguably on-topic for the lists. So the non-subscriber subscribed, and their posts went through okay, right? no. the WG was badly in need of a clue from folks outside of the WG - because the WG was failing to understand how its work would interact with and/or affect other applications or protocols outside of its purview. the would-be contributor did not want to subscribe to the list because he/she had no desire to participate in the day-to-day conversations of the working group. after all, the contributor normally worked at layer X while the WG was working at layer Y. still, the WG needed the contribution. it would have benefited from knowing that what it was doing was inherently flawed, and that its poorly-informed design decisions would do harm and/or cause its work to be less useful than anticipated. but the capriciousness of the mailing list maintainer prevented this from happening, and many months of hard work were wasted. (If not, and the moderator was in fact filtering all posts to the mailing list in question, then this example is a red-herring.) seems like you've left a big hole in your case analysis. Gas tanks explode - we ban cars? if the gas tanks explode under normal or even occasional use, we do in fact recall the car. you seem to believe that non-subscribers are inherently illegimiate, and that any barriers we erect to make it more difficult for them to post are therefore justified. looks like circular reasoning to me. Keith
Re: filtering of mailing lists and NATs
Christian I would much rather receive and delete another annoying Christian proposition to get rich quick or see lurid pictures than tolerate Christian any form of censorship. As has been pointed out, the non-member messages can be moderated. It takes about one second to look at a message and tell whether it is unsolicited commercial or not. So the downside is that the non-member message may be delayed for a bit until the moderator gets to it. I wouldn't call that censorship. (I think one has to be very privileged indeed to confuse a small inconvenience with censorship.) Unless, of course, you think that people have a RIGHT to send unsolicited commercial email to IETF mailing lists.
Re: filtering of mailing lists and NATs
Here is a suggestion. Require people to subscribe to a list to post to the list. This is in addition to requiring subscription to receive posts mailed to the list. Nanog adopts this approach and has been fairly successful in avoiding spam, I believe. Subscription to Post can be made contingent on the subscriber not agreeing to post material that is out of scope for the list and willing to abide by the list administrator's decision to moderate inappropriate postings. Free-for-all type of lists are inherently prone to spam. Thanks. cheers, suresh --- Keith Moore [EMAIL PROTECTED] wrote: however, I have seen a couple of occasions where I believe that a 'moderator' acted inappropriately in filtering messages that came from non-subscribers but were arguably on-topic for the lists. So the non-subscriber subscribed, and their posts went through okay, right? no. the WG was badly in need of a clue from folks outside of the WG - because the WG was failing to understand how its work would interact with and/or affect other applications or protocols outside of its purview. the would-be contributor did not want to subscribe to the list because he/she had no desire to participate in the day-to-day conversations of the working group. after all, the contributor normally worked at layer X while the WG was working at layer Y. still, the WG needed the contribution. it would have benefited from knowing that what it was doing was inherently flawed, and that its poorly-informed design decisions would do harm and/or cause its work to be less useful than anticipated. but the capriciousness of the mailing list maintainer prevented this from happening, and many months of hard work were wasted. (If not, and the moderator was in fact filtering all posts to the mailing list in question, then this example is a red-herring.) seems like you've left a big hole in your case analysis. Gas tanks explode - we ban cars? if the gas tanks explode under normal or even occasional use, we do in fact recall the car. you seem to believe that non-subscribers are inherently illegimiate, and that any barriers we erect to make it more difficult for them to post are therefore justified. looks like circular reasoning to me. Keith = __ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/
Re: filtering of mailing lists and NATs
Forgive me here, but I was pondering the problem of mailing lists filtering last night, and want to float an idea. The problem as I understand it is that non-subscribers to a given mailing list may contribute good ideas or may be spammers. And short of human-directed analysis it's impossible to know whether the email should be forwarded or not. Further, by having only one person decide on what's appropriate, there is the possibility for intentional or inadvertent censorship. Also, it's a significant burden for someone to have to manually filter all of the email from non-subscribers. So here is the idea: For email that comes from non-subscribers, forward it to N subscribers randomly selected from the current subscribers. (Maybe pick from the most recent posters, since they are most likely to be active.) If one of subscribers thinks the mail is useful, he forwards it to the group. If more than one approves, still only one copy goes forward. (Software somewhere would prevent duplicates.) As long N is large enough and picked at random each time it would reduce dramatically the possibility for censorship, fairly share the load, and protect the email list from spammers, yet still allow for non-subscribing folks to contribute. I know this is fairly low-tech, but it seemed like a reasonable and practical solution to spamming. James
Re: filtering of mailing lists and NATs
As has been pointed out, the non-member messages can be moderated. yes they can. but this requires a moderator who has the time to do it, who can consistently do it in a timely manner, who acts as a spam filter rather censoring content with which he/she does not agree, and who is trusted by everyone (or very nearly everyone) who wants to participate on the list. and yes this is much better than insisting that people subscribe to a list in order to post. and it has worked fairly well for a number of the lists that I run. but I think it would be difficult to find a moderator for the ietf list that meets the above criteria. Keith
Re: filtering of mailing lists and NATs
Here is a suggestion. Require people to subscribe to a list to post to the list. worked great for the NAT WG list, which successfully used this technique to discourage input from people harmed by NAT. Keith
Re: filtering of mailing lists and NATs
So, here are the choices: 1. Save thousands of people from having to deal with multiple spams per day, at the cost of presenting a minor inconvenience to a few, or 2. Require thousands of people to receive and deal with spam (or to learn all about mail filtering), in order to avoid inconveniencing a few. you have it backwards. all subscribers of the list are 'inconvenienced' if we discourage legitimate contributions from folks who are not willing to jump through arbitrary and time-consuming hoops that we impose on them just because a few people insisted (even in the face of evidence to the contrary) that they knew what was best for everyone else. This assumes that list filtering cannot be done sensibly. This assumption is false; it can be done sensibly and is done sensibly all the time. And when it is done sensibly the amount of inconvenience is unnoticeable. Sure, there are plenty of lists that don't do filtering sensibly (including, alas, some IETF WG lists), but there are many others that do. Whether or not list filtering can be sensibly applied to a list with the characteristics of the main IETF list is just a matter of resources. The necessary technologies exist to cope with all the trickiness the IETF list presents and more. All we have to do is agree to apply them and find the resources to make it happen. calling those hoops a 'minor inconvenience' is also misleading. Only if the lists aren't managed correctly. Keith, I have to say that you are becoming your own worst enemy in this discussion. By insisting on an absolute policy of no filtering at all your ability to influence the policy that eventually is adopted is being compromised. As a result we are increasingly likely to end up with a list policy imposed that doesn't accomodate some aspect of real world behavior that could have been dealt with. I also find the comparisons with NAT to be strained at best. Ned
RE: filtering of mailing lists and NATs
So, here are the choices: 1. Save thousands of people from having to deal with multiple spams per day, at the cost of presenting a minor inconvenience to a few, or 2. Require thousands of people to receive and deal with spam (or to learn all about mail filtering), in order to avoid inconveniencing a few. you have it backwards. all subscribers of the list are 'inconvenienced' if we discourage legitimate contributions from folks who are not willing to jump through arbitrary and time-consuming hoops that we impose on them just because a few people insisted (even in the face of evidence to the contrary) that they knew what was best for everyone else. There is a fine line between anti-spam and censorship. I would much rather receive and delete another annoying proposition to get rich quick or see lurid pictures than tolerate any form of censorship. This translates into an engineering requirement. Anti-spam filters, like all filters generate false positive, i.e. declare as spam something that is in fact legitimate, and false negative, i.e. declare as legitimate a message that in fact is spam. The openness requirement of the IETF translates in a requirement to eliminate false negative. This is the IETF, we ought to be able to engineer that. -- Christian Huitema
Re: filtering of mailing lists and NATs
So, here are the choices: 1. Save thousands of people from having to deal with multiple spams per day, at the cost of presenting a minor inconvenience to a few, or 2. Require thousands of people to receive and deal with spam (or to learn all about mail filtering), in order to avoid inconveniencing a few. you have it backwards. all subscribers of the list are 'inconvenienced' if we discourage legitimate contributions from folks who are not willing to jump through arbitrary and time-consuming hoops that we impose on them just because a few people insisted (even in the face of evidence to the contrary) that they knew what was best for everyone else. This assumes that list filtering cannot be done sensibly. This assumption is false; it can be done sensibly and is done sensibly all the time. And when it is done sensibly the amount of inconvenience is unnoticeable. Sure, there are plenty of lists that don't do filtering sensibly (including, alas, some IETF WG lists), but there are many others that do. I also think that list filtering can be done sensibly, and I agree that this is mostly (though not entirely) a matter of resources. what I am objecting to is the notion that 'sensible filtering' (particularly on the IETF list) equates to 'filtering postings from non-subscribers'. calling those hoops a 'minor inconvenience' is also misleading. Only if the lists aren't managed correctly. which is, in my experience, all too often the case. and the knowledge required to 'correctly' manage a list seems to be in short supply. it would be useful to collect such knowledge into an RFC. Keith, I have to say that you are becoming your own worst enemy in this discussion. By insisting on an absolute policy of no filtering at all your ability to influence the policy that eventually is adopted is being compromised. but I have never insisted on such a policy. I have only insisted that it's not appropriate to expect people to subscribe to the list in order to contribute to the discussion. in fact I use various kinds of filtering on the lists that I maintain (different degrees depending on the nature of the list), so I agree that filtering can be useful and appropriate. As a result we are increasingly likely to end up with a list policy imposed that doesn't accomodate some aspect of real world behavior that could have been dealt with. as you might imagine I am also frustrated by the tendency of this kind of debate to polarize people around extreme positions, rather than to encourage brainstorming about solutions that would address the entire spectrum of interests and concerns that are expressed. at the same time, I feel that it's important to argue against proposals for quick fixes that seem shortsighted. we have too many of those already. we need to understand the problem from a variety of perspectives before insisting that our proposed solutions are appropriate to impose on everybody. I also find the comparisons with NAT to be strained at best. I'm sure we can all come up with examples of 'solutions' that served one interest while harming others, or that served short term goals while doing harm in the long term. NATs aren't an especially unusual example of this, they're just an example that can be understood by most of the list. Keith
Re: filtering of mailing lists and NATs
In die Tue, 22 May 2001 12:26:40 -0400 Eric Rosen [EMAIL PROTECTED] scripsit: As has been pointed out, the non-member messages can be moderated. It takes about one second to look at a message and tell whether it is unsolicited commercial or not. but this means - that there is a person who has the right to decide whether the message is spam or not - that this person is willing to bear the burden for the sake of the whole community. I happen to do this for some lists, but it's a nuisance, I may assure you. ciao, .mau.
Re: filtering of mailing lists and NATs
James Aviani wrote: So here is the idea: For email that comes from non-subscribers, forward it to N subscribers randomly selected from the current subscribers. (Maybe pick from the most recent posters, since they are most likely to be active.) If one of subscribers thinks the mail is useful, he forwards it to the group. If more than one approves, still only one copy goes forward. (Software somewhere would prevent duplicates.) Then you have to educate the subscribers on how to approve messages. -- /===\ |John Stracke| http://www.ecal.com |My opinions are my own. | |Chief Scientist |==| |eCal Corp. |All your problems can be solved by not caring!| |[EMAIL PROTECTED]| | \===/
Re: filtering of mailing lists and NATs
I think I might set a filter to look for this thread in the subject line of my email and dump it. It only takes a minute to set it up. __ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/
Re: filtering of mailing lists and NATs
James Aviani wrote: I know this is fairly low-tech, but it seemed like a reasonable and practical solution to spamming. This is a interesting if not good idea. Some of the details may need to be worked out (like perhaps certain people opt in or opt out of being a moderator), but the technical implementation is probably the easy part. If you've given the IETF a solution without causing a theological debate over the 'technical purity' of it, you've left your mark for posterity. John
RE: filtering of mailing lists and NATs
My mail filters must be very effective. The 20 messages on this thread in the last 2 days constitute over a months worth of spam I have been aware of. Now if I could only figure out how to construct an automated filter for: if list = IETF and content = 'personal inconvenience rant' then permanently delete Tony FWIW: I agree with Keith's original tenant that technology applied without my express awareness consent (like NAT in the general case) is inappropriate. I also agree with KRE that spam is a social rather than technical problem, and any centralized technical approach will be worked around. Rather than complain that someone else is not doing the work, maybe those who don't want to take the time to construct their own filters should ask the list if anyone else might be running their favorite mail tool and is willing to share an existing rule set. If you want to make sure I never see your message just include the strings '$' or 'subscribe' in the subject, or send with any of these strings anywhere in the header: @none 163.net 163.com 21cn.edu.cn 263.net 263.com 363.net 363.com auxaux.com china.com sina.com cn99.com com.cn cpri.net dicult.co.jp dta.net.cn elong.com f9.mail.ru fj.cn fj.fz.cn jx.cn Kysi Ferul kyungin-c.ac.kr mediforums.com nbzh.com netease.com xxx@
Re: filtering of mailing lists and NATs
John Stracke [EMAIL PROTECTED] writes: [Randomly selected moderators.] Then you have to educate the subscribers on how to approve messages. Include a short explanation in the message of why it is sent, and offer to follow a URL to approve the message. One of the randomly choosen subscribers presumably knows how to follow a link. -- Stanislav Shalunov http://www.internet2.edu/~shalunov/ This message is designed to be viewed at 600 mph.
Re: filtering of mailing lists and NATs
From: grenville armitage [EMAIL PROTECTED] ... Who knows. I suspect it would be a *vastly* long time before the ratio of 'blocked mailing list' to 'personal email addresses' becomes so high that spammers will special-case their code just to target mailing lists. Today mailing lists are accidental inclusions on spammer master target lists. That last is clearly false for much of the spam that hits IETF lists. At least some spammers evidently already understand that one message through a working and large list will hit a lot of valid addresses, often very well targeted addresses. Besides, mailing list traffic tends to be white listed and so bypass individual spam filters. They already deal with email addresses that get stale and bounce, Serious spammer do not care about stale or bouncing addresses. That's demonstrated by the dictionary attack spammers who have lists of 100's to 1000's of user names that they try at every domain they hit. If you have a vanity domain, then watching for dictionary attack bounces and they wiring those addresses to automated body filters can be very effective measured in low false positives and false negatives. the trick is to convince them our mailing list address is similarly 'stale'. This *is* social engineering, by us, of them, using technology. That assumes that that spammers prune their lists. However, they clearly do not. My best body spam trap address today is a misspelling of my username that first started getting hit several years ago, and that has *never* been valid, and bounced for years until I recently wired it to body filters. The mispelling was apparently a harvesting software bug, because many other people reported seeing equivalent bad addreses. Vernon Schryver[EMAIL PROTECTED]
Re: filtering of mailing lists and NATs
Vernon Schryver wrote: [..] Besides, mailing list traffic tends to be white listed and so bypass individual spam filters. Which is why some of us would encourage the use of techniques that make mailing lists less attractive to opportunistic spammers. I feel dizzy. cheers, gja
Re: filtering of mailing lists and NATs
Maurizio but this means Maurizio - that there is a person who has the right to decide whether the Maurizio message is spam or not Maurizio - that this person is willing to bear the burden for the sake of the Maurizio whole community. Maurizio I happen to do this for some lists, but it's a nuisance, I may Maurizio assure you. I do this for the mailing list of the MPLS working group, so I'm aware of what a nuisance it is. But as far as mailing list management goes, it's not nearly as big a nuisance as trying to figure out which of the error messages to owner-mpls are bogus and which are real. (The mailing list has 3000 members and each message to it results in 100 error messages.) It's not hard to decide whether a particular message is unsolicited commercial email or not, that's not something that people disagree about.
Re: filtering of mailing lists and NATs
From: grenville armitage [EMAIL PROTECTED] Besides, mailing list traffic tends to be white listed and so bypass individual spam filters. Which is why some of us would encourage the use of techniques that make mailing lists less attractive to opportunistic spammers. I feel dizzy. I've run spam filters for a big commercial outfit, where people reasonably preferred to deal with spam than to fail to communicate with customers or prospects. In such situations, unless your false positive rate rejects fewer than 1 legitimate message per month, you should be castigated and your filters turned off. IETF lists have sufficient reasons to be just as open. All of the proposals for filtering the IETF's lists would have false postive rates far worse than 1/month, where delays of more than 24 hours count as a false positive. Because of the nature of the traffic on the main IETF list, I suspect the false positive rate would be approach 10% (except in threads like this where the false negative rate be about 100%, because we're all subscribers making this noise). In other words, there are reasons why I only suggested that the IETF-filtered list use the DCC body filtering. There is something else about the proposals to impose additional filters on this list that really bugs me. I suspect that many of those demanding that this list be filtered did not bother to do anything about the recent spam, while those of us opposed all did do something. There is only one thing that prevents those who want a spam-free IETF list from having it. In theory, someone could subscribe a reflector to the main list or the overseas filtered list, and then run it like an ordinary moderated list. Others who want such filtering could subscribe to it. In theory, everyone would be happy. Unfortunately, there is that one thing preventing global contentment. At least one of those who want such filtering would have to do some extra work. This obviously would not implement or run itself with the demands that someone else take care of it. Vernon Schryver[EMAIL PROTECTED]
filtering of mailing lists and NATs
it occurs to me that most of the methods that have been proposed for filtering spam from mailing lists have a lot in common with NATs. in both cases, the proponents say (in effect) if it works for me and for my small set of test cases, it must be okay to impose this on everyone. if some legitimate traffic is excluded by my filters, they is of no consequence - they should be willing to jump through whatever hoops that I believe are appropriate. and if people have to abandon practices that they find useful in order to to get around my filters, that is of no consequence either, because they do not need to be doing those things anyway Keith
Re: filtering of mailing lists and NATs
Keith Moore wrote: it occurs to me that most of the methods that have been proposed for filtering spam from mailing lists have a lot in common with NATs. actually, have more in common with firewalls. firewalls serve a filtering purpose, and (gasp!) people have learned to configure proxies into their www, etc, clients. we've gotten over it. and so can it be with mailing lists. cheers, gja
Re: filtering of mailing lists and NATs
it occurs to me that most of the methods that have been proposed for filtering spam from mailing lists have a lot in common with NATs. actually, have more in common with firewalls. I beg to differ. People install firewalls to filter their own incoming and/or outgoing traffic. Personally I think firewalls are overrated, but people can filter traffic on their own networks if they want to, using whatever criteria they think best. Those who are complaining about spam on this list have the ability to filter their own incoming traffic. What they're wanting is for someone else to filter their traffic for them, and for everyone else on the list also, using poorly-chosen filtering criteria on which there's no consensus. Keith
Re: filtering of mailing lists and NATs
Aidan Williams wrote: [..] To extend the analogy again in the opposite direction: now that software is available to tunnel random traffic over HTTP, we can expect firewall filtering to get harder, and become less effective. Why would this not happen for email lists too? Most spammers strike me as opportunistic and not overly interested in special-case-handling a couple of subscribe-to-send lists, given the hundreds and thousands of target addresses they purchased on a CDROM. Yes, they could get around pre-subscribe schemes. Yet it seems likely most wouldn't bother, and would instead just end up ignoring us. Which would be nice. cheers, gja
Re: filtering of mailing lists and NATs
Keith Moore wrote: I beg to differ. People install firewalls to filter their own incoming and/or outgoing traffic. D'oh. I thought firewalls where also used to filter traffic one did *not* ask for. Stuff that wasn't apriori declared part of one's community of interest. Seemed a reasonable analogy. [..] Those who are complaining about spam on this list have the ability to filter their own incoming traffic. Kinda, sorta. No reason not to propose schemes that operate closer to the source, even by one hop. What they're wanting is for someone else to filter their traffic for them, Yup. and for everyone else on the list also, Right again. using poorly-chosen filtering criteria on which there's no consensus. It is a fragile universe one inhabits where asking people to subscribe to the community of interest before posting is equated to censorship. If that's what you mean by poorly-chosen then... oh well. Thread fizzles to an end. cheers, gja
Re: filtering of mailing lists and NATs
Keith Moore wrote: however, I have seen a couple of occasions where I believe that a 'moderator' acted inappropriately in filtering messages that came from non-subscribers but were arguably on-topic for the lists. So the non-subscriber subscribed, and their posts went through okay, right? (If not, and the moderator was in fact filtering all posts to the mailing list in question, then this example is a red-herring.) Gas tanks explode - we ban cars? Cigarette butts cause fires - we ban cigarettes? Moderators go loopy sometimes - we ban subscribe-before-post mailing lists? You got me there, guv! I concede. cheers, gja
