from:"Stephen J. Turnbull"

Mark Sapiro writes:

  The footer attached to non-digest messages is in the list's web admin
  interface at Non-digest options - msg_footer and that attached to
  digests is Digest options - digest_footer.
  
  You can set these to anything you want or set them empty to not add
  footers at all.

Ah, Mark forgot to add his usual reminder that empty means not even
whitespace (or did you change that recently, Mark?)  Keep hitting
backspace until you're sure there's nothing left!
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Mailman 2.1.18-1 in cPanel

Russell Clemings writes:

  At least the current release of v11.44, as of last week.

Well, good for them!  That doesn't mean all their clients will
upgrade, of course, but it sure does make it a lot easier.



--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] emails with extension .se rejected

M Winther writes:

  If I use any of my my Swedish email addresses (.se) my messages
  keep bouncing on account of iajsdiscussionlist.org: DNS server
  failure. Yahoo groups intermittently has the same
  problem. However, in this case the messages just disappear. So I
  must use my google email address. I have always had this problem
  with my .se emails. Sometimes when posting to a journal, for
  instance, my message is regarded as spam. I suppose it has to do
  with the fact that .se is not a known American state.

This doesn't sound like it's related to Mailman at all, especially
since other mail-related applications have the same problem on
occasion.  From what you write here, I can only suggest you talk to
the admins of your mail host.

  I have also observed that messages from other contributors never
  arrive in my inbox, although they appear in the archive. But this
  occurs more rarely.

This doesn't either.

  The list software also tends to corrupt messages in the archive by
  inserting digits instead of characters (for citation characters,
  etc.), when the user is *not* using plain text with dumb citation
  signs, etc.

If it's possible to take a look at the archives, please let us know
where to find them.  I suspect that what you are describing is called
HTML entities, which are a way of describing any character in the
world in a way such that any web browser can display it, no matter
what language the computer speaks to its user.  But I can't say
without actually looking at the file.

  I suppose this has to do with the moderator copying a big message
  for the sake of reviewing, and then posting it without regard to
  the format.

That's hard to say.  You'd have to ask the moderator about that. 

  In messages of mine, citation characters (right arrow) are sometimes 
  randomly inserted.

Would that happen to correspond to the case where the word From
appears at the left margin?  If so, that is not Mailman's doing.  It
is done by the mail system somewhere along the line.  Again, a URL to
a corrupt message would make further discussion a lot simpler.

  I use plain text that is line wrapped. Moreover, a drawback is that
  messages cannot be removed afterwards. I don't know how professors,
  concerned about their professional reputation, can tolerate
  this. After all, when partaking in discussion lists, we sometimes
  say things which we later regret, on account of changed views, for
  instance.

The fact is that you cannot reliably retract anything once you've sent
an email.  You can't call it back, and arbitrarily many copies can be
made for free.  Furthermore, the recipient's copy or the archive copy
is just as official as the sender's file copy.  If you want control
over the medium, use a blog.  (Even then people can keep copies, but
at least your copy is the official one.)
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Install from repository, or build from source?

Steve Matzura writes:

  I am a site admin for a system built on Debian version 7 (Wheezy). The
  current available mailman package distribution version is 2.1.15 but I
  want to use 2.1.18-1, which means, unless I miss my guess, it's got to
  be built from source. If this isn't so, I'd greatly appreciate knowing
  where and how to obtain the installable package. I've tried including
  the version number, etc., in apt-get, but as of right now,k no go.

There is no package at Debian yet; even sid is currently on 2.1.16:
https://packages.debian.org/sid/mailman

Source builds aren't that hard, though.  It's also not that hard to
put new source into an old source package and update things, usually.
(Well, to be honest, I haven't done that in at least 10 years!)
Patches can be a little tricky (especially if they've since been
integrated into upstream), but the continuity is often worth it.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] multiple mailing list passwords

2014-05-30 Thread Stephen J. Turnbull

Tanstaafl writes:

  It *does*...

It does??  As you described it, he can let passwordmaker choose his
password.  But he says he can't do that.  Or he can specify the whole
password as the prefix, which is insecure.  And AIUI that's not
acceptable to him either, as far as I can see he's very concerned
about security.  So what's the third option that is both secure and
allows use of the current password?

As far as I can see, you're just saying the requirements are stupid.
I tend to agree (for several reasons), but unfortunately it's also
common that one needs to follow them anyway, and AFAICS the OP is in
that situation.

  now who is missing what?

I could very easily be missing something that nobody has put into
words yet, of course.


--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Logo in posts

2014-05-29 Thread Stephen J. Turnbull

Odhiambo Washington writes:

  I have read the FAQs so I don't believe I missed one, but it could be
  possible. Is there a way, with full personalization, that I can include a
  logo in posts on a list? A remote possibility...

Whose logo?  The list's or the user's?  Where do you want this logo to
appear?  Do you use HTML messages?

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Logo in posts

2014-05-29 Thread Stephen J. Turnbull

Odhiambo Washington writes:

  An organization is 'sponsoring' a list and wanted to have their
  logo included on all messages on the list. I understand that this
  means the list has to somehow modify the message as it leaves going
  to subscribers.

I would think so, yes.

I was thinking you could abuse the X-Face or Face headers, but
apparently they're only supported by a few MUAs popular on Linux.

  As regards convert_html_to_plaintext I can have that disabled if
  it will allow me to include the sponsor's logo somewhere in the
  body of the message, but footer is best option.

Including in the footer is probably not an option for a plain text
message.  It probably would only take a few lines of Python code to
attach the file as a MIME part in a custom Handler, but then the
presentation would be up to the users' MUAs, and it seems likely to be
ugly.

For HTML messages, I think there was some way for Mailman to insert
the footer material in the HTML rather than attach it, but that was
probably a 3rd party patch.  It's also unreliable because you have no
idea what formatting a given MUA may put in its HTML, but you probably
can tune it to give good results for the most popular MUAs used by the
list's posters.

Again for HTML messages, it might be possible to do something tricky
with an IFRAME element to hold the original message, and put the
footer at the bottom of the main BODY.  (The original message would
live in a separate MIME part and the src= ref would use the cid: URI
scheme.)  I think this would work pretty well in most modern browsers,
but I don't know how smart the MUAs would be.  (Completely untested, I
don't even have a proof of concept.)


--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] multiple mailing list passwords

2014-05-29 Thread Stephen J. Turnbull

Tanstaafl writes:

  Chill Stephen.

Chill yourself.  You had two rounds to figure out what he was asking
for, and missed it twice.  Then you tell him he doesn't understand
totally how it works.  I really *was* puzzled as to how you could
think the system you described met his requirements.

Now I think I understand:  To a man with a hammer

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] multiple mailing list passwords

2014-05-28 Thread Stephen J. Turnbull

Tanstaafl writes:

  You can create multiple accounts for the same URL with passwordmaker, so 
  I think you just don';t understand totally how it works.

I have no clue what you're talking about.  The OP shares a password
with several other users, and does not have the right to change it, or
to ask others to change their workflows.  You need to explain how
passwordmaker can do its thing for an *existing* account *and
password*, or it's a non-starter for him.
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] dmarc_moderation_action isn't working

2014-05-28 Thread Stephen J. Turnbull

Mark Sapiro writes:

  If there are no 'DMARC' entries in Mailman's logs, it most likely means
  the imports I show above didn't succeed in the python that Mailman is
  using, in which case dmarc_moderaction_action will not be done at all.

If dmarc_moderation_action is not none (precisely speaking, accept,
I think?), and the dnspython import fails, Mailman should log this.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Yahoo Groups' From munging and X-Original-From

2014-05-26 Thread Stephen J. Turnbull

John Levine writes:

  This is one of the most annoying things about Yahoo and AOL's misuse
  of DMARC -- they're practically forcing people to use hacks to show
  unauthenticated fake From: lines.

Not only that, they're doing it themselves. :-(


--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] [Spam] Re: Digest option for Yahoo and AOL subscribers?

2014-05-26 Thread Stephen J. Turnbull

John Levine writes:

  My understanding is that DMARC WAS going through the standardization
  process, and actually was to the state where experimental use was
  justified (and in some sense actually required). ...
  
  No, not at all.  DMARC was designed and implemented by a small closed
  group of large companies listed on the DMARC web site at
  http://www.dmarc.org/about.html

That's a tiny bit unfair.  Our sister list, mailman-developers, has
been aware of DMARC for at least three years, and our opinions and
even participation have been solicited on at least two occasions.

  The DMARC group has asked the RFC Editor to publish the spec as a
  non-standards-track non-IETF independent submission.  There was
  briefly talk of making it standards track until the DMARC group
  realized that gave the IETF change control, and we likely would change
  it, which they didn't want.  The RFC Editor is currently thinking
  about it, and probably will publish on the theory that even if it's a
  bad idea, it might as well be documented.

If it goes that way, we should write a one-line BCP:

  Best Current Practice for DMARC p=reject

If your name isn't J. P. Morgan, don't.

Well, make that two lines:

AOL and Yahoo!, wake up!  THIS MEANS YOU.

Hm.  AFAICT sec. 3(a)(ii) of Legal Provisions Relating to IETF
Documents allows us to make only the necessary changes to their
document, and submit our version to the standards track.  That might
be amusing! :-)

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Digest option for Yahoo and AOL subscribers?

2014-05-26 Thread Stephen J. Turnbull

Mark Rousell writes:

  It seems to me that if a protocol so easily allows (or even
  effectively encourages) usage that craps on existing legitimate
  Internet usage then the protocol (and its designers) must be in
  part to blame.

I don't see any real difference between ESP abuse of p=reject and
spam itself, though.  They both use others' resources to accomplish
one's own purposes while harming 3rd parties.  as you may know,
well-meaning people have long argued freedom of speech as a moral
justification for spam and Usenet bots and so on.  Well, well-meaning
people are arguing spam-fighting as a moral justification for ESP
(ab)use of p=reject now.  To a yahoo with a hammer, every problem
looks like a thumb.  (With all due disrespect to jwz)

  Oh yes, the protocol has been well designed but it has been well
  designed by its backers who were naturally looking at it *from a
  certain perspective*. The protocol has been well designed to
  achieve certain aims and it is likely to be successful at achieving
  them (including via Yahoo's and AOL's particular implementation,
  inappropriate though it is).

Actually, that's apparently false.  John L linked to or posted a graph
provided by AOL which makes it quite clear that *except for one
particular spammer* DMARC p=reject had *no* effect on spam claiming to
originate from AOL.  It just returned to pre-off-the-charts spamming
level.

It *does* seem to be successful at reducing phishing, for now.
Whether it's reducing damage due to phishing, or just weeding out the
less sophisticated felons, I don't know, and I don't think anybody
does.

  If a perhaps wider range of perspectives had been involved, i.e. if
  it had been developed through IETF, then perhaps misuse/abuse of
  the sort that Yahoo and AOL have demonstrated would have been less
  easy or less tempting for them.

Maybe, but I don't really see that.  As John L points out, at present
DMARC is a private protocol between consenting adults, and even if
the IETF publishes a competing standards track RFC, Yahoo! and AOL can
continue to (ab)use it.

   Yahoo! and AOL simply don't care who
   gets hurt as long as they can present it to their own users as a
   necessary measure to combat spam (and other mail abuse).
  
  Exactly. But they have gone ahead and done it, and they have gone
  ahead and done it because they can

IMO, we could put a period here, because I don't see this:

  and because the protocol as it stands almost encourages (and
  certainly does not discourage) such behaviour.

Well, it's quite clear from the document that DMARC is intended to
protect domain names from being used in phishing attacks.  AOL and
Yahoo! did not (and AFAICS cannot) suffer from severe phishing
problems.  They explicitly refer to their spam problem (which
continues) as justification.  There is nothing that the document
authors can do to stop that (except maybe resign in protest if they
work for such a domain :-).

The fact is that p=reject has been in use at many domains for a long
time with no problem.  The DMARC consortium is surely aware of the bad
effect it would have on reliable delivery to conventionally configured
mailing lists; we've told them often enough, and I doubt we're the
only ones.  Yahoo!'s and AOL's use of p=reject was an act of
desperation AFAICS; even a MUST NOT in an RFC would not have stopped
them.

  If it is true that the designers never foresaw Yahoo's and AOL's
  style of misuse

No, what Murray wrote was that it was understood in the working group
that ESP (ab)use of p=reject was inappropriate, and I understood
that he believed that AOL and Yahoo! were part of that consensus.  He
went on to say later that he didn't have any insight as to why they
went ahead and did it.

  then this seems to me to confirm my point: That a wider range of
  perspectives, which the IETF would hopefully have brought to it,
  might have helped make possible misuses/abuses clear.

We have known for a long time that use by ESPs like GMail (which
hasn't yet), Hotmail (which hasn't yet), Yahoo!, and AOL would cause
lots of problems for their users, and given the stubborn response of
Mailman list operators on this list and mailman-developers, they
surely were well aware that very few lists would be prepared.  So they
went and DoS'ed their own users!  (Of course they also clearly planned
to blame, not the victims, but any innocent bystanders.  Still, they
should have known that their users would get DoS'ed, and they did it
anyway.)

What wasn't known (to me, anyway) was the nasty effect that this would
have on bounce processing.  AFAIK, nobody anticipated that.  I don't
see how broader participation would have helped -- the ranking expert
(Mark, take a bow!) on bounce processing has been aware of DMARC for a
long time.  I doubt that Yahoo! and AOL have the technical abilities
to figure it out for themselves (they don't know how Mailman bounce
processing works).  So I don't think a more IETF-based process would
have

[Mailman-Users] DMARC mung not munging in 2.1.16 (Debian)

2014-05-25 Thread Stephen J. Turnbull

Ron Guerin writes:

  With great sadness, I'm trying to deal with the DMARC problem certain
  providers have decided to create for everyone else, and for some reason,
  even after turning the mung option on in the web interface, there's no
  munging going on. (wrap doesn't wrap either)
  
  I have ALLOW_FROM_IS_LIST = Yes in mm_cfg.py . I have restarted Mailman.

Debian has a habit of moving configuration files around.  On my
Debian system:

-rw-r--r-- 1 root root 4629 Apr 16 05:15 /etc/mailman/mm_cfg.py
lrwxrwxrwx 1 root root   22 Feb  3 22:33 /usr/lib/mailman/Mailman/mm_cfg.py - 
/etc/mailman/mm_cfg.py

Figure out where your mm_cfg.py file or files are, make sure that the
one you're editing is the one that actually controls Mailman.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Digest option for Yahoo and AOL subscribers?

2014-05-25 Thread Stephen J. Turnbull

Richard Damon writes:
  On 5/25/14, 11:30 AM, Mark Rousell wrote:

   Whilst Yahoo and AOL are the ones who have chosen to
   use/misuse/abuse DMARC in this way, it could also be said that
   DMARC (and all its backers on its current form) are to blame
   precisely because DMARC *allows* Yahoo's/AOL's behaviour.

The p=reject policy option is useful, perhaps necessary, to prevent
phishing at financial institutions.  My bank (Tokyo-Mitsubishi-UFJ) is
in a total panic to the point where they are running a major
television campaign (multiple channels, hitting all the major
demographics) displaying a typical MUA (Outlook, of course) showing a
typical phishing message and putting a big red X over the password
input field.

   If the standard has been properly finished and properly thought
   through from all angles then ways could surely have been found to
   allow it to be used without harming existing, standards-compliant
   behaviour.

DMARC's purely informational protocols have been in use successfully
for years, and nobody ever noticed.  Some banks have been using
p=reject for quite a long time (more than a year), and nobody ever
noticed.

   The consortium behind DMARC simply weren't willing to wait or
   play along.

I don't think the evidence supports that belief.  The design of the
protocol has been very careful, with multiple ways to mitigate the
kind of effects we saw in April.  Yahoo! and AOL simply don't care who
gets hurt as long as they can present it to their own users as a
necessary measure to combat spam (and other mail abuse).

  My understanding is that DMARC WAS going through the standardization
  process, and actually was to the state where experimental use was
  justified (and in some sense actually required). The problem that
  happened is that Yahoo jumped into the limited clinical trial and
  experimented with millions before we had a chance to find out the side
  effects of the medicine.

According to one of the editors of the Internet Draft (message to a
closed list), use by ESPs of p=reject was never envisioned by the
working group, and he believed (until it actually happened) that
Yahoo!  and AOL knew that because they have active representatives in
the group.  I'm not sure I really believe that, since one of the DMARC
proponents on Mailman channels clearly believes that any problems are
the fault of misconfigured lists, and one of the editors of the DMARC
Internet Draft has a Yahoo! affiliation listed.

  I suppose that the communities response should have been to just kick
  off all Yahoo (and later AOL) users from mailing list (as that is really
  one meaning of the DMARC setting announced), but the community had too
  much compassion for the innocent users

In many cases, there's no compassion involved, just a hard-headed
business calculation about whether the list can afford to offend the
paying customers.  In any case, it's pretty clear that

  a lot of innocent users ... really don't want to go through the
  hassle of changing email providers, and are more apt to just drop
  off mailing lists.

which both AOL and Yahoo! would find convenient for their own busienss
models.  (I don't think that's their aim, I just don't think they'll
shed any tears as long as their spin control is successful.)

So I certainly don't recommend it if you don't have substantial and
unshakably legitimate influence over your subscribers.

*I* can and do play hardball, and (as mentioned in a previous post)
the fiasco at yahoo.com triggered a reaction in the Japanese research
and education communities (including an official advisory from the
Ministry of Education, Culture, Science and Technology), so that
students and to some extent faculty and researcher have switched to
GMail en masse -- entirely unnecessary since yahoo.co.jp doesn't seem
to publish a DMARC policy at all!

But my situation is very unusual.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Digest option for Yahoo and AOL subscribers?

2014-05-24 Thread Stephen J. Turnbull

Allan Hansen writes:

  I just realized that setting the digest option could be a temporary
  solution for my Yahoo and AOL subscribers

Just make sure you set it for *all* users, not just those using Yahoo!
and AOL.  The important thing is that non-AOL/Yahoo! subscribers be
protected from the denial of service attack that the DMARC p=reject
policy induces.  You must also make it clear that changing their own
setting back is likely to result in them losing posts and getting
booted off the list.

Also be prepared for a spate of posts that break threading because
both subject and reference message IDs refer to the digest message
rather than the actual posts.

Steve



--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Digest option for Yahoo and AOL subscribers?

2014-05-24 Thread Stephen J. Turnbull

Richard Damon writes:

  From what I have seen, any version of Mailman before 2.1.16 (and
  preferably 2.1.18) just isn't compatible with DMARC

Please, it's the other way around! ;-)  DMARC is the interloper, and
some insiders fear that this mess will derail progress to RFC status
of the useful informational part of DMARC as well as the pernicious
p=reject policy.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Who authored the message?

2014-05-20 Thread Stephen J. Turnbull

Natu writes:

  One difference between my method and yours is that my mail logs
  will show that somebody actually replied to that address where as
  with yours the reply would stop at the senders SMTP server.  Not
  that significant, but it might be useful to know if users are using
  those addresses.

If you want that information you might as well run the forwarding
service.  You're imposing what may be significant costs for non-
technical users, who are likely to be confused by the fact that your
server at rjl.com is responding to a message they sent to aol.com.

  I could add something that will make it clear that the addresses is
  not emailable, though my sense is that most users would get that
  the way it is, especially if they tried to email it and it failed.

But the percentage of AOL and Yahoo! users who would understand is
likely to be much lower, if not a small minority.  People use those
services *because* they don't want to learn about email, they just
want it to work.  And the posters are likely to get upset if you make
it hard for them to receive personal replies; a send -- DSN -- resend
cycle may take a long time (especially if the sender has to ask
technical support WTF? :-/ )

I thank my lucky star that almost none of my users use those domains,
and so far 100% of those that do have thanked me for explaining the
problem and switched to posting from GMail or whatever.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Mailman + Exim + strange headers

2014-05-20 Thread Stephen J. Turnbull

Natu writes:

  If there is a dkim signature and it fails google will treat it as
  spam

Note that, taking your words literally, this is against the DKIM RFCs
-- a failed signature is supposed to be treated the same as a lack of
a signature.

That doesn't mean that Google can't or doesn't use it.  In particular,
if the percentage of spam in the failed or no signature class goes
up (which it will if you start signing), your failed or no signature
rating will slump.  That effect might be enough to send unsigned mail
from your site to the spambucket.

But you can be sure that a valid DKIM signature will improve your
site's reputation.


--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Possible to include unique link to archive in each message?

2014-05-19 Thread Stephen J. Turnbull

Mark Rousell writes:

  Can Mailman include the web archive URL for a message in the message
  header itself?

Mailman 2 can't.  Mailman 3 is able to do it, but I'm not sure what
the state of the art on implementing this conveniently for list owers
is.

  As far as I can see, Mailman should have all the info it needs to be
  able to add such a header or footer field.

Nope.  Mailman core doesn't have the sequence number as used in the
archive (it is not necessarily the same as the one that would be put
in the Subject header).  It's the province of the archiver, which is a
separate progrom.  So Mailman has no idea what the sequence number is.
What Mailman 3 provides is a way for the archiver to tell Mailman what
the URL will be.
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Mailman + Exim + strange headers

2014-05-19 Thread Stephen J. Turnbull

Sergio Durigan Junior writes:

  As you can see, the first two Received: headers got messed up somehow,

I don't understand what you mean by messed up.  They looked
perfectly readable and RFC-conforming to me.

  and the lines are not being prefix by \t but by one single space
  char.

This is the RFC 5322 recommendation for folding header fields (use a
space, not a tab).

  use Exim without Mailman, so maybe it is Mailman or some weird scenario
  of Mailman + Exim modifying the e-mails without permission.

Older versions of Mailman (I'm not sure how old) are known to
sometimes modify the headers by unfolding them and then refolding them
when reassembling the message.  This may still be true of the most
recent versions.  I believe this is considered a bug but hard to fix,
and probably not worth fixing for that reason.  Is this actually
causing problems, or is it just ugly?




--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Avoiding DMARC unsubscribes by enabling VERP_PROBES

2014-05-18 Thread Stephen J. Turnbull

Peter Shute writes:

  Why isn't this the default setting? Is there some disadvantage to
  it?

Until now, you only needed it when one of your peers was seriously
broken.  DMARC p=reject now means that AOL and Yahoo! are breaking
other hosts en masse.

Disadvantage, yes.  It requires resources from hosts which aren't
broken, both yours and those of completely innocent third parties
whose only crime is to participate in mailing lists on your host.
It covers up the fact that the messages that bounce are also wasting
your and third parties' resources.  *These* resources are significant,
because they involve cryptographic checks on whole messages, messages
which are typically already bloated by a factor of 5 or so by HTML,
and sometimes much more by cute background images and the like.

We really really want to stop these bounces in their tracks.
*Everybody* does; they do not help anybody.

Regards,

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Dealing with rate limiting from Roadrunner/Time Warner

2014-05-17 Thread Stephen J. Turnbull

Conrad G T Yoder writes:
  On May 15, 2014, at 8:48 PM, Stephen J. Turnbull step...@xemacs.org wrote:

   The log you display is not a true bounce.
  
  Gotcha.  I guess someone thought it was a true bounce and
  configured their servers appropriately. :^) 

Could be.

The other possibility is that you have enough Roadrunner subscribers
that when a post goes out, one connection gets through, a bunch get a
temporary fail and requeued.  4 hours later, your MTA tries a bunch
and one gets through.  A bunch minus 1 get a temporary fail, and
requeued.  8 hours later, your MTA tries a bunch minus 1 and one gets
through.  A bunch minus 2 get a temporary fail, and requeued.  This
happens up to 5 times with approximately exponential backoff.  If the
5th retry fails, it will be considered a permanent failure (by your
MTA).  So if a bunch is bigger than 5, you'll get true bounces.
If any of these retries overlaps with a new post, the problem gets
compounded.  (All the concrete numbers depend on your MTA's
configuration, but they're pretty typical, I believe).

At this point, it should be clear why I consider Roadrunner's policy
to be braindamaged.  They're wasting your resources and DoS'ing their
own users.

BTW, it should be possible to reconfigure your Mailman and MTA so that
Mailman sends the maximum number of addresses per SMTP connection (for
this you need to turn personalization OFF) permitted by Roadrunner in
one shot, and the MTA does as well.  It may be possible to get your
MTA or Mailman to sequence contacts to Roadrunner so that only one
SMTP connection is open at a time.  Maybe Roadrunner will tell you
what the relevant numbers are, or even give you a higher limit.

Nevertheless, as long as they have this policy, the possibility of a
retry cascade exists.  And the numbers that Roadrunner offers may be
much smaller than your MTA's optimal configuration from your point of
view.

  I have not yet tried the nuclear option - the infamous EECB
  (Executive Email Carpet Bomb).

Bu-wha-ha-ha-ha!  But I hope things don't go that far.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Remove user from all list approved senders list

2014-05-17 Thread Stephen J. Turnbull

Peter Shute writes:

  Thanks, found it. I didn't realise there were sub menus for that stuff.

That's not good.  Is there something we could do to help make it
obvious that (many) more settings are accessible?

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Executive summary of DMARC issues

2014-05-16 Thread Stephen J. Turnbull

Gary Algier writes:

  I ran some tests this morning.  I created an Exchange distribution list here 
  and added myself five ways on the list:
  1. On our Exchange server (how I receive internal emails)
  2. On a local Linux server running sendmail and dovecot (how I receive real 
  mail)
  3. A Yahoo address.
  4. A Gmail address.
  5. An iCloud address.
  
  I then sent an email to the list and to my work sendmail address.

Where did you send the mail from, what address was in From, and what
host did the DKIM signing?  Does the domain listed in From have a
DMARC record?

  The DKIM checks seem to be good.  So, it seems that nothing has
  changed in the content or checked header.  It must be SPf.

It could be SPF, but if it is it has nothing to do with DMARC.  DMARC
accepts either SPF or DKIM as evidence of authenticity.  That is,
either may fail as long as at least one succeeds.

If it is indeed SPF, then it doesn't matter what you use.  The problem
is that the host where the distribution list or mailing list is hosted
is not SPF-authorized, and almost certainly not which MTA or MLM you
use.

I'm not sure if you care about DMARC, or just whether it gets
through.  But if the latter, I'm not at all clear on exactly what
you're trying to test.

  % dig +short TXT _spf.mail.yahoo.com
  v=spf1 ptr:yahoo.com ptr:yahoo.net ip4:206.108.40.0/27 ip4:199.16.139.0/26 
  ?all

This is mostly unrelated to Yahoo's behavior when receiving messages.

Amusingly enough, RFC 7208 deprecates the ptr mechanism strongly.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Get newer version of Mailman for Debian 6?

2014-05-15 Thread Stephen J. Turnbull

Sascha Rissel writes:
  Hello there,
  
  I am running a vServer on Debian6.
  Via apt-get install mailman I installed and set up Mailman 2.1.13, which
  is running fine with 5 mailing lists on my server.
  
  Motivated by all those discussions about Yahoo's DMARC on this list, I
  wondered whether I can upgrade my Mailman installation to a newer version.
  
  I tried apt-get update and apt-get upgrade but I did not receive a
  newer version of Mailman.

I think it's probably easier to use aptitude (or some other UI) rather
than apt-get.

A search at Debian (https://www.debian.org/distrib/packages#search_packages)
shows that the current stable version of Debian (wheezy, aka Debian 7)
includes Mailman 2.1.15, which is too old to have any DMARC mitigation
features.  The current unstable version includes 2.1.16, which has
some early attempts at mitigating DMARC, but apparently 2.1.17 and
2.1.18 are not packaged at all.

  Unfortunately I don't consider myself as good enough in Linux
  usage,

In that case I think your best bet is to upgrade Debian to stable
(rather than a specific version of Debian).  Then when they release a
new stable version you will be automatically upgraded.  I personally
run testing (and have used unstable in the past), but it's been
many years since I bothered to fiddle with these settings.  You should
ask on Debian channels how to arrange to get the most recent Mailman
as soon as a package is released.

See also https://www.debian.org/releases/stable/releasenotes for
information on upgrading for your hardware.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Dealing with rate limiting from Roadrunner/Time Warner

2014-05-15 Thread Stephen J. Turnbull

Conrad G T Yoder writes:

   Are these true bounces (ie, permanent delivery failures) or just the
   temporary failures due to rate limiting, causing delays of many hours
   or days in delivery?
  
  It is a true bounce - mail is being rejected.  The error message is
  phrased as a temporary failure, but the message is bounced at the
  SMTP transaction.  “At some point,” they stop bouncing.  RR won’t
  say when that happens.

The log you display is not a true bounce.  If your host's MTA is
configured properly, you may get DSNs saying that you don't need to do
anything but wait, but your users' bounce counts are not increasing
due to messages like the one below, and they probably are eventually
being delivered (although that can't be determined from the log below).

  u...@roadrunner.com: delivery temporarily suspended: host
 cdptpa-pub-iedge-vip.email.rr.com[107.14.166.70] refused to talk to me: 
  421
 4.7.1 - Connection refused - 66.33.216.56 -  Too many concurrent
 connections ( 2 ) from source IP

This is absolutely clearly with no doubt whatsoever a problem that
only can be resolved by reconfiguring the MTA at one end or the other.
I gather that your MTA is a DreamHost responsibility, so they are
either amazingly incompetent or lying to you.  I don't blame them for
not wanting to deal with it, by the way, only for trying to blame the
victims (you and your Roadrunner subscribers).  Roadrunner's policy is
brain-damaged, DreamHost's is merely pragmatic (it's very costly to
try to work around a peer site's brain damage).

The only effective measures you can take that I can see based on what
you have written so far are (1) find a competent and responsive host
for your list (there may not be any, though; I'm not sure what to do
to get reliable timely delivery to a system like Roadrunner), or
(2) tell your Roadrunner subscribers that their host is incapable of
supporting reliable mail delivery and that you can't do anything; if
they care about reliable delivery of mailing lists including yours,
they should use an address at a competent provider (GMail is the only
competent and approximately socially responsible freemail provider I
know of).

   This may be partly true, if you are falling afoul of the too many
   recipients per message limit.
  
  I will probably turn that on soon - it is available to me.

This probably won't help, because based on the message above your MTA
will helpfully just execute the deliveries in parallel, and you'll get
*more* delivery failures, not less.

  On this instance, they haven’t been very helpful.  For the most
  part, though, they have been decent.  The largest of my lists that
  I admin has over 7600 subscribers, with about 15 emails going out
  daily - that’s about 115K total emails a day, and for the price I
  am paying DH, I haven’t found anywhere near a better deal.  So I
  accept these (occasional) issues along with the deal I’m getting.

This issue doesn't look occasional to me.  However, if what is most
likely a permanent inability to achieve timely delivery to subscribers
at Roadrunner is acceptable to you, I don't have any complaints. :-/

Steve


--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] A word to the wise: spam-checking and RFC conformance

Hi all,

I just discovered that the spam-checker for a non-Mailman list I
subscribe to (I suspect SpamAssassin but I can't confirm yet, the MLM
is ListServ) is introducing 8-bit characters into the header.

What appears to be happening is that the original post uses a
Content-Transfer-Encoding of 8bit or binary, and contains some 8-bit
octets (typically fancy quotes and the occasional accented character
in the posts I receive), and the spam checker copies an excerpt from
the body into the X-Spam-Report field *verbatim*, including the
non-ASCII characters.

If your spam-checker doesn't produce an X-Spam-Report field or
similar, you don't have the problem.  If it does, you might want to
check its behavior with respect to body excerpts.

Regards,
Steve

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Executive summary of DMARC issues

Gary Algier writes:

  I have been following the discussion of the DMARC issues and Mailman's 
  attempts to live with it.  I was wondering if anyone has an Executive 
  Summary of the DMARC issue in a general sense.

How about the following:

DMARC is a set of protocols for Internet mail that are used to by
participating hosts to exchange information about mail traffic.  This
information is primarily useful for fighting email fraud, including
spam and phishing.  The basic procedure is that recipient hosts which
participate in the protocol check whether the apparent sending domain
(the domain that appears in the address in From header field)
participates, using a query to the DNS defined by the DMARC protocol.
If so, the recipient host will send to that domain some information
about mail which purports to be from the sending host, but fails to
validate.

The validation refers specifically to, and only to, the domain in the
From header field.  The DMARC protocol itself does not validate the
mailbox (user), although some originating hosts may do so.  Nor can it
guarantee that the user's account hasn't been used fraudulently.

For participants in the DMARC protocol, valid mail provides a strong
guarantee that the domain of the address in the From header field
is authentic, with some strong restrictions required to ensure success
of validation.  These restrictions include:

(1) When authentic mail is delivered directly from the sending host to
the recipient, validation will succeed.

(2) When authentic mail is forwarded verbatim by all intervening
Internet hosts (including mailing lists and user mailbox
forwarding), validation will succeed.

Authentic mail may fail to validate in many common circumstances,
where a forwarding host alters certain headers or any part of the body
of the message.  This may occur for forwarding hosts which add
advertisements to the footer or the like, or if a forwarding host
reformats the body (it shouldn't do that, but some do).  There are
also restrictions on some parts of the mail.  For example, the From
header field must contain exactly one address (multiple author
addresses are permitted by the mail standards and are occasionally
useful, though rarely seen nowadays).

Failure to validate is also frequent on mailing lists.  Common
transformations that *will* cause mail distributed by a mailing list
to fail validation include (but are not limited to)

- Adding a list tag or sequence number to the Subject header field.

- Adding a footer, typically containing unsubscribe information for
  public discussion lists or legal disclaimers for corporate lists.

- Removal of banned content, such as executable file attachments or
  excessively large attachments.

Because many common practices with Internet mail can cause authentic
mail to fail DMARC validation, although mail which successfully
validates may be trusted to be from the domain in the address in the
From field, failure to validate in general *does not* imply lack of
authenticity.

Under some circumstances, a sending domain may be willing to severely
restrict its users' use of their addresses.  For example, they may not
be allowed to post to mailing lists, and recipients may be advised to
*always* keep their addresses updated to the domain where they
actually read their mail.  This is common practice with banks and
other financial institutions.  Such organizations can be reasonably
sure that mail which fails to validate is fraudulent (although the
intent may be merely mischievious rather than truly felonious).  They
may take advantage of a further feature of the DMARC protocol, the
policy, which is the action that the sending domain suggests that
recipients take with mail that appears to be from that domain but
fails validation.

The normal policy is none, that is, the recipient should deliver the
mail as though the apparent sending domain did not participate in
DMARC.  (Of course, if the mail fails spam or virus checks, it would
be quarantined or discarded as usual.)  The second level is
quarantine, which means that the mail should not be delivered
directly to the user's mailbox in the ordinary way, for fear of
fraudulent use of the sending address (typically phishing).  The third
level is reject, which means that the sender recommends that the
mail not be delivered at all, to completely prevent fraudulent misuse
of their domain.

Unfortunately, unless the sending domain has truly draconian policies
about the use of addresses in that domain, use of quarantine or
reject makes it virtually certain that some authentic mail will not
be read in a timely fashion or (in the case of reject) never
delivered at all.

The reject policy may also result in denial of services to third
parties (one common case is having delivery disabled from a mailing
list, or even being unsubscribed), due to rejected mail being
bounced back to the forwarding host.  At present we know of *no*
hosts which distinguish DMARC bounces from other

Re: [Mailman-Users] Executive summary of DMARC issues

Barry S. Finkel writes:

  Is this also true?
  
  Users from DMARC-reject domains send mail to mailing lists, and the
  resulting mail from the mailing list is rejected.  Enough
  rejections can cause the mailing list possibly to be blacklisted
  for sending lots of spam mail.

The rejections themselves will be received by the mailing list
(assuming RFC-conforming recipients), so I think this is unlikely in
theory.  I haven't heard of this happening, and Mark says he hasn't
either.

The failure notices received by DMARC senders are another matter.
These are not supposed to be used for blacklisting, but as we already
know, Yahoo! and AOL are desperate.  OTOH, they're already known to be
hostile to mailing lists; getting blacklisted by them is not news.  I
suppose we could worry that they would publish their blacklists to be
used by others, but it was pointed out to me by Murray Kucherawy that
MAPS was shut down by lawsuits, and it seems likely that a public
whitelist or blacklist would attract them too.

My estimate is that this is not something to worry about because it
will be mitigated by any of the options already implemented, and it's
not something to complain about because there's no evidence it
actually happens.  IMO, we need to avoid crying wolf; there are way
too many people (including a number of Mailman list operators) with a
vested interest in painting Yahoo! and AOL as well-intentioned but
overwhelmed by circumstances beyond their control.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Executive summary of DMARC issues

Peter Shute writes:

  When MS365 forwards the mails sent to the distribution list, should
  that make the DMARC authentication fail? I thought that only
  happened if you made changes like adding a prefix to the subject
  line like Mailman does.

If it forwards verbatim *and* the sending domain signs the mail with
DKIM (the common case), DMARC validation will succeed.  Without DKIM,
DMARC validation is guaranteed to fail.  However, even in the sender
uses DKIM, *any* change *whatsoever* to the body will cause validation
to fail, and there are several changes to the header that could cause
it to fail.  Furthermore, which parts of the header are protected by
the DKIM signature are determined by the sender, not by DMARC AFAIK.

If distribution lists are pure forwards, MS365 will be OK.  But I find
it hard to believe that that level of functionality is popular with
users -- there's a reason why all popular MLMs implement subject
prefixes, body headers and body footers, and it isn't because it's
the Microsoft way.


--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Dealing with rate limiting from Roadrunner/Time Warner

Conrad G T Yoder writes:

  Has anyone had to deal with bounces due to rate limiting from
  Roadrunner/Time Warner?

Are these true bounces (ie, permanent delivery failures) or just the
temporary failures due to rate limiting, causing delays of many hours
or days in delivery?

  http://postmaster.rr.com/spam#ratelimit
  
  If so, what did you do to resolve the problem?  (That is, if things
  “got resolved.”)

Haven't had a problem (RR is on the list of if you don't like losing
mail, don't do that mailboxes for my lists).

  My hosting provider, DreamHost, continues to assert this is my
  problem and not theirs.

This may be partly true, if you are falling afoul of the too many
recipients per message limit.  In that case, if the option is
available, then you should set personalization on for lists will more
than a very few RR subscribers.  If not, see full of below.

  I of course do not have control of the Mailman installation and/or
  the server.

NightmareHost is full of s--t.  The fundamental problem is RR's
policy.  They implicitly say we don't care if our users get their
mail,[1] and they reserve the right to arbitrarily refuse delivery no
matter what you do.  That may be your problem, but there's nothing
you can do about it.

It's true that you *may* have a mitigation option in personalization,
but that is under control of NightmareHost.  If they do not permit you
to personalize, there's nothing under your control that you can do.
All the other mitigation strategies (whitelisting, feedback loop)
involve the participation of NightmareHost AFAICS, since RR rate
limits on the basis of IP, which is owned by NightmareHost, I assume,
and the limit on recipients per SMTP connection is in mm_cfg.py, which
is not under your control at all.

Does NightmareHost offer you any advice?  Or are they just saying if
you don't like our service, find another one?  I think find another
one is a great idea, personally.

Sorry I can't be more optimistic.


Footnotes: 
[1]  This is generally true of the big services.  They deliberately
accept the tradeoff of unreliable delivery of desired mail in return
for less spam.  They prefer not to spin it that way, but that's what
they do.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Reply-To lines with from_is_list munging.

2014-05-13 Thread Stephen J. Turnbull

Mark Sapiro writes:
  On 05/12/2014 01:25 AM, Stephen J. Turnbull wrote:
   
   How about multipart/alternative:
   
   message header
   multipart/alternative
   
   part header
   message/rfc822# original message in all its glory
   
   part header
   traditional cooked list message
  
  
  Interesting idea, but I think the part order is reversed. The simplest,
  most universally readable part is supposed to be first with parts of
  increasing complexity coming later.

That's precisely the point.  Most MUAs choose to display the *last*
form that they understand, but there's no guarantee that they'll
understand earlier ones, so they should (but see below) keep trying.

As Bugs Bunny says, Eh-he-he-eh, ain' I a stinka?! ;-)

   Then amend the existing MIME RFCs to say that MUAs SHOULD (MAY?)
   simply display the original message in some appropriate way.  No?
  
  I really wonder if that would help. Section 5.2 of RFC 2046 [...].
  While this doesn't explicitly say MUAs SHOULD or MAY simply display the
  original message in some appropriate way, it certainly conveys that
  sentiment to me, yet here we are over 17 years later with apparently
  some mainstream MUAs that don't do that.

I know, but what can we do?  There are very few of us who could get
away with telling our subscribers, well, then, get a *real* MUA!!,
and even fewer who can do that, and want to.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Can Someone Explain This?

sherwin writes:

  mid...@lists.ibiblio.org: Command died with status 2:
   /usr/lib/mailman/mail/mailman post midfex. Command output: Group 
  mismatch
   error.  Mailman expected the mail wrapper script to be executed as group
   mailman, but the system's mail server executed the mail script as 
  group
   nobody.  Try tweaking the mail server to run the script as group
   mailman, or re-run configure,  providing the command line option
   `--with-mail-gid=nobody'.

I would bet a week's worth of beer that the Mailman installation has
been upgraded recently, and they forgot to give the wrapper program
sgid privilege and/or forgot to give it group mailman.

It's also possible that the actual problem is with the mail server,
but usually mail servers are not given sufficient permissions to run
other programs as arbitrary users or groups.

To fix it, run the bin/check_perms program in the Mailman installation
to see if I'm correct.  If I am, run bin/check_perms -f to actually
make the necessary changes.

Of course you need to have root permission on the Mailman host to do
this.
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Reply-To lines with from_is_list munging.

Lindsay Haisley writes:

  What goes into an address comment is, or should be, purely
  informational on a human level, and ignored on a computational
  level.

Unfortunately, we can't depend on that:

   There are a few possible mechanisms that attempt mitigation of
   [display name] attacks, such as:

   o  If the display name is found to include an email address (as
  specified in [MAIL]), execute the DMARC mechanism on the domain
  name found there rather than the domain name discovered
  originally.

DMARC draft, sec. 15.2.  This is discussion of matters outside the
scope of DMARC itself, not a normative specification, and the document
itself says there are legitimate uses of email addresses in display
names (or comments).  But that hasn't stopped the spam-fighters in the
past; it may not stop them this time.  AFAICS, putting an address from
a DMARC domain anywhere in the mail leaves you subject to a possible
DMARC reject unless you satisfy from alignment for that domain
exactly as specified in DMARC.

That's not implemented by anyone now, and may never be.  And
obfuscating the address as in the OP may help, but for my previous
work address that would be

stephen dot turnbull dot 1 at econ dot ohio-state dot edu

which is 57 characters.  You pays your money and you takes your
choice, I guess.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Subscription flood

Mark Sapiro writes:

  They probably aren't using the subscribe form on the listinfo page but
  rather posting the data directly to the subscribe CGI. Try moving
  mailman's cgi-bin/subscribe aside to totally disable web subscribe.

Yeah, this seems like a different attack from the last one I heard
about (a CGI on a 3rd party site that would sign the victim up for
about 400 *different* MLs), but that one also hit the subscribe URL
directly.

How hard would it be to use security-by-obscurity, ie, to just move
the subscribe URL to a different location and change the links on the
subscribe pages?
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Reply-To lines with from_is_list munging.

Lindsay Haisley writes:

  A nice fix, albeit probably total pie-in-the-sky, would be the
  establishment of a MIME Content-Type: multipart/list-post, a variation
  on (or extension of) mulpart/mixed.  MUAs SHOULD (in the RFC 2119 sense)
  effectively hide the outermost enclosing MIME envelope with this
  Content-Type and present the contents according to rules that would
  apply were the enclosing MIME envelope not there.  As far as the mail
  system is concerned, the headers on the envelope are the effective ones.
  As far as the MUA is concerned, for presentation purposes, the envelope
  content is what counts.

The problem is that the DMARC people don't give a damn about the mail
system (and the PHBs behind the actions at Yahoo and AOL could care
less in both senses, apparently).  They're entirely concerned with
presentation.

And the technicians who designed DMARC are *right* to be concerned
about presentation, because it is presentation that the crooks use to
hook their prey.  In other words, if we come up with a way to present
mail that doesn't bear their signature[1] as if it came straight
from one of their domains, that can be abused by the crooks.

When (not if!) that abuse happens, the forces behind DMARC will come
back and say O no!  You can't do THAT!  And they (the PHBs,
I mean) will break the system again ... and again ... and again.

So, unfortunately, I think there is *no* fix based on presentation.
The only real fix is users who are sophisticated enough to avoid
spammers, which can't be perfect (some people just aren't, and
everybody slips occasionally), but can certainly be enhanced by better
filters.

Well, there's that other fix, the one that involves lists as we love
them joining the dinosaurs. :-(

All-hail-Dave-Hayes-and-the-AI-newsreader!-ly y'rs,



Footnotes: 
[1]  Any list that isn't a pure address exploder will be unable to
maintain the signature.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Results of testing posts to yahoogroups from AOL

Mark Sapiro writes:

  I finally got around to testing this.

Thanks, Mark!

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Reply-To lines with from_is_list munging.

Richard Damon writes:
  On 5/9/14, 10:13 PM, John Levine wrote:

   The correct response is either for senders to stop publishing DMARC
   policies that don't match the way their users use mail (fat chance),
   or for recipient systems to skip the DMARC checks on mail from sources
   that are known to send mail that recipients want but that doesn't
   match DMARC's narrow authentication model, e.g., mailing lists and the
   Wall Street Journal's mail-an-article button.

GMail is already doing this, although we don't know the algorithm
precisely.  If GMail continues and others join, ostracism of providers
who continue to use inflexible bouncing policies instead of smart
filters becomes more plausible.

I know that's not satisfactory for people whose lists are populated by
AOL and Yahoo users, but I don't know what to say to them.  Their
users are DoS'ing their mailing lists with their addresses, even if
they don't know it.

  But the wrapped message could pass the DMARC DKIM signature check, if it
  will exactly matchs the message that came from Yahoo/AOL. (which the
  phish won't). This says that the List Headers, modified subject, list
  headers and footers should be added to the wrapping message, not the
  wrapped message, which also says that the MUA shouldn't throw this away,
  but combine these with the original message (but in a way that makes it
  clear which is which).

Sure (and that is what I intended when I suggested wrapping in the
first place), but (a) MUAs don't support DMARC yet, and all the signs
say that the yahoos will deliberately delay implementing MUAs that do,
and (b) many MUAs don't support wrapped messages well at all.

As John put it,

  Failing that, all we have left is hacks, none of which are
  satisfactory.

We'll see how the on-going talks at the IETF go.  Some results should
be forthcoming shortly (that's hearsay, and I can't say any more
because that's exactly what I was told by a source close to the center
of the process).

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Reply-To lines with from_is_list munging.

2014-05-08 Thread Stephen J. Turnbull

Joseph Brennan writes:
  
  Stephen J. Turnbull step...@xemacs.org wrote:
  
 Honestly, they (one of the principal DMARC spec authors works for
 Yahoo) ignored their own advice, imagine how well that would go
 over in some other industries.

I didn't write that, and I dissent from the implied sentiment.

Cheers

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Reply-To lines with from_is_list munging.

2014-05-08 Thread Stephen J. Turnbull

Glenn Sieb writes:

  Then please work on your phrasing.

That times time and effort, which I will start saving right now.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Add PayPal to DNs publishing DMARC p=reject

Peter Shute writes:
  
  Stephen J. Turnbull wrote:
  
   The DMARC WG advocates putting list-post in From in place 
   of a DMARC p=reject address.  I advocate accepting their 
   advice for stock Mailman, and avoiding other non-conforming 
   workarounds until the market demands them.  If it gets noisy, 
   feel free to cave in faster than you did on Reply-To munging.wink /
  
  Can you explain that for the uneducated, please?

Ouch!  Sorry for the tech talk, often it's a useful habit, but not
always.

  What do you mean by list-post? Is that the list address?

There are several addresses that Mailman uses that might plausibly be
called the list address.  The one you are thinking of is often
called List-Post because there is a header, hidden by most mail
clients, by that name, to allow mail clients to automatically
recognize the posting address (some provide a separate command for
reply-to-list).  It is the address where members send posts.

But there's also the list owner's address (one might think of that as
headquarters, and therefore the list address).

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Add PayPal to DNs publishing DMARC p=reject

Peter Shute writes:

  Thanks, I understand now. If the result of this is that replies go
  to everyone on the list, this is something we don't want for our
  list. Private replies becoming public means trouble, and we have
  enough of it already when people Reply All by accident.

In that case, in Mailman 2.1.18-1, you probably get the best of all
worlds by setting

'from_is_list' to 'Munge From'

which puts the list in From, deleting any other addresses from
From (thus disabling DMARC), and then puts the poster in Reply-To,

'reply_to_list' to 'Poster'

which leaves the Reply-To header as it finds it.  Finally, set

'personalize' to 'Full Personalization'

which puts the recipient in To.  The first two are on the General
Options page, the last on the Nondigest Options page.

The rules for these options are complicated, but if I've thought
correctly about this, in most cases the header of the post as
distributed to subscribers will say

To: each-subscriber@home
From: the-list@your-org
Reply-To: the-poster@home

Although the-list is *visible* in From, conforming mail clients
will *not* pay attention to it (the rules say Reply-To takes
precedence over From as the author's address), and even a Reply All
will produce a message addressed as

To: the-poster@home
From: each-subscriber@home

In order to also CC the list, the replying subscriber would have to
deliberately copy/paste the list address into To, Cc, or Bcc.
This depends on the replying subscriber's mail program, so there are
no guarantees, but it seems very unlikely to me that any of your
subscribers will inadvertantly CC the list with that configuration.

The only downsides are that (1) the list appears to claims to be
authoring all the posts, and send each privately to each subscriber
(but I wouldn't be surprised if few subscribers notice more than
something changed) and (2) full personalization uses more resources,
potentially a lot more.  On the other hand, with reasonably modern
equipment and say 5 lists each with 500 subscribers and 10 posts each
per day, the server will literally spend more time waiting for the
next post than it does delivering them.

Network bandwidth is a more important consideration, because if you
have many subscribers at one domain, you can tell that domain to
deliver to a long list of those subscribers, and then send the message
once.  But if you personalize, then each message is (slightly)
different, and must be sent separately.  If you want advice about
resource usage in your situation, don't hesitate to ask here.  I have
no experience with that configuration, but I suspect Mark has the
numbers on tap, and I'm sure many of our lurkers do.

Hope this helps,

Steve

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Add PayPal to DNs publishing DMARC p=reject

Rob Lingelbach writes:

  Is it possible the ‘personalize’ option moved elsewhere in
  2.1.18-1?  I’ve just updated to that version and don’t see it on
  the Nondigest Options page.

Sorry, I haven't updated to 2.1.18-1 yet, I'm reading source and
missed a crucial qualification at the top of the suite.

Because personalization can consume a lot of resources, the site admin
needs to enable personalization with OWNERS_CAN_ENABLE_PERSONALIZATION
in mm_cfg.py, then it will show up on the admin site.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Reply-To lines with from_is_list munging.

Glenn Sieb writes:

  What my list owners want out of my lists, and what rules they
  decide on for their lists, is not my business. By extension, it is
  not yours.

If you just want to vent, please say so.  I thought you were asking
for help.

If you want help, then the questions I asked are essential to doing a
good job for your list owners.  There are two reasons for that.

(1) Users often request a feature that they believe accomplishes a
certain goal, but it does not.  All too often implementing that
feature does not satisfy their need, with attendant frustration
all around.  Letting the developer design the feature to achieve
the goal often (although not always) does a better job of
satisfying needs.

(2) Often either the current implementation of the program or the
nature of the world means that not all needs can be satisfied, and
a compromise must be suggested.  Knowing the goals (reasons why)
can help the designer suggest a better (accomplishes more goals
more fully) or more palatable (emphasizes more important goals)
compromise.

See my dialog with Peter Shute for an example of how such design can
succeed.  It's rare that we get 95%[1] success that way because of (2),
but my lack of understanding of his lists' requirements displayed at
the start is the usual case.

  I'm just trying to see if there are better options out there.

And I'm just trying to understand what better means to you and your
list owners and subscribers.


Footnotes: 
[1]  Not yet 100% success because of the increased resource
requirement, which can be a blocker.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Reply-To lines with from_is_list munging.

Jim Popovitch writes:
  On Wed, May 7, 2014 at 6:47 PM, Mark Sapiro m...@msapiro.net wrote:
   We are trying to talk with DMARC proponents,
  
  You won't be successful until those people themselves figure out what
  they are doing

That's true, but those folks (or, more accurately, their bosses) have
their shorts in a knot over the recent attacks.  I don't have a lot of
sympathy for the corporations which have a long history of half-baked
implementations, but our best bet is to help them figure it out.

  (and then they agree to quit using the Internet as a testbed)  :-)

But there is no other.  I can't really blame them for eventually going
live, I just wish they tried harder to work and play well with others.

  Honestly, they (one of the principal DMARC spec authors works for
  Yahoo) ignored their own advice, imagine how well that would go
  over in some other industries.

Happens all the time.  Ford Pinto gas tanks, space shuttle O-rings,
the list goes on.  Let's have some perspective: nobody died this time.
And I doubt the principal authors ignored their own advice; some PHB
did it.
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Add PayPal to DNs publishing DMARC p=reject

Peter Shute writes:

  So does this mean that any solution is going to be a choice between
  ease of replying to the list and ease of accidental replying to the
  list?

Yes, and that's an unsolvable problem.  Some replies should be public,
some should be private, and only the user can know which is which.  We
can bias things one way or the other, but we can't really do much on
the list side to improve accuracy of addressing.

MUAs could help a bit more than they do, but they're just programs,
too.  In the end, you have to assume the user knows what she's doing,
and that isn't always true.



--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Add PayPal to DNs publishing DMARC p=reject

Peter Shute writes:

  If it means that Reply vs Reply All work differently for list
  messages from different domains,

It does.

  will it only lead to users becoming hopelessly confused? Is there
  anyone who's already using this who could report on the reactions
  from users?

Good question.  Anybody?
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Add PayPal to DNs publishing DMARC p=reject

2014-05-06 Thread Stephen J. Turnbull

Barry Warsaw writes:
  On May 06, 2014, at 02:15 PM, Stephen J. Turnbull wrote:
  
  No, the point is that a phishing mail with
  
  From: Chase Bank Customer Service serv...@chase.com.invalid
  
  will sail right past DMARC, as currently set up.
  
  So too will serv...@chase.com.ru without Mailman ever getting
  involved, and I bet that will be just as effective at phishing as
  .invalid.

Et tu, FLUFL?

The point is that if Mailman provides this, it becomes a standard
way to get a DMARC p=reject address past DMARC p=reject, and people
*may* develop an it may say .INVALID, but it's OK reflex.

As I wrote to John Levine on mailman-developers, if operators want to
experiment with it, that's one thing.  But does *Mailman* want to take
part in encouraging that it's OK *because* it's .INVALID meme?  Do
we want to encourage phishers to use something that looks like a
Mailman feature, and have the DMARC WG come back with something that
involves anything that looks like my domain?

The DMARC WG advocates putting list-post in From in place of a DMARC
p=reject address.  I advocate accepting their advice for stock Mailman,
and avoiding other non-conforming workarounds until the market demands
them.  If it gets noisy, feel free to cave in faster than you did on
Reply-To munging.wink /

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] DMARC and Reply-To lines with from_is_list munging.

2014-05-06 Thread Stephen J. Turnbull

Glenn Sieb writes:

  So I updated to 2.1.18-1 today. Now we have a Reply-To that has the
  poster's email and the list's email address.
  
  A few of the lists I run block emails with more than one recipient, so
  now this is going to be an adventure. (Ok, more like a nightmare, as
  right now it appears my choices are make reply-to only the list
  (anonymous_list) or make reply-to the poster and the list.)

What is the intent of the restriction?  Are you trying to get the
users to use reply to author by punishing them with a black hole if
they don't, and then set Reply-To to list-post so that nobody ever
gets a personal reply?  Or is this intended to prevent people from
including 3rd parties in the OP (of course, you can't -- they can
always BCC and you'll never know)?

I suppose your users would get upset if you used
dmarc_moderation_action = 'Wrap Message' instead of whichever_option =
'Mung From'?

Given Mark's reply, probably you'll need use a custom Handler,
whatever the requirements.  Is that acceptable (ie, you have the
necessary accesses)?  N.B. It's possible to restrict use of Handlers
to particular lists by giving them list-specific pipelines.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Add PayPal to DNs publishing DMARC p=reject

2014-05-05 Thread Stephen J. Turnbull

Lindsay Haisley writes:

  $ dig +short -t txt _dmarc.paypal.com
  v=DMARC1\; p=reject\; rua=mailto:d...@rua.agari.com\; 
  ruf=mailto:d...@bounce.paypal.com,mailto:d...@ruf.agari.com;
  
  This probably is a problem of lesser magnitude than Yahoo! and AOL

FWIW, I don't consider it a problem at all (most definitely YMMV, of
course).  I think this is what DMARC *should* be used for.

My interpretation is that this is a particular author (a corporate
one) allowing her MTA to digitally sign her mail, and soliciting the
help of email for those who can't implement Diffie-Hellman off the
top of their heads email providers' MTAs in the effort to protect the
author's customers from 3rd party fraud.

I don't know what paypal-inc.com is for, so I can't speak to that
one.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Add PayPal to DNs publishing DMARC p=reject

2014-05-05 Thread Stephen J. Turnbull

Peter Shute writes:

  How does Yahoo's DMARC policy reduce the benefit of Paypal's?
  Because servers can't follow the reject recommendation without

No, it's because users get used to ignoring warnings about DMARC
issues.  If it was *only* your bank, you'd learn to pay attention to
them.  But when you (FVO you susceptible to phishing in the first
place, of course!) see a pile of DMARC workarounds every day for 70%
of your correspondents, how do you respond to this?

All of our mail to you have come back to us due to DMARC rejects,
so we need to use this unusual address.

Please confirm your blah-blah-blah by clicking here and logging
in to our secure site.

2% of AOL customers will respond by clicking, at last report. :-(

Let's put it this way: When was the last time you saw an unvalidated
SSL certificate?  Is that timestamp equal to the last time you
followed up by checking the root cert's fingerprint on the authority's
secure site?  Or is the latter equal to -1? ;-)

  And does the emergence of legitimate p=reject policies mean it's
  now less likely Yahoo and AOL will back down?

What makes you think the banks didn't start doing this ages ago?
Apparently they merely haven't made an explicit announcement.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Add PayPal to DNs publishing DMARC p=reject

2014-05-05 Thread Stephen J. Turnbull

Peter Shute writes:
   On 5 May 2014, at 4:59 pm, Stephen J. Turnbull step...@xemacs.org 
   wrote:

   them.  But when you (FVO you susceptible to phishing in the first

  Sorry, what does FVO stand for?

Ah, excuse my abbreviations.  FVO = for values of; the intended
implication is that the you reading my post isn't the kind of you
who gets taken in by phishing emails.

  All of our mail to you have come back to us due to DMARC rejects,
  so we need to use this unusual address.
   
  Please confirm your blah-blah-blah by clicking here and logging
  in to our secure site.
   
   2% of AOL customers will respond by clicking, at last report. :-(
  
  They get a warning? I thought it just bounced, and the intended
  recipient never knew.

No, the point is that a phishing mail with

From: Chase Bank Customer Service serv...@chase.com.invalid

will sail right past DMARC, as currently set up.  In the message, the
complaint about the DMARC rejects was written by the phisherman, and
the strange address is explained by that preamble.  Thus reassured,
the victim then clicks.  Don't ask me to explain why they do that, I
don't really understand (I'm almost tempted to quote Niven and
Pournelle, think of it as evolution in action), but it's an
empirical fact that real people lose real money to these scams (2% of
AOLers click, according to AOL).

Now, it's *possible* that .invalid will trigger the latent common
sense in the 2%.  But I think that pretty unlikely to be completely
effective, and I suspect it won't be effective at all in the presence
of a disclaimer about the unusual address.  If .invalid can't
get by the victim's common sense, .REMOVE-THIS etc probably will.

The thing is that a bit of common sense will save you from any of
these scams.  But that's not enough to create good policies, because
it's very hard is to think of all the ways to abuse a very naive
victim, or a very young one, or an elderly one who's lost a step
mentally -- it takes a devious mind just to think of one!

Regards,

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] yahoo.com.INVALID as a DMARC defense [was: 2.1.18 internal documentation suggestions]

2014-05-03 Thread Stephen J. Turnbull

Andrew Partan writes:

  Until people figure out real ways of making DMARC work with forwrders
   mailing lists (see ietf-...@ietf.org for one place discussions
  are going on), I think it useful to have more work-around hacks out
  there so that people can experiment with them to see which ones
  more-or-less work in different situations.

That's what they said about Reply-To munging, too.

If people want to implement them themselves and try them out, heck,
I've been wrong before and I'll be wrong again.  But I don't think
*Mailman* should proactively implement RFC violations unless there's a
clear and present danger, and then the violations should try to be
minimal.[1]

DMARC, since it causes denial of service to third parties, is such a
clear and present danger.  Here I interpret minimal to mean try to
avoid to adding to the set of RFC violations out there.  I know it's
tempting to imply that yahoo.com is an invalid domain, but it's not at
all necessary given that substituting the list-post address is what
Yahoo itself suggests.  The original user is easily replied to via the
Reply-To hack.

Footnotes: 
[1]  Retroactive implementation, such as Reply-To munging, may be
appropriate in response to customer demand.



--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] 2.1.18 internal documentation suggestions

2014-05-02 Thread Stephen J. Turnbull

Andrew Partan writes:

  Do you have a setting to change From: user@domain to From:
  user@domain.INVALID - that is the hack I would like to use.

Seems reasonable, but for the reason Mark gave and because it makes
personal replies a little bit harder, I *personally* would tend to
avoid it, and I recommend being careful to observe what's happening if
you try it (watch logs, listen for user complaints, etc).
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] 2.1.18 internal documentation suggestions

2014-05-02 Thread Stephen J. Turnbull

Mark Sapiro writes:
  On 05/01/2014 09:33 PM, Stephen J. Turnbull wrote:
   Mark Sapiro writes:
   
 The transformations for anonymous_list are applied before any of these
 actions, so if actions other than No are applied on an anonymous list,
 they will apply to the anonymized message.
   
   This may be confusing?
  
  How about ... if actions other than No are applied on an anonymous
  list, they will be redundant.

You already said that above, so it won't hurt to say it here too.  But
what I'm worried about (which was not clear, sorry!) is that if
somebody *does* turn one of these options on, the behavior they
observe will be confusing.  Ie, I wonder if

It is probably *not* useful to apply these options to an anonymous
list, and if you do need to do so, the result may be surprising.

isn't the most accurate statement of affairs.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] 2.1.18 internal documentation suggestions

Larry Stone writes:

  Seems to me saying “Try to ensure that 'From:' is “aligned” with …”
  does it.

No.  The problem is the author's email provider (ie, the mail domain
of the person whose address is in the original From).  For most lists,
Mailman does *not* want From to be aligned with any particular
domain, it wants to just pass From through.  RFC 5322 quite clear
that this is the correct behavior (as are all of its predecessors).

  I’d prefer to put the header field name in quotes or otherwise
  distinguish it. Otherwise, it can be difficult to parse - is “from”
  the header or a preposition.

Sure.

  But, I don’t like “author’s domain”. Who or what is the author?

The author is the person responsible for the content of the message,
per RFC 5322 (and all of its predecessors).  You could substitute the
original 'From' address if that seems more intelligible to
non-technical admins.

  Why not just say “the Mailman server’s domain” since that’s what
  it’s going to be aligned with.

That may or may not be true, depending on how the host is set up.  For
example, it may not participate in SPF or DKIM at all.  DMARC
alignment is a rather complex concept.  I do not think it's a good
idea to have sloppy wording here.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] 2.1.18 internal documentation suggestions

Mark Sapiro writes:

  I'm not sure what to change at this point. I really don't want another
  change in the attribute name, but maybe.

Yeah, I know.  On the other hand, now that it really matters, this is
probably the last chance to make such a change.

  I'm also not sure about alignment as that is a technical term in the
  DMARC spec and may be more technical than we want here.

Sure.  But from rewriting is something we do for other reasons
(anonymous lists), and saying that message/rfc822 encapsulation is
from rewriting seems way too inaccurate to me.

   [FIXME: Should this respect the MIME vs. legacy encapsulation
   ('digest') setting?  If 'yes', that setting should move to General
   or so?]
  
  I don't want to go the FIXME route. It's too hard for this release.

OK.

  Also, are you suggesting doing this for all messages based on what is
  now Digest options- mime_is_default_digest or doing it per user based
  on the user's Get MIME or Plain Text Digests?

Per user, because of the issues we've heard about specific MUAs having
trouble with MIME encapsulation.

  Also, this (legacy encapsulation) really only differs from the Munge
  From option in that a few headers are copied to the body of the message
  and non-text/plain part are scrubbed, and I don't know how valuable it
  would be.

True.  I mention it because we've had PRs about MIME encapsulation
already.

 header munging settings below with the exception of adding via
 real_name to the display name in the From: for an anonymous list and
   
   ??  Adding real name to From in an *anonymous* list?
  
  real_name refers the the list attribute which is the list name with
  possibly different capitalization, but I see it should be changed.

OIC.  I don't think you need to mention it here; Mailman should just
DTRT.  If it's an anonymous list, the list owner should configure
'From' correctly, that's all.

  Hold is not an option for dmarc_moderation_action. it is the action
  which applies to messages From: a domain with DMARC policy p=reject an
  optionally p=quarantine. The possible actions are Accept, Munge From,
  Wrap Message, Reject or Discard

I don't understand why we need both this and list_is_from?  The latter
is a clear violation of RFC 5322, acceptable only because it's one of
the approaches the DMARC proponents (and Yahoo!) suggest for mailing
lists faced with a DMARC DoS attack.  Why not just deprecate
list_is_from in favor of dmarc_moderation_action?
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] 2.1.18 internal documentation suggestions

Mark Sapiro writes:

from_is_list (general): Replace the From: header address with the list's
posting address to mitigate issues stemming from the original From:
domain's DMARC or similar policies.

That's good!

[snip my suggestion :]

The following actions are applied to all list messages when selected
here. To apply these actions only to messages where the domain in the
From: header is determined to use such a protocol, see the
dmarc_moderation_action settings under Privacy options... - Sender filters.

Good! Maybe even encourage use of d_m_a?

No
Do nothing special. This is appropriate for anonymous lists.

[snip]

The transformations for anonymous_list are applied before any of these
actions, so if actions other than No are applied on an anonymous list,
they will apply to the anonymized message.

This may be confusing?

The Reply-To: header munging actions below interact with these actions
as follows:

first_strip_reply_to = Yes will remove all the incoming Reply-To:
addresses but will still add the poster's address to Reply-To: for all
three settings of reply_goes_to_list which respectively will result in
just the poster's address, the poster's address and the list posting
address or the poster's address and the explicit reply_to_address in the
outgoing Reply-To: header.

[Note: is the above what we want? I think so, but others are adding a
header something like X-Mailman-Originally-From:

IIUC, yes, that's what we want. OnlineGroups has some features
Mailman doesn't (yet?) to handle the reply munging issue AIUI.

(1) X- fields are deprecated. They don't actually help in creating
private protocols, and (not relevant to us here, I think) they
make it difficult to upgrade to the standardized version.

(2) I think this is pretty useless (with one exception), because most
MUAs won't display the information. Even with Show Source (how
many AOLers use that?), you have to dig through a thicket of trace
fields and spam scores. Better to log the information. But why
do that when we archive the original message as received? (In
fact, if we do this correctly it would be possible to post-archive
DKIM-verify messages! GSoC 2015? :-)

(see the The future options for mailing list managers section at
http://onlinegroups.net/blog/2014/05/01/dmarc-taking-responsibility-sending-group-email/)]

They don't seem to think it's terribly useful though, it's just a sort
of trace field. BTW, that blog also says

The attackers succeeded in accessing about 20% of AOL users’ email
accounts and obtaining details of their contacts.

I hope that means that AOL is now down to the 100 Stupidest On-Line
Americans, of whom 20 were fooled But I digress.

These actions do not apply to messages in digests or archives or
sent to usenet via the Mail-News gateways.

Is that true of dmarc_moderation_action, too? (I assume so and would
consider it a bug if not.)

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe:
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] 2.1.18 internal documentation suggestions

Mark Sapiro writes:

  dmarc_moderation_action is unreliable. If the DNS lookup times out, the
  message is assumed unaffected by DMARC.

Ouch.  I suppose you could hard-code a list of miscreants, er, domains
that have used p=reject and fall back on that (including a check for a
change in policy of DMARC DoS'ers that results in an email to owner).

  If I could, I would set dmarc_moderation_action to Reject with a gentle
  suggestion to find another ESP to post from, but I can't.

Heck, even I don't recommend that (I do it, but that's only because I
*can* -- as I've mentioned my users are all looking for excuses to
switch to GMail anyway if they haven't done so already).

  One particular member of the board of directors is a very vocal and
  not very technical Yahoo user who I think would have a fit if her
  mail were singled out for differing treatment, even if only that
  hers was munged and mine wasn't.

Tell her it's mandated by Yahoo! and hard-coded in Mailman (blame
Barry! and don't tell her about the Time Machine), *your* hands are
tied. :-)

  I have tried in the past to programmatically hold or reject me
  too posts and others that have way more quoted that original material.
  (Almost everyone top posts and quotes the entire message being replied
  to.)

But this is different: it really is censorship.  It's censorship I
agree with, it's censorship that doesn't actually infringe on freedom
of expression, but it is prohibiting certain (obnoxious) forms of
expression.

N.B. If top posting bothered me (in channels where it is customary, it
doesn't bother me any more :-), I would implement a Handler that
removes trailing quoted material and substitutes a link to the
archives (if possible to the In-Reply-To message).
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Disappearing messages

2014-04-30 Thread Stephen J. Turnbull

sherwin writes:

  I run a forum on Ibiblio and am seeing a strange phenomenon
  lately.

It's not a phenomenon, it's a policy of Yahoo! and AOL.  The other
services' users are collateral damage, as the US DoD likes to say.

  I would like to know what is happening to these missing messages that do 
  not show up

For pointers to the full dope:

http://www.mail-archive.com/mailman-users%40python.org/msg64211.html

  in Inbox's, Spam Folders, or Trash Folders.  Do I tell our AOL users to 
  stop sending messages to our forum?

I can't *recommend* that as it's not really the users' fault (BTW, you'd
also need to censor the Yahoo! users), but I'd love to see you do it. :-|

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Excessive bounces to list members on my list

2014-04-30 Thread Stephen J. Turnbull

Peter Shute writes:

  Another question is what happens if yahoo groups receive aol
  bounces. They might not use them to disable or unsubscribe members,
  which would limit the damage to just non delivery.

Yahoo! is a proprietary service, not in the habit of telling anybody
what they're doing or why.  You'll have to ask them.


--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] 2.1.18 internal documentation suggestions

2014-04-30 Thread Stephen J. Turnbull

Mark Sapiro writes:

  I changed it. In the 2.1.18 final it will say:
  
  -
  from_is_list (general): Replace the sender with the list address to
  conform with policies like DMARC.
  
  Replace the sender with the list address to conform with policies like
  ADSP and DMARC. It replaces the poster's address in the From: header
  with the list address and adds the poster to the Reply-To: header.
  
  If this is set to Wrap Message, just wrap the message in an outer
  message From: the list with Content-Type: message/rfc822.

Ouch!  I'd really like to change this variable's name!  (It's not very
easy to understand anyway; grammatically the semantics are whatever
you see in From is {a, the} list, not replace whatever is in From
with the list's address.)

Something like from_alignment, with values None (don't try to align
domains in From), 'shift author' (From into Reply-To, list into
From), or 'wrap message'.

May as well rewrite the doc ... here goes:

from_alignment:  Try to ensure that From is not misaligned with
the author's domain, to conform with protocols like DMARC.
[FIXME: I don't see how to avoid the double negative.  Help?!]

This setting replaces the from_is_list setting, which is now
deprecated.  Existing from_is_list settings will be respected.

Several protocols now in wide use attempt to ensure that use of
the domain in the author's address (ie, in the From header field)
is authorized by that domain.  These protocols may be incompatible
with common list features such as footers, causing participating
email services to bounce list traffic merely because of the
address in the From field.  *This has resulted in members being
unsubscribed despite being perfectly able to receive mail.*

The following actions are applied to messages where use of such a
protocol is detected by Mailman.  [FIXME: Is that correct?]

Valid values:

'no': Do nothing special.  This is appropriate for anonymous lists.
It is appropriate for dedicated announcement lists, unless the
From address is not within the Mailman host's domain.  [FIXME:
Maybe None is a better value here.  Of course that's not backward
compatible, but with the name change it would be possible to check
the old from_is_list.]

'shift author': Shift the address(es) in From to Reply-To
(preserving existing addresses in Reply-To), and insert the list's
[posting?] address in From.

'wrap message': Treat the message as a MIME forward with list in
From and the original message encapsulated in a MIME message/rfc822
part.  Subscribers will perceive this as a one message digest.
[FIXME: Should this respect the MIME vs. legacy encapsulation
('digest') setting?  If 'yes', that setting should move to General
or so?]

  These settings play as expected with the anonymous_list and Reply-To:

What does as expected mean?  (If *I* have to ask :-)

  header munging settings below with the exception of adding via
  real_name to the display name in the From: for an anonymous list and

??  Adding real name to From in an *anonymous* list?

  adding the poster's address to Reply-To: in almost all cases.
  
  If anonymous_list is Yes, there is no reason to set from_is_list to
  anything other than No.

Unnecessary with my wording above.

  If dmarc_moderation_action applies to this message with an action other
  than Accept, that action rather than this is applied

This doesn't seem correct.  True, if Reject (aka emit backscatter)
or Discard, the message will never reach this point.  But if it's
Hold, this processing will be applied if the message is accepted by
the moderator.  How about

See also dmarc_moderation_action (which will be applied earlier in
processing than this feature).
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Excessive bounces to list members on my list

2014-04-29 Thread Stephen J. Turnbull

Gregori Kurtzman, DDS writes:

Your address drimpla...@aol.com.dmarc.invalid is invalid, I hope
you're reading the list.

  Need some insight and help. I have recently taken over a list that
  is using mailman v 2.1.14. And we are getting a lot of bounce
  notices regarding members and de-activation's of their
  subscriptions due to this.  In the bounce notices I get as list
  manager I see the following  'This message failed DMARC Evaluation
  Also members are complaining they see no messages coming from the
  list or even their own posts.

Yahoo! and AOL have decided that you are not allowed to forward
messages from their members, and enforce it using a DMARC policy which
requests that receiving mail servers bounce the mail, effectively
causing a denial of service attack on their own users.  Large services
like Yahoo, AOL, and Hotmail seem to be respecting the policy despite
the adverse effect on their users (ie, getting unsubscribed).  GMail
seems to have a more nuanced response so that at least some mailing
lists get through.

The problem occurs with posts *by* users with Yahoo and AOL addresses,
although it is experienced by your entire subscriber base (that is,
any subscriber at a service which respects the DMARC policy).

There is no generally satisfactory way for mailing lists to deal with
this.  Several options and a lot of discussion are in the threads with
DMARC or Yahoo in the subject at

http://www.mail-archive.com/mailman-users%40python.org/

tl;dr summary is

http://www.mail-archive.com/mailman-users%40python.org/msg63959.html

The most palatable option for most list operators seems likely to be
setting the from_is_list option, but that requires Mailman = 2.1.16
(avoid 2.1.17, IIRC it has a related bug that needs to be patched for
from_is_list to work properly, and 2.1.18 is not quite released yet,
but should come out literally any day now).

Whatever option you adopt for your list, your AOL and Yahoo users
should be advised that their email provider's policy is causing the
problem, and that they can defend themselves by switching providers
(GMail seems to be handling the situation best at the moment).  You
probably don't want to *advocate* switching, but that's up to you.
However, it *is* an effective remedy for the subscriber, if they are
willing to pay the (often substantial) annoyance cost of switching
email addresses.

Regards
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] moving mailman

2014-04-29 Thread Stephen J. Turnbull

jdd writes:

  and by the way logs are not always easy to find. Mailman's are not (on 
  openSUSE) available in /var/log, but in /var/lib/mailman/logs

True.  Postfix has a utility postconf and Mailman 3 will have a
similar capability (mailman info IIRC; maybe Mailman 2 has it too?) 
that allows you to print out application configuration, including log
locations.  (Of course if you use syslog or something like that the
location will be determined by the syslog application.  *sigh* -- but
at least then it should be in /var/log somewhere.)

I don't *expect* anybody to remember this advice, but it *might* save
somebody some hairpulling someday, so I mention it. :-)

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Excessive bounces to list members on my list

2014-04-29 Thread Stephen J. Turnbull

Jim Popovitch writes:

  In 2 years people will be wondering how DMARC did hardly anything to
  slow miscreants, just like some wondered why SPF, DKIM, PGP, SenderID,
  etc didn't solved all of mankind's problems.

N.B.  PGP *would* solve the world's problems if the GPG folks would
spend more time on usability and less time on ever more arcane
algorithms with the intent of keeping out the FBI and the NSA (who, as
Jamie Z so pithily points out, have subpoenas and black helicopters[1]
respectively, so there's little point in having crypto they can't crack).

Footnotes: 
[1]  OK, CIA and Division, not the NSA, but the {FS,EF}Fanatics
don't worry about the difference.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Newsreader access.

2014-04-28 Thread Stephen J. Turnbull

Peter Shute writes:

  A fully integrated web forum would solve the problem. Yahoo groups
  can be accessed via the web or by email, but had other problems I
  can't remember.

Mailman 3 is taking some steps in that direction in the HyperKitty
archiver module.  Also, last year I mentored a GSoC project for the
Systers which involved a sort of dashboard for Systers resources --
where Mailman is a central resource for the Systers.  Shanu
implemented it as a webapp, of course.  It's very much a prototype at
this stage, though, and it may not ready for use by people who can't
deal with email.  It will definitely require a fair amount of the
administrator at this point (completely unpackaged).

If you'd like to encourage Shanu, I can introduce you.

Steve

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Gmail and DMARC

2014-04-26 Thread Stephen J. Turnbull

Jim Popovitch writes:

  TBH, I'm not sure what else there is to look for. :-)  GMail, every so
  often, is telling my Mailman that it needs to Auth in order to reflect
  From:gmail to other gmail customers.  It's like DMARC without
  following the DMARC standard (GMail has a p=none policy).

Is it possible that those users are using their GMail address from a
non-Google host?  That would explain the intermittent nature of the
challenge -- it doesn't have the GMail DKIM signature that identifies
GMail-from-GMail messages.  And it would also explain why it looks
like DMARC, since DKIM is one of the underlying protocols used to
implement DMARC.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Can I remove a particular archive from the list?

2014-04-21 Thread Stephen J. Turnbull

Richard Damon writes:
  On 4/21/14, 6:56 AM, Amit Bhatt wrote:
   As a list administrator, do I have any option to remove any particular
   post from the archive of all messages?
   Please share how to do this.
  
   Thanks,
  
   Amit Bhatt
  
  Not through the Web interface. I believe if you have *server* admin
  level access, you can edit the mbox file holding the messages, remove
  the message, and regenerate the archives.

Yes, that's exactly the procedure.  However, this will cause all posts
after that message to change URLs.  You have some hope of preserving
URLs (which may have been communicated as references on the mailing
list or in private mail, etc) if you edit the mail in the mbox file,
substituting a fake address in the Unix From line that starts a
message, removing all but the required From, Date, and Message-Id
header fieldss and substituting a fake name and address in From and a
local id in Message-Id, and replacing the entire message body with the
sentence This message was removed by the administrator.  Then
regenerate the archives.

This could actually be automated fairly easily I suppose, but I don't
know if it's a great idea.


--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and From header munging

2014-04-18 Thread Stephen J. Turnbull

Lindsay Haisley writes:

   It's very ugly, though, especially if for some reason you have no
   display name to work with.
  
  Agreed!  But the display name is free form and strictly informational.
  Could this not be the subscriber name of the author, if it's part of the
  subscription record?

Sure.  But that's putting even more burden on the Mailman server, and
in many cases it won't be close to 100%.  That's all I'm saying.




--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Bellsouth, etc.

2014-04-18 Thread Stephen J. Turnbull

Jim Popovitch writes:
  On Thu, Apr 17, 2014 at 2:32 PM, Stephen J. Turnbull 
  step...@xemacs.orgwrote:
  
   So maybe it does, but in my spamtrap I have only 67/4359 (1.5%)
   messages from Yahoo (based on grepping for ^From:.*yahoo and
   ^From: respectively), vs. 658/38748 (1.7%) in my saved mail folders.
   It seems to me that spam using Yahoo addresses is hardly a big
   problem, whether it's spoofed or using throwaway addresses.

  I'm curious, what numbers do you currently see for tumblr (also a yahoo
  company) spam?

Zero.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Emails from yahoo members, are getting rejected by yahoo, Service Unavailable.

2014-04-18 Thread Stephen J. Turnbull

Sylvain Viart writes:
  Le 18/04/2014 09:41, Alain Williams a écrit :
   I may have missed some topic, but why SRS
   (http://en.wikipedia.org/wiki/Sender_Rewriting_Scheme) doesn't
   come to rescue here?
   SRS rewrites the *envelope* sender.
  
   My understanding is that the YAHOO DKIM uses the From: header,
   not the envelope sender.
  
  Oh I see. Thanks.
  So to use mail's Header terminology, only the Return-path: is modified, 
  not the From:

No, the envelope sender often does end up in Return-Path, but it need
not.  The envelope sender is the entity in the SMTP MAIL FROM
command, not anything in the headers.

  Is this related to DMARC in general?

Yes.  DMARC is designed to work with From alignment, that is,
authenticating the domain in the from header.  It *can* also
authenticate the mailbox using DKIM, but there's no guarantee that a
third party can see.

The reason for this is that the DMARC authors are concerned about
phishing, which basically works by sending a fake From.  Therefore
they want to ensure that only the real domain can send From that
domain.

  May be not corrected by SRS, because of DMARC is more recent than
  SRS…

They're completely unrelated.

  Or this is a yahoo challenging with a somewhat too strong
  configuration?

Yahoo is following the DMARC standard.  They don't believe it is too
strong.  See above.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Bellsouth, etc.

Lindsay Haisley writes:
  On Wed, 2014-04-16 at 15:34 -0500, Mike Starr wrote:

   I know there aren't any teeth behind RFCs but it might at least get
   their attention.

The real problem is that RFCs are based on working practice,
preferably acknowledged best practice.  DMARC is an experiment which
is seriously flawed on the policy side, but has the potential to
provide a lot of useful information for spam-fighting (I mean real
spam-fighting, not the posturing that Yahoo! is involved in at the
moment), not to mention lightening the burden on ISPs and list
operators who implement DKIM and SPF.  Until Yahoo!'s experiment has
played out (which will take months), an anti-DMARC RFC is moot.  After
that, it will take years to get it through the IETF.

Note that DMARC itself is an Internet-Draft (ie, proto-RFC).  If you
want to fight this, the related mailing list is the right place.
However, looking at some of the threads there are rather high-powered
folks already on the list (eg, the guy who edited most of the SMTP
RFCs, and the guy who edited most of the RFC 822 series).  You had
better go in having booked up, or you will get ignored to death at
best.  Put it this way: *I* may go look over their archives, but it
will be quite a while before I'm willing to speak to anything except
technical details of how it affects mailing lists.

  Doubtful, but the sentiment is noble.  My guess is that the people
  at Yahoo who implemented this, and possibly also the designers of
  DMARC, don't fully understand the RFC process and have a limited
  attention span and very narrow focus of attention as far as such
  things are concerned.

Nope.  If E. Zwicky (DMARC editor) is who I think she is, I owe her a
kitten.  No dummy.  Murray Kucherawy doesn't seem to have two heads or
a half-brain, either.

  Their understanding (and knowledge) of accepted best practices
  regarding email and mailing lists is woefully limited.

I rather doubt that.  The DMARC I-D has gone through several editions
(I-Ds have a life-span limited to 6 months, the current renewal
happened just about the time of Yahoo!'s policy change), suggesting
that the NetGods and the commercial providers have been thinking
pretty carefully all along.  I think that where understanding and
knowledge is lacking is on *this side* of the fence.  Few, if any, of
us have to make decisions about how to spend many millions of dollars
on additional bandwidth, 90% of which (according to some accounts) is
spam.  That's a pile of money on the line for these guys.

  My guess also is that as a result, all of this kerfuffle has
  probably caught a number of these people by surprise.

Indeed.  I suspect that they didn't do their homework and simply count
how many subscribers receive mail with List-* headers in them.

I think they probably also were surprised by how fast Yahoo is
hemmorhaging email users.

Steve-rushes-not-where-angels-fear-to-tread-ly y'rs,
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Bellsouth, etc.

Larry Kuenning writes:

  Query:  On a very low-traffic mailing list (i.e. one where the list 
  admin doesn't think it too much trouble), would it be a reasonable 
  workaround for the list admin to paste the content of a 
  message-to-be-moderated (i.e. one From: a yahoo address) into a new 
  message _of his/her own_ and send _that_ to the list?

Yes, although given the available alternatives in the web admin pages
I don't think this is worth the trouble for almost anybody (I
understand that you have a very special situation with a relatively
old Mailman that's working just fine, thank you, for you, but that's
pretty unusual nowadays).

  This message could include the original From: address _in its body
  text_ (not its headers) along with a brief reference to the yahoo
  problem to explain the unusual format.

  1.  Other subscribers replying to the message will get MUA-generated 
  text saying Larry List-Admin wrote instead of Sonia Subscriber 
  wrote.  Those who pay attention and take a little trouble can change 
  that before clicking Send, but many won't.

Change the display name to Sonia Subscriber/lla (the usual
convention for letters written by a secretary but signed by the boss).

  2.  Similarly, other subscribers wanting to reply privately will send 
  their replies to Larry List-Admin instead of Sonia Subscriber if they 
  aren't careful (and some of them won't be).

Add a Reply-To: so...@her-place.net header field.

The recommendations above violate the letter but conform to the spirit
of RFC 822 and successor standards.

  The list admin can forward these replies, but in a few cases they
  may contain confidential material that the admin shouldn't have
  seen.

The above practices should mitigate this issue.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Trying to understand charset encoding in mailman

Hi, Laura!

Laura Creighton writes:

  But the Europython mailing list is configured so that its messages
  come out
  
  Content-Type: text/plain; charset=us-ascii

This isn't from the list or site configuration, this is from the
poster's mail user agent (MUA).  The mailing list does not choose the
charset for the message; the MUA does.  For example, grepping my
archive of python-dev messages I see 3 different variants of UTF-8
(capitalization and quoting), us-ascii, iso-8859-1, and window-1252
(each in several variants).

Mailman already has about 200 lines of logic to handle cases where the
footer charset is incompatible with the message's charset.  Have you
tried simply changing the Python escape to a literal EN DASH in the
web interface?  I hope Mailman is smart enough to convert that to
Unicode internally, and all should Just Work[tm].

If that doesn't work, change the EN DASH to --, and report it as a
bug.  We'll see what we can do in 2.1.19, before EuroPython is held in
Göteborg or Łódź. :-/

  Since \x96 is an unrecognised character in us-ascii,

It's not even a character here, it's a raw byte, which may or may not
get recognized correctly by Mailman depending on the list's preferred
charset.  Somebody was way too tricky for their own good.

  But unless I have overlooked something, there is no way to make a charset
  change on a per-list basis through the mailman administrative interface.

There's no way to make a charset change in posts at all; it's not
Mailman's job to do that, really.  I suppose we could convert all
posts to UTF-8, which would make the logic mentioned above a lot
simpler, but that would probably annoy a few people and might not work
for some variant charsets.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Trying to understand charset encoding in mailman

I see you've already responded, but there are a few things I'd like to
clarify.

Laura Creighton writes:

  But you and I could quite easily both want English(USA) as the
  default language for our lists, but you also want us-ascii while I
  want utf-8.  The way things stand now, we cannot both use the same
  mailman host, and both get what we want, correct?

In this particular case, you can choose UTF-8, and you both get what
you want.  US-ASCII is a subset of UTF-8.  Anybody else *may* have a
problem if they have sufficiently ancient software that it can't
handle UTF-8, but that's a rapidly vanishing issue.

In fact *right now* you and I and Wang Han Lo can use English,
Japanese, and Mandarin on the same list at the same time, each posting
and setting our subscription options in our preferred language.  It's
only footers and headers that have this issue, and that's at least
partly because even today there's no reliable way to mix charsets in a
message (too many users still use MUAs-that-suck).  For most purposes,
Mailman is pretty well internationalized, it's just that some corner
cases remain ugly.

The other cause is historical accident.  Email is very messy -- it's
one of the oldest Internet protocols.  Mailman itself goes back to a
time when neither Python nor its email package had a coherent way of
dealing with multilingual applications.  So we've been overhauling
various parts of Mailman 2 as necessary.  And users -- well, many
Mailman list admins think that they type Japanese or German rather
than EUC-JP or ISO-8859-15, and undoubtedly the charset-per-language
architecture was intended to make life easy for them.

Why nobody ever got around to properly internationalizing the headers
and footers (ie, allowing charsets defined per list) I'm not sure.  I
suspect it's because few users ever tried on international lists: they
just use English in the footers as lingua franca.  This is the first
time I've seen somebody reporting issues with the internationalization
of the footer, and I've been following Mailman since 1999 or so.

Getting it right by design ... well, that's why we need Mailman 3.  We
know a lot more about lots of things than we did when Mailman 2 was
designed.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Bellsouth, etc.

Lindsay Haisley writes:

  Stephen, thanks for your generous reply, and your insights.  It
  does seem to me, though, that when megabucks are riding on
  additional bandwidth, and if Yahoo is serious about controlling
  spam, they might start by putting some resources behind putting
  their own house in order.

Nobody can control spam in the current architecture of Internet mail.
What needs to be done is author identification, that is, digital
signatures.  But that requires cooperation from users, which is
anathema to the freemail providers.  So p=reject, and to a lesser
extent DMARC itself, are basically PR stunts IMO, see below.

  Someone, maybe it was you, posted on this forum earlier that perhaps 90%
  or more of spam with a yahoo.com origin (or one of their international
  DNs) actually _does_ come from Yahoo

Wasn't me.  I don't have that data, and don't know where to get it
offhand.

So maybe it does, but in my spamtrap I have only 67/4359 (1.5%)
messages from Yahoo (based on grepping for ^From:.*yahoo and
^From: respectively), vs. 658/38748 (1.7%) in my saved mail folders.
It seems to me that spam using Yahoo addresses is hardly a big
problem, whether it's spoofed or using throwaway addresses.

  and that their response to abuse notifications is abysmal to
  nonexistent.  So it looks to me as if one of two things is
  happening here.  Either the right hand doesn't know what the left
  hand is doing (or not doing), or this is a blatant, cynical attack
  on network neutrality designed to push people toward Yahoo's own
  list service.

I think the main thing is that the decision-makers (who are basically
business people) see this as a marketing/PR problem.  I don't think
it's an attack on network neutrality per se so much as a PR stunt to
be perceived as doing something about spam and phishing.  I wonder
if they're not positioning themselves to do something big in finance
or expand in handling payments to vendors who use their e-business
platforms -- which would make a tough on phishing stance very
important to them, as it is for banks.

  Has anyone seen or heard any figures on how much this DMARC fiasco has
  cost Yahoo in terms of the number of email end-users who have left their
  service?  Someone mentioned that it was substantial enough to probably
  get their attention.

I did but that was based on my personal experience, with (as I wrote
elsewhere) users who are not very attached to any particular email
address yet.  I don't see how anybody could get reliable figures,
though, except Yahoo! themselves based on statistical analysis of
outbound traffic and maybe an increase in the number of accounts that
.forward to other accounts.

Steve
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] DMARC and From header munging

Lindsay Haisley writes:

  Mailman to change the From: address to a VERP-like address with the
  author's address encapsulated within an address @ the list server.
  Any mail received by the list server for this address would have
  its address parsed by Mailman and be redirected to the original
  author's real email address.  Would this pass RFC compliance?

Technically, it probably does.  The problem is that Mailman doesn't
handle those mails, the MTA does.  It would be reasonably easy to set
up a filter and have the MTA pass the message to that filter.

It's very ugly, though, especially if for some reason you have no
display name to work with.

A bigger problem, as stated what you've done is to set up an open
relay.  So you would need to either maintain a database of valid
addresses forever, or do some crypto trickery so that only valid
addresses would be forwarded.  The latter would involve key
management, etc.

N.B. I read a very similar suggestion somewhere, probably in the DMARC
Internet-Draft or in their FAQ.
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] DMARC and Gmail

Lindsay Haisley writes:

  I've been working with the list admins of one of FMP's hosted lists and
  they've seen over 100 addresses unsubscribed from the usual suspects -
  yahoo.com, att.net, Comcast, etc., but no Gmail accounts and there are
  228 of them on the list.  Nonetheless, the PC World article [...]
  lists Gmail as being one of the cooperating email service providers
  honoring Yahoo's DMARC p=reject policy.

I wouldn't trust the popular press to be fully accurate.  Even one
test delivery failure would probably be counted as honoring, and
it's not obvious that you need to specifically test mailing lists,
since DMARC doesn't explicitly allow treating different DMARC failures
differently.

  I've been telling list admins to recommend that subscribers drop
  their Yahoo accounts in favor of Gmail.

That remains good policy AFAICT.

  What's the story here?

There are several possibilities.  One is that DMARC doesn't define the
semantics of reject.  (Why doesn't that surprise me?)  Here's what
they say:

   15.4.  Rejecting Messages

   This proposal calls for rejection of a message during the SMTP
   session under certain circumstances.  This is typically done in one
   of two ways:

   o  Full rejection, wherein the SMTP server issues a 5xy reply code
  as an indication to the SMTP client that the transaction failed;
  the SMTP client is then responsible for generating notification
  that delivery failed (see Section 4.2.5 of [SMTP]).

   o  A silent discard, wherein the SMTP server returns a 2xy reply
  code implying to the client that delivery (or, at least, relay)
  was successfully completed, but then simply discarding the
  message with no further action.

   Each of these has a cost.  For instance, a silent discard may
   prevent backscatter (the annoying generation of delivery failure
   reports, which go back to the RFC5321.MailFrom address, about
   messages that were fraudulently generated), but effectively means
   the SMTP server has to be programmed to give a false result, which
   can confound external debugging efforts.

A silent discard by Google is consistent with your observation,
since no bounce would be generated.

However, it is not consistent with Mark's experimental outcome.[1]  So
apparently, at least in their implementation of DMARC, Google takes
their Don't Be Evil slogan quite seriously.

It is clear to me that the silent discard method is the right way to
handle a DMARC p=reject policy.  Although the receiving MTA is giving
a false result in some sense, in fact the DMARC-using domain can
request a specific failure report which will enable the domain to
determine why non-delivery occurred despite an SMTP success.  If they
don't request such a report, too bad for their users.

Note that the annoyance mentioned in the 4th paragraph includes
denial of service to completely innocent third parties, ie, the
DMARC-triggered unsubscribes that have been observed.


Footnotes: 
[1]  His message arrived while I was composing this one.

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Bellsouth, etc.

Jose I. Rojas  writes:

  We have a community group mail list which we run using Mailman and
  have lately had a problem getting our emails to members who have
  Bellsouth and Yahoo email addresses. I've seen the posts about DMARC
  but am not that tech-savvy to figure out what this means and how to
  resolve.

What it means right now is that posts with @yahoo.com in the From
header field will not be delivered to users whose subscribed addresses
are at a long list of large email service providers.

If emails posted by users with @gmail.com and @harvard.edu etc
addresses are getting through to everybody, but emails from
@yahoo.com members are not, then the problem may very well be
Yahoo!'s DMARC policy.

  Our ISP tells us domain is RFC-compliant and problem must be with
  Bellsouth or Yahoo.

That's not very helpful of them.

  How do we resolve this?  What is the fix?

If in fact the problem is Yahoo!'s DMARC policy, you can't resolve it
and there is no fix.  Simply put, Yahoo! does not permit their users
to post to modern mailing lists that conform to the mail standards.
There are four possible workarounds, depending on the access you have
to your mailing list's configuration:

(1) You can tell your members with @yahoo.com addresses to post from a
different domain.  This is what I personally recommend, as it (a)
conforms to Yahoo's stated policy and (b) makes Yahoo users
unhappy with their provider, whose behavior is causing denial of
service to thousands, perhaps millions, of mailing list users.

My experience with this approach is no complaints, but my users
are unusual in that they don't really care about their yahoo.com
addresses for various reasons.  People who do most or all of their
mail using Yahoo addresses will find this painful.  Depending on
how actively you want to protest Yahoo's behavior, you may or may
not be willing to impose that pain.

(2) You can break your mailing lists by using the author_is_list
option in Mailman 2.1.16 and later.  This option will only be
available if the site configuration has ALLOW_AUTHOR_IS_LIST set
to Yes.  This will cause the list to replace the author's
address with its own address in From.  However, your domain may
not permit this, as it's a clear violation of the mail RFCs.

(3) There is a patch to have Mailman encapsulate posts from yahoo.com
addresses in a one-message digest.  This is RFC-conformant, but
some users may have difficulty reading such mail.  (Frequently
reported on iPhones.)  It also requires using a third-party patch
for Mailman, which may be prohibited by your ISP or beyond your
technical capability in the short run.

(4) You can operate Mailman in pure pass-through mode.  I believe it
is sufficient to configure Mailman to (a) have a completely empty
header (not even whitespace) (b) a completely empty footer (c) no
list prefix in the Subject header field.  This is conformant to
the RFCs, but may place you in violation of anti-spam law (because
for most users there will be no visible indication of how to
unsubscribe from the list).


--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] DMARC and Gmail

Alain Williams writes:

  They should have allowed/defined a new 2xy code that could be
  returned, eg 253 which means ''Mail accepted but will be
  discarded''.

That's problematic.  It would require an extension negotiated via EHLO
at least, and maybe a new SMTP RFC, since there's no registry for
extensions to the SMTP reply codes.  It might not be harmful, since
most modern MTAs are 2821-conforming, and so must interpret 253 as a
2yz success == 250, even if they don't understand 253 specifically.
I note that RFC 821, the current standard, does *not* have this
requirement, though.  Still, it could work, I guess, since DMARC
policies are outside-of-RFC agreements anyway.

  However: it still means that some people on mail lists occasionally
  don't get stuff - this will cause confusion at best or could be
  dangerous (if the mail list has a critical function).

Sure, but that's the tradeoff that DMARC explicitly makes.  DMARC
thinks that rejecting spam and phishing is sometimes more important
than delivering legitimate mail, and that the provider of a mailbox is
the appropriate entity to make that decision.

It's not limited to mailing lists, either.  Anybody who has a
forwarding mailbox is at some risk (in a personal .forward this is a
simple pass-through preserving the DKIM signature so it should be OK,
but I've seen commercial forwarders who add junk in the footer), and
it breaks the common patterns where a website allows you to request a
mail to a friend or an email service provider allows you to use
different From addresses (all of my mail from my @xemacs.org address
is sent from a different domain, and of the large webmail providers at
least Gmail provides this feature, and I use it occasionally).
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Accessing sub-letter search lists

Conrad G T Yoder writes:

  Since we’re on the topic of items residing at bugs.launchpad.net,
  what are the odds of Bug #1067953
  (https://bugs.launchpad.net/mailman/+bug/1067953) getting a little
  love?  Not quite as easy a change as this one, but would benefit many
  I’m sure.  Everyone subscribe! 

Pretty good, now that you've posted this -- the Mailman developers are
sprinting at Pycon as I write this, I believe, or maybe still at
dinner since none of them are on IRC.

A few words to the wise, in the interest of DAO[1].  At least for me
personally I would be slightly more likely to look, and very likely to
look sooner, if you posted the title as well as the bug number.

In general, Mailman developers are mail-oriented (including Barry who
last I heard was a Launchpad developer).  I don't know if anybody who
can do something about bugs actually pays attention to subscription
counts.  Can't hurt, but posting here (or on -developers if you
actually want to participate in fixing by contributing code or code
reviews) is likely the best way to be a squeaky wheel.  Please avoid
me too posts unless nobody responds in a few days.  At least Mark
and I read *every post* to mailman-users.

Once discussion on a fix starts, it will typically move to -developers
for the requirements and design discussion, then to the issue itself
(this is formalized for developing Python itself, we're less informal
but take our cue from Python).


Footnotes: 
[1]  Developer Attention Optimization. :-)

--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] DMARC