Re: [Mailman-Developers] Camera-ready option to mitigate DMARC issues

2016-11-06 Thread Stephen J. Turnbull 
Alessandro Vesely writes:

 > Camera-ready is cleaner than anything I have heard till now.

Clean, maybe, but Mark explained why the scheme is fragile even if you
can get participation.  The killer problem is personalization, but the
other problems are also intractable.  Serial numbers in the Subject is
a case where the variable content is not predictable by the author or
originating system.

 > Probably it is not workable, but I cannot understand why.  It works
 > well in several publishing environments, typically journals, which
 > distribute templates to authors.  Why can't it work for mailing
 > lists?

It can, it just depends on high participation rates to be useful.  As
I pointed out, if you can get that participation, you don't need any
changes to Mailman, you just don't add footers to the body or list
tags to the Subject.

So why don't you try the experiment on some of your lists and see how
much participation you get?
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Camera-ready option to mitigate DMARC issues

2016-11-06 Thread Mark Sapiro 
On 11/06/2016 09:58 AM, Alessandro Vesely wrote:
> 
> Camera-ready is cleaner than anything I have heard till now. 
> Probably it is not workable, but I cannot understand why.  It works well
> in several publishing environments, typically journals, which distribute
> templates to authors.  Why can't it work for mailing lists?


Mailing lists are different. They are subject to attack from spammers
and others which is different from the environments you are looking at.

I don't think your suggestions are workable. If you were to submit an
actual implementation, we could evaluate it's specific feasibility, but
in the absence of that, I won't spend time looking for flaws in
something that doesn't exist.

Note that Mailman lists can already be configured to make no DKIM
signature breaking modifications to messages. Most lists are not
configured that way because the modifications, which go beyond the
simple addition of a footer or a subject prefix, are desired by the list.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Camera-ready option to mitigate DMARC issues

2016-11-06 Thread Alessandro Vesely 

On Sun 06/Nov/2016 09:17:53 +0100 Stephen J. Turnbull wrote:

Alessandro Vesely writes:


The idea is to add a footer only in case it is not present,


Aside from the technical difficulties that Mark describes, this
suffers from a really big defect: for this to be actually useful,
you'd need near-100% participation (Authenticated Received Chain has
the same problem).


No, the difference is that ARC applies at the receiving side, so you need 100% 
compliance of list subscribers.  Camera-ready applies at the sending side, so 
the list can still apply any anti-DMARC workarounds if it fails.



Once you've got that, no new Mailman code is needed: just don't use a footer
in the first place!




And it requires effort on the part of providers we already know are
irresponsible and inconsiderate because they provide personal mail services
disrupted by "p=reject", or on the part of users at those sites, many of
whom blame mailing lists, not their irresponsible and/or inexpert providers,
for difficulties with posting.


Yes, but then it would be less disgusting to punish users of intractable 
providers.


If you don't get near 100%, then you need to provide the workarounds
for all users and originating sites that don't participate anyway, to
all of your subscribers -- and "nonparticipants" will include the
posters who are most likely to blame the lists for any delivery
problems, and get upset about "different treatment" (eg, From-
munging).


Yes, that's already in place.  It is considered a somewhat dirty solution. 
Camera-ready is cleaner than anything I have heard till now.  Probably it is 
not workable, but I cannot understand why.  It works well in several publishing 
environments, typically journals, which distribute templates to authors.  Why 
can't it work for mailing lists?


Ale
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Camera-ready option to mitigate DMARC issues

2016-11-06 Thread Alessandro Vesely 

On Sat 05/Nov/2016 19:51:13 +0100 Mark Sapiro wrote:

On 11/05/2016 04:11 AM, Alessandro Vesely wrote:


The idea is to add a footer only in case it is not present, similar to
what is done with subject_prefix.  By properly setting both of them, a
sender can submit what can be called a camera-ready post.  Since no
change applies, no DKIM signature breaks.  Hence,
dmarc_moderation_action is not needed for such posts.  It is not even
necessary to check author's domain policy.


Mailman could conceivably keep track of whether it has changed any
headers or anything in the body of the post, but it's more complicated
than that. The first big problem is the Munge From or Wrap Message
transformations are applied before any msg_header or msg_footer is added
(or maybe added).


Is it possible to abort processing the in-memory msg and revert to the file? 
Doing so --after thorough checks-- would prevent breaking the signature by 
altering the order of recipient, switching MIME values from token to 
quoted-string or vice-versa, and similar changes that memory representation may 
unwittingly imply.



I.e. in both MM 2.1 and MM 3, the DMARC mitigations are applied in the
incoming handler pipeline before the message is queued for delivery
processing.


All right, so we cannot save that lookup.


Various decorations such as adding msg_header and msg_footer and modifying
To: depend on "personalization" and have to be done by delivery processing
on a per-recipient basis. In fact, the "camera ready" notion can't apply to
any post that is going to be personalized in any way. This in itself would
limit the usefulness.


Sure, personalization cannot be compatible with camera-ready.


It would be more feasible to do this by the poster adding a
"X-Camera-Ready:" header to the post saying don't transform my message,
but this is unacceptable as it would bypass content filtering,
personalization and various other things.


X-Camera-Ready: may be useful to automate at senders'.  For an author doing it 
by hands, having to set a header field is an added difficulty...



I'm not familiar with Mailman administration, so I ask your opinion.
How long would it take to code this option?


How many angels can dance on the head of a pin?


Ah, not so many of them are still able to perform that, nowadays ;-)


Camera-ready posts would be created by hands, by cleverly configuring
some email client, or by using purposely written add-ons.  They could
also be done by MSAs who care about the damage they cause by publishing
p=reject --the process can certainly be standardized and automated.


How does the sender's automated process even know what msg_header and
msg_footer will be added by the list?


MTAs can learn List-Post addresses when they receive mail.  When they see one, 
they can, say, change the envelope recipient to an internal mailbox which 
processes the decoration (and maybe adds X-Camera-Ready:).  The decorating 
module could be carved out from Mailman, and complemented so as to let it 
download its parameters from well known locations or some such.


Ale
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] MM3 DMARC mitigations

2016-11-06 Thread Stephen J. Turnbull 
Removing known MM-DEV subscribers, the CC list is getting long.

David Andrews writes:
 > At 11:06 AM 11/5/2016, Mark Sapiro wrote:
 > 
 > >However, I've just become aware that Microsoft has implemented another
 > >"feature". [...] one of the tests is the To: and
 > >From: addresses are the same. That means that any message To: a list
 > >with DMARC mitigations applied will be sent From: the list and any
 > >recipients using these Microsoft services will see that warning in the
 > >list message[1].

The fact is, every time we work around the actual semantics of
"p=reject" (described in the block quote below), the spammers can hide
behind our usage, I expect they will, and those tricks will be given
spam (or worse, phish) points.

Maybe it's time to default to rejecting posts from p=reject domains,
with the explanatory message:

Your domain publishes a "p=reject" DMARC policy, which is a
statement to recipients that they allow you to send only
authenticated direct mail.  This is a mailing list which re-sends
your mail after processing, and therefore you are not allowed to
post according to your email provider's policy.  Please repost
from an address which allows you to post to full service mailing
lists.

Note: A few large providers claim to permit posting to mailing
lists, but publish "p=reject" anyway.  They privately acknowledge
doing so to protect users from spammers and phishers who have
stolen millions of address books and other private information of
users from them.

Of course this could be conditional on the presence of a header OR a
footer OR Subject-munging in the list config.  I'm semi-serious, but
more likely is a "pass-through" tactic: ie, no headers, footers, or
Subject-munging on posts from "p=reject" sites.  The point is that
spammers can omit those things, but they can't emulate the valid
authentication on posts from DMARC sites.  Unfortunately, some lists
do need to add disclaimers and the like, so can't use the pass-through
approach.

The only tactics that actually work in the sense that spammers can't
use them are (1) ARC (but that requires cooperation from receiving
hosts that is unlikely[1]), (2) "pass-through" operation, and (3)
preemptive rejection/discard.

 > This message has started appearing on messages on a list I subscribe 
 > to at work, the state of MN, and they use hosted office 365 etc., and 
 > the messages are almost always from legitimate senders, so going to 
 > be a problem.

My condolences.

Footnotes: 
[1]  It needs to be *all* receiving hosts.  Some of them will
undoubtedly be sufficiently lacking in email expertise that they
delegate these decisions to Office 365 etc.

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Camera-ready option to mitigate DMARC issues

2016-11-06 Thread Stephen J. Turnbull 
Alessandro Vesely writes:

 > The idea is to add a footer only in case it is not present,

Aside from the technical difficulties that Mark describes, this
suffers from a really big defect: for this to be actually useful,
you'd need near-100% participation (Authenticated Received Chain has
the same problem).  Once you've got that, no new Mailman code is
needed: just don't use a footer in the first place!  And it requires
effort on the part of providers we already know are irresponsible and
inconsiderate because they provide personal mail services disrupted by
"p=reject", or on the part of users at those sites, many of whom blame
mailing lists, not their irresponsible and/or inexpert providers, for
difficulties with posting.

If you don't get near 100%, then you need to provide the workarounds
for all users and originating sites that don't participate anyway, to
all of your subscribers -- and "nonparticipants" will include the
posters who are most likely to blame the lists for any delivery
problems, and get upset about "different treatment" (eg, From-
munging).

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9