Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Hendrik Boom (hend...@topoi.pooq.com):

> I *hate* the way that the Google Groups mailing lists refuse to 
> include the mailing-list headers so you cannot just 
> reply-to-list.

*glyph of surprise*

Are you sure?  I just checked a recent example, and List-Post is there
and completely correct and appropriate.  That is the signifier dictated
by RFC 2369 section 3.4 as the method for posting.  (In atypical cases, the
software administrator might cause it to point to a moderator, or to
some other location for submission, instead of to the mailing list's
address.  In the specific atypical case of a mailing list that does not
allow posting, e.g., an announcements list, the List-Post field may
contain the special value 'NO'.)

Selected headers from the example posting:

Date: Fri, 21 Dec 2018 09:13:32 -0800 (PST)
From: goossbears 
To: BerkeleyLUG 
Message-Id: 
Reply-To: berkeley...@googlegroups.com
Mailing-list: list berkeley...@googlegroups.com; contact
berkeleylug+own...@googlegroups.com
List-ID: 
X-Google-Group-Id: 61884646931
List-Post: ,

List-Help: ,

List-Archive: ,

List-Unsubscribe: 
,



In this particular case, the listadmin has also made the regrettable choice
of munging Reply-To: to cater to technophobes[1] -- but that is a different
matter, and also not dictated AFAIK by Google.


[1] http://marc.merlins.org/netrants/reply-to-still-harmful.html
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Hendrik Boom]

On Thu, Dec 27, 2018 at 10:17:41PM +, Simon Hobson wrote:

> 
> Nice thought, but do you really think that the likes of Google give a sh*t 
> about some little mailing list somewhere, and which should be using Google's 
> services anyway - how dare they use their own solution !
> The reality is that the "big boys" have implemented these breakages - they 
> knew beforehand that they would break almost all forms of forwarding, but 
> their solution to that "problem" was simply to declare any form of mail 
> forwarding as "improper" and therefore breaking it wasn't their fault. I 
> can't help thinking that their marketing people saw an opportunity to make 
> life harder for small scale competitors.

I *hate* the way that the Google Groups mailing lists refuse to 
include the mailing-list headers so you cannot just 
reply-to-list.

-- hendrik
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Miles Fidelman (mfidel...@meetinghouse.net):

> Ahh... missed that.  Didn't really notice anything until this huge
> string of emails.  Sigh...

Eh, no worries.  I half-realised that's what probably happened.

[publishing SPF & DMARC/DKIM records in DNS for a mailing list host:]

> True, but it sometimes helps.  And it's easy enough if one has
> access to one's nameserver records, as anyone who runs a list
> manager usually does.

Just as a matter of personal perspective/opinion:  I watched the
introduction of DKIM (né DomainKeys) by Yahoo and considered it so
botched that I wanted nothing to do with it.  When Yahoo extended DKIM
to create DMARC, it seemed to me Yahoo had learned nothing from the
DomainKeys/DKIM experience and screwed up a second time.  

By contrast, all of the complaints against SPF (the real ones, not the
bullshit non-sequitur complaints like 'SPF doesn't block spam' and
'spamhaus domains can and do publish SPF records) divide neatly into two
categories:

1.  I object to /etc/aliases and ~/.forward breaking and refuse to use
SRS in their entries.  (Dude, wrong decade.)

2.  I want to be free to originate outgoing SMTP from arbitrary
not-previously-planned IP addresses and not have it be suspected of
forgery.  (Dude #2, good luck with that.  Also, still the wrong decade.)

Both factions kept advising me it's Bad and Wrong for me to publish an
SPF record saying 'Please reject as forged any mail purporting to be
from my domains that isn't from IP address 198.144.195.186', to which I
always responded 'Why the Gehenna is that _bad_?  It's exactly what I want
to happen, because all mail genuinely from my domain comes from my IP.
If users, even those who have shell on my machine, are forging my domain
from other IPs, I _want_ that mail to fail as forged, because it's
actually forged, and users should not try to do that.'

Anyway, as far as I'm aware, nobody is distrusting mail legitimately
from my domains for lack of DMARC attestation.  I keep asking people
suggesting DMARC what demonstrable benefit my domains would get that
they don't already get from a very clear and emphatic SPF policy, and
nobody's yet given me a compelling answer.

If things change and I _do_ see signs of penalising domains with
emphatic SPF policies but no DKIM/DMARC, then I'll reconsider.

(Above speaks, obviously, just for me and my domains.  I'm not part of
the Dyne/Dng administration team, just a friendly Devuan sysadmin and
recently-Debian-leaning Operations guy.)

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Miles Fidelman]

On 12/27/18 11:07 AM, Rick Moen wrote:


Quoting Miles Fidelman (mfidel...@meetinghouse.net):


Speaking as someone who hosts a couple of dozen email lists, I
really don't understand what the fuss is about here.

The fuss involved people having paid no attention to the announcement of
Dng's DMARC-mitigation munging starting on December 6th, and so being
confused by that and the appended Reply-To.  And some people, including
you, still aren't getting that, even though it got re-explained
yesterday, encore une fois.



Ahh... missed that.  Didn't really notice anything until this huge 
string of emails.  Sigh...






If one runs a list, and wants folks on gmail, AOL - any service that
honors p=reject - then one has to:

1. adjust headers so that list mail appears to originate from the
list manager, not from the original author

Yes.  But only where p=reject or p=quarantine..



Yes, indeed.




2. publish DKIM & SPF records for the machine hosting the list

No, there's no obligation for either of those, to ensure deliverability
of mail munged by the MLM software for DMARC-mitigation purposes.



True, but it sometimes helps.  And it's easy enough if one has access to 
one's nameserver records, as anyone who runs a list manager usually does.






Updating the mailman settings, and publishing the appropriate DNS
records, is really a no brainer for any halfway competent list
administrator.

Indeed, the Dng listadmins did exactly that, on Dec. 6th, following
instructions I provided.  The whole thing was covered here at the time,
so I'm puzzled that so many people seem to have failed completely to
read the explanations.



Sorry to add to the noise - there was just so much of it.  My bad.

Cheers,

Miles

--
In theory, there is no difference between theory and practice.
In practice, there is.   Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Simon Hobson (li...@thehobsons.co.uk):

> Perhaps I'm missing something, but doesn't SRS provide a gaping wide
> chasm for spammers to pile through ?

I would call _gaping_ chasm an exaggeration -- but it is certainly
abusable (to the extent cross-domain aliases become known or
discoverable in public).  

Someone trying to send Don Marti spam via alias 'd...@linuxmafia.com'
will implicitly rope my linuxmafia.com MTA (mail transfer agent = SMTP
daemon) into the evil task of pumping spam delivery attempts at Don's
zgp.org MTA, a regrettable case of 'Let's you and him fight' -- which is
why I've just now permanently disabled (now that I remembered the
problem) all cross-domain /etc/aliases entries.  (I've retained
intradomain aliases, such as ones that send root@, postmaster@, abuse@,
and hostmaster@ to the appropriate user mail spool.)

Relevant to this picture is the _other_ difference between MLMs (mailing
list managers) and other SMTP mail reflectors:  MLMs are _smarter_,
giving opportunities to reject or sequester abusive mail patterns the
other reflector types cannot.  E.g., even by default, GNU Mailman will
intercept and hold or reject mail with too many recipients, overly large
mail, mail implicitly addressed (mailing list address specified only in
Bcc), and a number of other similar heuristics.  Also, in this decade,
almost no mailing lists pass through without review mail from
non-subscribed addresses -- and spammers have still shied away from
making their spambots go through 3-way confirmation to join mailing
lists before trying to post to them.

So, yeah, you have a point, and I thank you for the reminder that I was
long overdue to disable those aliases.

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Simon Hobson]

Rick Moen  wrote:

> Back in the day, I gave out /etc/aliases entries to friends that
> leveraged the 'mafia' theme of my linuxmafia.com domain,

In our case it was simple alias entries ina  database queried by Postfix - but 
same effect and same problem.

> SRS (sender rewriting scheme) was SPF creator Meng Wong's kludge for
> salvaging /etc/alias and ~/.forward (when used cross-domain) from
> unintended collateral SPF damage.

Perhaps I'm missing something, but doesn't SRS provide a gaping wide chasm for 
spammers to pile through ? It always seemed to me a bit like server C getting a 
header that's been re-written in scuh a manner by server B that server C is 
expected to accept it as though server B is pinkie swearing that the forwarded 
mail is genuine and did come from server A. Or more precisely, server B 
effectively saying "this message from some other domain, well pretend it's 
coming from my domain"- so all a spammer has to do is forge (in a correct 
manner) the re-written from address and the spam bypasses SPF.
I guess that's why DKIM etc came along.

> Wong provided a Perl wrapper script to rewrite the SMTP envelope on the 
> outbound copy, emulating what MLMs do.

it was a few years ago now, so details are "a bit fuzzy" to say the least. In 
our case using Postfix, it needed some plugin to do it - and I think this 
plugin re-wrote all addresses regardless of where the email was headed. Due to 
the way the two services were done, the greylisting (part of policyd, aka 
Cluebringer) was done on the re-written address, and since this (IIRC) changed 
each day then few emails ever got the "seen this triplet before, straight 
through" treatment and so nearly all mail was delayed. Funny how users get to 
expect "instant" email even though there's never ever been any guarantee of 
instant delivery :-/

But at least my service did something that apparently the likes of Google and 
Microsoft couldn't manage - I did not have to silently delete mail that failed 
spam or embedded nasties checks. I rejected the messages so that any properly 
configured server would notify the sender that the message wasn't delivered. I 
was always proud of that bit.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Simon Hobson (li...@thehobsons.co.uk):

> Correction noted.

I truly appreciate your hearing it in the spirit intended.  Thank you,
Simon.  (We'll want to wind this up soon, as it has little to do with
Devuan, and discussion has veered away from Dng's adoption of Mailman
DMARC-mitigation munging starting 2018-12-06.)

> However, in my defence my issues (which I no longer have to deal with)
> were with mail forwarding in servers rather than mailing lists (IIRC
> our mailing list hosting had dwindled to just a couple of announce
> lists before the problem raised it's head) - so a different set of
> related issues which was primarily SPF at the time. I did get as far
> as having a look at SRS - but unfortunately the plugin for Postfix was
> incompatible with the greylisting I used due to the order of
> operations which prevented whitelisting of "known" greylisting
> triplets.

I think I know/remember a little bit about this.

Not all forwarding is alike.  MLM (mailing list manager) forwarding, the
operation where the MLM retransmits a received post to each subscriber,
involves writing an entirely new SMTP envelope on the outgoing
subscriber copies.  Other forwarding mechanisms such as /etc/aliases and
~/.forward entries do _not_.  Those just hurl the received message back
out with envelope unchanged.  (SRS was a kludge proposed for non-MLM
forwarders on account of this difference to help them preserve SPF
validity, a matter I'll return to shortly.).

Back in the day, I gave out /etc/aliases entries to friends that
leveraged the 'mafia' theme of my linuxmafia.com domain, e.g.,
'c...@linuxmafia.com' reaches Chris di Bona, then a co-worker at VA
Linux Systems and now Linux Community Manager at Google.
'd...@linuxmafia.com' was a natural fit as an alias for _Linux Journal_
editor Don Marti's personal mailbox dma...@zgp.org.  And so on.
However, following wide adoption of aggressive hardfail SPF policies,
those and all other cross-domain /etc/aliases entries more-or-less broke
(well, became selectively unreliable, depending on the sending domain),
because any mail transiting the alias would arrive at the other-domain
end-destination unable to pass SPF scrutiny for the claimed sending
domain, which in turn was because my MTA at linuxmafia.com wasn't in the
sending domain's SPF-published list of authorised sending IPs.

SRS (sender rewriting scheme) was SPF creator Meng Wong's kludge for
salvaging /etc/alias and ~/.forward (when used cross-domain) from
unintended collateral SPF damage.  Wong provided a Perl wrapper script
to rewrite the SMTP envelope on the outbound copy, emulating what MLMs
do.

At the time, I couldn't be bothered reimplementing all of those
cross-domain /etc/aliases entries using an SRS wrapper, so they have
simply become not-very-reliable reflectors, and what I tell users is 
'/etc/aliases and ~/.forward are no longer best practices for
cross-domain mail redirection, unless you're willing to do more work
than I personally am volunteering for.'

But the point is that MLM-redirection, by contrast, never had that
problem because of its smarter way of handling the envelope header.

In hindsight, SRS-wrapping seems like small potatoes compared to the
order-of-magnitude-greater hassle of DKIM and DMARC (but I personally
elect to eschew all three).

-- 
Cheers,"He who laughs last, lasts."
Rick Moen   -- Leo Rosten
r...@linuxmafia.com
McQ! (4x80)
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Simon Hobson]

Rick Moen  wrote:

> Simon, I appreciate your pitching in to attempt to answer this question.
> A few necessary corrections, though:

Correction noted. However, in my defence my issues (which I no longer have to 
deal with) were with mail forwarding in servers rather than mailing lists (IIRC 
our mailing list hosting had dwindled to just a couple of announce lists before 
the problem raised it's head) - so a different set of related issues which was 
primarily SPF at the time. I did get as far as having a look at SRS - but 
unfortunately the plugin for Postfix was incompatible with the greylisting I 
used due to the order of operations which prevented whitelisting of "known" 
greylisting triplets. Customised solutions were beyond my skill set - not to 
mention, the issues of leaving a maintenance time-bomb for any admin taking 
over*.

* When I left, a host developed a hardware issue. There was enough spare 
capacity to simply move the VM to another host - a few hours to copy the mail 
folders. Instead the know it all in charge took nearly a week to get something 
working because the concepts were beyond him. It was hard to laugh out load as 
I knew what it would be doing to the customers - many of whom I knew personally 
through having provided support over the years.


Rick Moen  wrote:

> Why messages fail DMARC is convoluted, and I'd frankly rather spend my
> time on other things.  If you are wanting to spend a lot more time on
> this, here's a fine place to start:
> https://wiki.asrg.sp.am/wiki/Mitigating_DMARC_damage_to_third_party_mail

Thanks for that, an interesting site.



Steve Litt  wrote:

> I'd suggest we ban email from gmail, yahoo, protonmail, and the rest
> that demand strict adherence to DMARC.

Nice thought, but do you really think that the likes of Google give a sh*t 
about some little mailing list somewhere, and which should be using Google's 
services anyway - how dare they use their own solution !
The reality is that the "big boys" have implemented these breakages - they knew 
beforehand that they would break almost all forms of forwarding, but their 
solution to that "problem" was simply to declare any form of mail forwarding as 
"improper" and therefore breaking it wasn't their fault. I can't help thinking 
that their marketing people saw an opportunity to make life harder for small 
scale competitors.

From the users' PoV, if a random mailing list or forwarding server doesn't work 
with such broken domains then clearly it has to be the little mailing list or 
forwarding server that's broken. For many years at a previous job we ran a mail 
server for customers - going back to before everyone and his dog were offering 
such services. We always recommended customers to create a second account in 
their mail software to (at a minimum) collect their mail - but many would 
simply refuse to countenance the complication - and instead we had to forward 
"i...@customersdomain.co.uk" to "someobscureaddress24673...@isp.com".
This worked just fine for many years - until that is, the big boys went out and 
broke it.


___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting Miles Fidelman (mfidel...@meetinghouse.net):

> Speaking as someone who hosts a couple of dozen email lists, I
> really don't understand what the fuss is about here.

The fuss involved people having paid no attention to the announcement of
Dng's DMARC-mitigation munging starting on December 6th, and so being
confused by that and the appended Reply-To.  And some people, including
you, still aren't getting that, even though it got re-explained
yesterday, encore une fois.

> If one runs a list, and wants folks on gmail, AOL - any service that
> honors p=reject - then one has to:
> 
> 1. adjust headers so that list mail appears to originate from the
> list manager, not from the original author

Yes.  But only where p=reject or p=quarantine..

> 2. publish DKIM & SPF records for the machine hosting the list

No, there's no obligation for either of those, to ensure deliverability
of mail munged by the MLM software for DMARC-mitigation purposes.  

> Updating the mailman settings, and publishing the appropriate DNS
> records, is really a no brainer for any halfway competent list
> administrator.

Indeed, the Dng listadmins did exactly that, on Dec. 6th, following
instructions I provided.  The whole thing was covered here at the time,
so I'm puzzled that so many people seem to have failed completely to 
read the explanations.

> The folks who administer lists.dyne.org just need to do it.

{headdesk}

I'm sorry, but how were the several explanations of this matter,
including those just yesterday, unclear?

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[Rick Moen]

Quoting info at smallinnovations dot nl (i...@smallinnovations.nl):

> So far i have just installed DMARC one time but if i remember it
> correctly either SPF or DKIM had to be correct to accept the e-mail or
> not. To quote my source "A message will fail DMARC if the message fails
> both (1) SPF or SPF alignment and (2) DKIM or DKIM alignment.". DKIM
> would be a hard nut to crack for MLM but SPF should not be a problem
> then? Or do I overlook something?

Why messages fail DMARC is convoluted, and I'd frankly rather spend my
time on other things.  If you are wanting to spend a lot more time on
this, here's a fine place to start:
https://wiki.asrg.sp.am/wiki/Mitigating_DMARC_damage_to_third_party_mail

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Admins can you fix/set the header overrides?

[info at smallinnovations dot nl]

On 27-12-18 03:20, Rick Moen wrote:
>
> {sigh}  Nobody listens.
>
> There is nothing needing a 'fix' (unless you wish to argue with
> operators of domains publishing aggressive DMARC policies (p=reject or
> p=quarantine) and convince them that such is an unwise policy).  In a
> world where DMARC is being rolled out by many domains, mailing lists can
> either attempt to mitigate the DMARC-caused damage, or do nothing and
> let some subscribers and their readers figure out the hard way why
> certain posters aren't being received at certain domains (e.g., GMail)
> and are getting gradually auto-unsubscribed by Mailman on account of
> excessive 'bounce scores'.
>
So far i have just installed DMARC one time but if i remember it
correctly either SPF or DKIM had to be correct to accept the e-mail or
not. To quote my source "A message will fail DMARC if the message fails
both (1) SPF or SPF alignment and (2) DKIM or DKIM alignment.". DKIM
would be a hard nut to crack for MLM but SPF should not be a problem
then? Or do I overlook something?

Grtz.

Nick




signature.asc
Description: OpenPGP digital signature
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng