Re: [mailop] +addressing ... any reason to NOT use it? {dkim-fail}

[Ned Freed via mailop]

> I try to use +addressing whenever I sign up for something new.  I'd say at
> signup it works like 95% of the time. Plenty of forms still don't think
> the + is a valid thing to find in an email.  I've had 3-4 retailers so far
> (LandsEnd was one of them I think) who originally let me setup an account
> with the + in it, but then when I went to log into the account after the
> fact, wouldn't let me in.  One of them only disliked it on mobile.  So
> there I was with a valid account I couldn't get back into with the address
> I used.

Paypal did this - used have no problem with +, then they did. And they didn't
let you change your email address.

> PITA, still not sure if its worth it.

It's wonderful when dealing with political campaigns. Use a different subaddress
for each, and you can tell who is buying/selling lists.

And if they don't support it, well, you didn't really to donate to that
campaign, did you?

Ned
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] +addressing ... any reason to NOT use it? {dkim-fail}

[Ned Freed via mailop]

> It seems I missed the announcement, but ...

>   Plus Addressing in Exchange Online | Microsoft 
> Docs

> So ... Achievement Unlocked!
> It's now supported in Office365 as well as, "HotMail"!
> Is there anyone else who does NOT support it at this point?
> Are there any major senders who don't like a "+" in the username of an email 
> address?
> And if so, why?

Speaking as someone who has been using +subaddress forms extensively for >25
years, the main problem is broken web forms that don't handle +'s well. Maybe
now that Exchange online supports it there will be some impetus to fix them. Or
not.

> Valuable tool, to say the least.

Very.

Ned
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] DMARC policy application {dkim-fail}

[Ned Freed via mailop]

> On 2020-05-07 11:04 BST, Evan Booyens via mailop wrote:
> > Often the hacked account is used to send mail from a spoofed domain.

> Why would an email provider permit its users to spoof either the
> mail-from or the header-form?

Mailing lists work a lot better if you don't mess with the header-from.

Ned

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Reasons to add plain text alternative to email? {dkim-fail}

[Ned Freed via mailop]

> I want to thank all contributors to this discussion for their feedback.

> What I learned from this:
> - There are still people that prefer plain text.
> - Text alternative may be used by accessibility tools.
> - Text alternative should be complete and readable.
> - For bulk mail html only should work just fine.

And perhaps most important of all, plain text may be used for indexing,
And proper indexing is essential in the mobile space.


Ned

> Thanks,
> Maarten Oelering
> Postmastery


> ___
> mailop mailing list
> mailop@mailop.org
> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Reasons to add plain text alternative to email?

[Ned Freed via mailop]

Generate the plain text alternative if you can. But if you can't, or aren't
sure you can, just send the HTML and don't generate the multipart/alternative
structure.

Far too many multipart/alternative messages are arriving these days with:

(a) Blank plain text parts.
(b) Plain text parts that say, "see the HTML part", "get a better client", and
similar crap.
(c) Plain text parts that are just a copy of the HTML.
(d) Plain text parts that are a "processed" version of the HTML, where
"processing" doesn't involve actualy removing the markup.

These practices cause problems even for clients that show the HTML part
unconditionally because some indexers will always use the plain text if it is
present, on the theory that the whatever generate the message is in a better
position to generate proper text.

Ned

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Respecting MTA-STS {dkim-fail}

[Ned Freed via mailop]

> If we want to try and respect MTA-STS, when doing STARTTLS, the sender
> needs to send the right information in the TLS SNI (Server Name
> Inidication) extension. An MTA-STS-honoring SMTP client expects to
> validate the X.509 certificate of the receiving MTA, but that MTA might
> be known by a dozen names, unless the SNI is provided.

The key point here is "respect MTA-STS". If I have MTA-STS set up for
my SMTP server, it's incumbent on me to also have an appropriate
TLS stack in place.

> For example, if i'm trying to reach out to mail.example.biz but it
> happens to also serve mail.example.com on the same address at port 25, I
> definitely need to tell it which hostname i'm looking for, so that the
> server can offer me the mail.example.biz certificate instead of the
> mail.example.com certificate.

Actually, that's not how it works. In the case of MTA-STS the client is
required to put the MX host the client connected to in the SNI extension. (RFC
8461 section 7.1.) The same set of MX hosts can be (and frequently are) used
for multiple domains, so it's entirely possible for the use of the SNI
extension to be unnecessary.

> In some MTA implementations, such as Postfix 3.4, there is a parameter
> (smtp_tls_servername), which sends the value to the remote SMTP server
> in the TLS SNI extension.

> However, the documentation seems to suggest that there could be problems
> with this parameter:

> http://www.postfix.org/postconf.5.html#smtp_tls_servername

> Some SMTP servers use the received SNI name to select an
> appropriate certificate chain to present to the client. While
> this may improve interoperability with such servers, it may
> reduce interoperability with other servers that choose to abort
> the connection when they don't have a certificate chain
> configured for the requested name. When in doubt, leave this
> parameter empty, and configure per-destination SNI as needed

> Does anyone have any statistics of how frequent of an occurrence this
> actually is, is it actually such a major problem that turning this on
> will cause significant issues?

Why do you think such a statistic would be relevant? Clients only need to use
the SNI extension when engaging in MTA-STS - and if a server engaging in
MTA-STS isn't properly configured in regards to the necessary certificates
there's going to be problems regardless.

> It seems like Gmail does send SNI, likely
> unconditionally, since it attempts to negotiate TLS1.3, where SNI is
> expected. This suggests that it is likely safe to send SNI, but it would
> be good to find out.

First, there's no requirement that MTA-STS servers support SNI unless they are
responding to multiple MX host names. And if they are, they have to have valid
certificate chains for all of those host names, SNI or no.

Failing to transfer mail when these conditions aren't met is kind of the point
of MTA-STS.

> Those of you running MTAs, can you gather from your logs all the servers
> you've connected to in the past, and attempt to connect to them with SNI
> and collect those which abort connections, so we can find out what needs
> to change to make this parameter safe to enable? Does anyone have any
> contacts at any of the mail gamification sites that can add this check?

Enabling this parameter isn't the same as supporting MTA-STS. There's quite a
lot more involved on the client side.

Ned

> If we wish to respect MTA-STS, we need to get servers who are doing this
> to stop doing this.

> Sorry for postfix-users, who have already heard this, I wanted to reach
> a wider audience.

> --
> micah

> ___
> mailop mailing list
> mailop@mailop.org
> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail - Multiple destination domains per transaction is unsupported {dkim-fail}

[Ned Freed via mailop]

> I don't quite get this. Your outbound MTA is grouping separate domains
> together into one queue based on MX?

There are various ways to do it, but the basic idea is to reuse connections
and even transactions based on different domains translating to a common
set of MXes, for some defintion of "common".

Variations include doing it for all messages, for specific MX lists, for
domains who MX lists contain a specific subset MX list, by MX IP, etc.

Some people swear by these mechanisms. Others swear at them.

> Or is it trying to relay some other mail through Google when at least
> one recipient is hosted on a Google MX?

That would be very broken, and AFAIK nobody does that.

Ned

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] HEADER LENGTH as per RFC2822 {dkim-fail}

[Ned Freed via mailop]

> In article <75f5a58a-f53f-5222-9e3a-791f21594...@linuxmagic.com> you write:
> >>> "An unfolded header field has no length restriction and
> >>>therefore may be indeterminately long."
> >>
> >> You're right.  I missed that.
> >>
> >> But I suspect I'm not the only one who missed it, so I would try to keep 
> >> headers under 998
> >> anyway.  It usually isn't hard.
> >>
> >
> >Sure that isn't a typo in the RFC? Makes for more sense to say, a
> >'folded header field has no length restriction' eg, many lines long
> >long.. (Didn't read the context around it, just replying off the cuff)

> The changes between RFC 2822 and 5322 were kept as small as possible,
> but that sentence was new.  I'm pretty sure it means what it says.

The 998 octet limit in RFC 2822 was added fairly late in the game because some
people were saying that the lack of any limit in the format meant you were free
to have lines of any length as long as transport restrictions weren't a factor.

That language was in turn misinterpreted as a limit on unfolded header fields.
That was when the no limit on unfolding language was added.

In terms of unfolded header length, a long To: or Cc: list can easily exceed
998 octets, and RFC 5322 doesn't allow the use of multiple fields. I get
multiple messages every day containing such headers.

Even ancient versions of sendmail support unfolded lengths of 4K. Expect
trouble if you're not allowing for at least that much.

There are also military applications that require a complete recipient list in
the header. I've seen some of those that were regularly 2Mb or more in length.

Ned

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Gmail & TLS SNI {dkim-fail}

[Ned Freed]

> In MX delivery without DNSSEC, if Eve injects an MX record:

>   gmail.com. IN MX 1 my-spy-agency.example.org.

> then using the hostname from DNS means that the client will happily go
> talk to my-spy-agency.example.org, using that as the SNI, and validating
> against that same domain, then present back an "all's good here boss"
> status, saying that it's happily confirmed the identity of gmail.com in
> validation.

> The SNI identity should match the identity to be validated, else there's
> no point doing any validation.  Having clients willing to send SNI which
> they're not willing to accept as a valid value for verification is
> broken.  Since the client can't a-priori know which is which (legitimate
> gmail.com server or Other), when using hostnames from insecure DNS, the
> client shouldn't send SNI unless and until it has an identity which it's
> willing to validate.

> If using DANE, or if using MTA-STS where the hostname from DNS has been
> vetted against the whitelist from MTA-STS first, then everything changes
> and SNI becomes important.

AFAIK this does not happen in MTA-STS, that is, at no time is the MX hostname
obtained from the DNS checked against the "mx" list from the MTA-STS policy.
Rather, the DNS-ID of the certificate returned by the server is checked against
the "mx" list from the MTA-STS policy. This means that the mx hostnames may not
align with the certificates.

If you believe otherwise, I'd appreciate a pointer to where in the
specification it says that MX hostnames are supposed to be checked against
the "mx" list.

Ned

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] question regarding support for international characters {dkim-fail}

[Ned Freed]

> Besides, I have a sneaking suspicions that those who take the step of offering
> addresses in scripts that have these issues are going to be so busy dealing
> with visual similarity, address fakery, and similar issues that we'll be
> lucky if they do any sort of normalization at all, let alone dive deep into
> the rabbit hole.



My impression is that for the forseeable future most of the EAI interest
will be in India and maybe China where the similarity issues are
different, so in practice you're right, the funkiness of all the languages
written in Latin characters will get no attention.


Curious, isn't it, that the MSP EAI support we've talked about here is
exclusively for other people's addresses, not for their own users?

Or maybe it isn't curious at all, and the really hard part of this problem
is provisioning for arbitrary scripts.

In any case, I'm quite pleased that provisioning is beyond my purview.

Ned

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] question regarding support for international characters {dkim-fail}

[Ned Freed]

>> The Gmail and Hotmail support handles other people's UTF-8 addresses
>> in mail but they still don't provide UTF-8 addresses on their own
>> systems.
>
> From what I can tell, Gmail and outlook.com's support is basically "just send
> UTF-8", that is, it will send EAI messages without the server offering the
> extension.



I know the people involved and can check.



> I agree that this isn't difficult. What's difficult is keeping track of the
> EAI-ness of a message as it goes through processing like alias expansion, 
which
> can turn an non-EAI message into an EAI message or vice versa.



> Support for the nested encodings message/global creates may also be
> nontrivial.



I don't even try.  In the places where it matters, I scan the envelope and
message headers for characters with the high bit set.


This isn't a place where you care about message/global, since the presence
of a message/global object doesn't make the message an EAI message. Indeed,
the only good thing about message/global is that it presents as an opaque
container that allows you to tuck one EAI message inside another without
requiring transport level EAI processing.

But this doesn't mean that MTAs, or more precisely MSAs, can get away with
not processing message/global. Message fixups on SUBMIT need to reach
inside, and since some of them may be legally mandated, it's not something
we can ignore.

As for checking for high bits in the outer header, that ignores
utf-8 in nested MIME parts. It isn't clear to me what the consequences of
that are going to be.


This is wrong, but
it doesn't seem much wronger than far more complex approaches.  Haven't
thought too much about message/global but in the MTAs I use, it's only a
MUA problem.


MTAs, maybe. But your typical MTA also acts as an MSA.


>> The hardest part, which I haven't done yet, is generalizing
>> the address mapping that MTAs do on incoming mail. ...



> This I frankly don't care about, as I believe that doing it in a meaningful
> language-specific way is impossible.



I meant interpreting addresses in mail to my own mailboxes, the
generalized version of case folding and subaddresses.  Maybe you're right
that undotted i's won't work in a lot of places, but I'd be surprised if
they didn't work in Turkey.


First of all, undotted i's are going to work everywhere as long as you don't
expect to be able to switch case. We're only talking about having things work
when someone makes a case change when entering an address.

And in such cases, prepare to be surprised, because the odds are astromonically
small that it will work. Even in Turkey.

It's possible - I think - to code a solution where different case conversions
are applied on a per-domain basis. And ISPs/MSPs might even be willing to
run such software preferentially and configure it correctly for
their own domains.

But what about other Turkish domains? Is someone going to maintain a list of
them all so that address matching can be done properly? And will it be kept up 
to date?


And even if this all came to pass, do per-domain case rules actually work?
Are domain aligned this way? I'm fairly sure the answer to this question is
"no".

And while considering all of this, keep in mind that that standards punted on
this issue. There's no requirement that any normalization be done, let alone
provide case-insensitivity.

Besides, I have a sneaking suspicions that those who take the step of offering
addresses in scripts that have these issues are going to be so busy dealing
with visual similarity, address fakery, and similar issues that we'll be lucky
if they do any sort of normalization at all, let alone dive deep into
the rabbit hole.

Ned


___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] question regarding support for international characters {dkim-fail}

[Ned Freed]

> In article 
> 
>  you write:
> >-=-=-=-=-=-
> >-=-=-=-=-=-
> >-=-=-=-=-=-
> >
> >Hello folks
> >
> >I've been tasked with finding out what the general consensus is on the
> >support in email headers for International characters such as  UTF-8
> >Charcacters and including things like accented characters like � and � and
> >can also include Asian and Cyrillic characters.
> >
> >I know there's an RFC from 2012, but my Product Dev people are interested
> >in knowing how wide-spread the actual adoption is.

> Funny you should ask.  I'm doing some work for the UASG group to document how
> internationalized email (known as EAI) works.

> UTF-8 in everything except the actual addresses can be in MIME body
> parts and encoded-words in mail headers.  Those have been around for
> at least a decade and should work everywhere.

For some value of "work", yes. While the risks of the so-called "mailsploit"
vulerabilities were overblown, encoded-words still have issues, even after all
this time.

The good news is that it's been a while since I've gotten a report of an issue
elsewhere in MIME.

> RFCs 6530-6533 defined an SMTP extension called SMTPUTF8 which, to
> oversimplify a little, allows UTF-8 anywhere you can have ASCII,
> including in both the local part and the domain part of the addresses.
> This modifies both the messages themselves and the address in the
> SMTP dialog MAIL FROM and RCPT TO.

It also imposes an end-to-end support requirement, that is, everything along
the MUA->MSA->1*(MTA->)MDA->MS->MUA path must be upgraded to support EAI. And
this requirement exists even when none of the current recipients of the message
are EAI recipients, due to the possible presence of EAI addresses in the header
and MAIL FROM.

This is proving to be highly problematic in practice. And the additional
support beyond what the standard requires needed to take care of it properly in
an operational context is IMO nontrivial.

> Uptake has been slow, but Gmail quietly added support last year, and
> Hotmail/Outlook/Live added support about a month ago.  Some of the
> large Chinese services like Coremail support it as do some Indian
> services like Xgenplus.  Yahoo/AOL/Oath have as far as I can tell no
> plans to support it.

Not to sound like a broken record, but... for some value of "support", maybe.

> The Gmail and Hotmail support handles other people's UTF-8 addresses
> in mail but they still don't provide UTF-8 addresses on their own
> systems.

>From what I can tell, Gmail and outlook.com's support is basically "just send
UTF-8", that is, it will send EAI messages without the server offering the
extension. The only time it checks is when the actual recipient address is EAI
- in that case the extension is required. But if the message is EAI only
because of EAI headers, they just blast away.

Now, you can certainly argue that "just send it" is the "right" thing to do.
Average users don't know what's ASCII and what's not, and don't care to know.
They just want their mail delivered. So when they send a message to two people,
one with an EAI address and one with a regular address, and one copy bounces
for no obvious reason, I'd have to call that a pretty serious violation of the
least astonishment principle.

Nevertheless, it's contrary to what the standards require - that the message
bounce - so calling it "support" is a bit of a stretch.

As far as open source MTAs go, the support in Postfix appears to be standards
compliant - EAI messages sent to non-EAI recipients are bounced. See:

  http://www.postfix.org/SMTPUTF8_README.html

for specifics.

FWIW, I think the correct thing to do is downgrade EAI messages sent to non-EAI
recipients on non-EAI hosts, but doing this is a significant PITA for a bunch
of reasons I'm not going to get into now. Also FWIW, this is what our MTA
defaults to, but we also support "reject" and "just send it" in case
someone wants those behaviors.

> It is my impression that the main interest is currently in
> India since some bits of the government are planning to hand out
> e-mail addresses to go with the biometric IDs, and a lot of Indians
> are literate in their own languages, which are written in their own
> scripts, but not English.

Which may be sufficient to induce the larger MSPs and ISPs in the US and Europe
to upgrade. But not the smaller ones, who Just Don't Care. Maybe, if
all of the open source MTAs and message stores start supporting EAI, they
can be asked to flip the necessary switches. But I'm not optimistic.

Amusingly, the thing that may drive US adoption is the desire to put emoji in
addresses, which seems to be widespread. Sigh.

> Having recently written EAI support into my own qmail system I can say
> that the basic address handling was a lot easier than I expected,
> since most mail code these days is already 8-bit clean sort of by
> accident.

I agree that this isn't difficult. What's difficult 

Re: [mailop] Anybody have a pointer to a clued qwet.net mail person?

[Ned Freed]

> One of my users is seeing repeated fails to send to hosted mail for nsf.gov
> from qwest.net:

> first MX:

>  ... while talking to stn-mtpe-01-03-p.inet.qwest.net.:
> >>> DATA
>  <<< 451 4.3.2 Please try again later
>  ... Deferred: 451 4.3.2 Please try again later
>  <<< 503 5.0.0 Need RCPT (recipient)

This is actually both a client and server quality issue. When pipelining is
supported (and this server does support it) the client may send a DATA command
before confirming that any RCPT TO command is accepted. And when the RCPT TO is
reject, as it was here, the server is supposed to reject the DATA command. RFC
2920, section 3.2:

 (3)   SHOULD issue a positive response to the DATA command if and
   only if one or more valid RCPT TO addresses have been
   previously received.

The RFC doesn't specify whether the negative response should be 4yz or a 5yz
(it should - my bad), but since the underlying error is "no valid recipients"
use of a 5yz is not unreaonable.

It follows that (a) A high quality client should be able to deal with the use
of 5yz in this case, and respond according to the RCPT TO error, and (b) A high
quality server will keep track of whether or not a temporary failure was issued
to any RCPT TO, and if it was issue a 4yz no valid recipients error.

> second MX:

>  ... while talking to cec-mtpe-01-03-p.inet.qwest.net.:
> >>> DATA
>  <<< 554 5.7.1 Access denied
>  554 5.0.0 Service unavailable
>  <<< 503 5.0.0 Need RCPT (recipient)

Of course you don't know the reason for the rejection, but this does look
bogus.

Ned

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Storing 821 envelope recipients in an 822.Header?

[Ned Freed]

> /me is going to go with Envelope-To, as it's going to be the easiest to
> explain to users "this is from the envelope at SMTP delivery time, not the To:
> or Cc: or anywhere else".

FWIW, we chose the closely related X-Envelope-To: for this function many years
ago. (At the time best practice was to use X- prefixes on nonstandard headers.)

If we were doing it today we'd use Envelope-To:.

Ned

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Re: [mailop] Anyone from AOL on this list?

[Ned Freed]

> I just started seeing this:
>   Site aol.com (152.163.0.68) said after data sent: 421 4.2.1 Dragnet 
> Timeout
>   Site aol.com (152.163.0.99) said after data sent: 421 4.2.1 "Service 
> unavailable. Please try again later."

> Anyone else?

I've seen the second one of these once. There was no preceeding feedback
report, just a bunch of bounced list messages.

I filed a service ticket asking what this meant and why wasn't there a feedback
report. A few days later I was told that the problem was fixed, and that if I
wanted to avoid this in the future I needed to set up a feedback loop, see this
URL for an explanation of how to do that.

Sigh.

Ned

___
mailop mailing list
mailop@mailop.org
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop