On 8/21/20 6:37 AM, Rich Freeman wrote:
This stuff can be interesting to discuss, but email is SO entrenched
that I don't see any of this changing because of all the legacy issues.
You would need to offer something that is both easier and better to
use to such a degree that people are willing to change.
"something that is /both/ *easier* and *better*".
That's a VERY tall bar to clear.
1. Modernizing the actual data exchange using http/etc.
I don't think you'll get much argument that SMTP isn't the way
somebody would do things if they were designing everything from
scratch.
SMTP may not be the best, but I do think that it has some merits.
Merits that the previously mentioned HTTP/2 alternative misses.
Remember, SMTP is a protocol, or a set of rules, for how two email
servers are supposed to exchange email. These rules include the following:
1) Proper server greeting, used to identify the features that can be
used. (HELO for SMTP and EHLO for ESMTP.)
2) Identifying the sender.
3) Identifying the recipient(s).
4) Sending the message payload.
Each of these discrete steps is a bi-directional communication.
Communication that can alter all subsequent aspects of a given SMTP
exchange.
The server could be an older server that only speaks SMTP and as such
doesn't support STARTTLS. Thus causing the client to /not/ try to use
STARTTLS in accordance with current best practices. Other ESMTP
features come into play here like message size, 8-bit, notification
features, etc.
The receiving server has the ability to assess the sending server's
behavior at each point to use that as an indication to know if the
sender is a legitimate email server or something more nefarious and
undesirable. -- There is a surprising amount of email hygiene based on
sending server's behavior.
The receiving server may carte blanch reject messages from specific
senders / sending domains and / or specific recipients, particularly
non-existent recipients.
SMTP has low overhead in that the message body is not transferred from
the sending server to the receiving server if any part of steps 1-3
aren't satisfactory to the receiving server.
I'm not an API expert but I imagine that just about any of the modern
alternatives would be an improvement.
Maybe. Maybe not.
What is the actual problem with SMTP as it is?
What /need/ to be improved? What could benefit from improvement even if
it's not /needed/?
http would be a pretty likely protocol to handle the transportation,
Why HTTP?
Did you mean to imply HTTPS?
Why add an additional protocol to the stack?
TCP / SMTP is two layers.
TCP / HTTP / $Email-protocol-de-jure is three layers.
UDP / HTTP / $Email-protocol-de-jusre is three layers.
Why introduce an additional layer?
Why introduce an additional dependency in the application stack?
Why co-mingle email with other non-email traffic? -- Or are you
wanting to run HTTP(S) on TCP port 25?
likely with something like JSON/xml/etc carrying the payload.
Why add an additional encapsulation on top of the additional layer?
What does JSON / XML / etc. on top of HTTP(S) provide that SMTP doesn't
provide?
What is the gain?
Is the gain worth the cost of doing so?
You might want to tweak things slightly so that recipient validity can
be tested prior to transferring data.
Definitely. (See above for more details as to why.)
But now you are doing multiple exchanges.
If we extrapolate this out to also include sender validation, and
probably hello messages, you are back to four or more exchanges.
How is this better than SMTP on TCP port 25? What is the benefit? What
does it cost to get this benefit?
A mail server doesn't want to accept a 20MB encrypted payload if it can
bounce the whole message based on the recipient or a non-authenticated
sender or whatever.
Which is one of the many reasons why there are multiple exchanges back
and forth.
However, in principle there are plenty of modern ways to build a
transport protocol that would be an improvement on SMTP ...
Please elaborate.
Please include what network layer (3) protocol(s) you want to use and why.
Please include what application layer (7) protocol(s) you want to use
and why.
Please include any encoding / encapsulation you want to use and why.
... use more standard libraries/etc.
There are many libraries to interface with SMTP.
Also, handing a message off to an SMTP server alleviates the application
from having to deal with failed deliveries, queuing, and retrying.
Why add that complexity into each and every application? Especially if
you don't have to?
Note: Pulling it in via a library is still indirectly adding it to each
and every application.
How is SMTP /not/ standard?
Also: https://xkcd.com/927/ -- Standards
I think this is actually the part of your proposal that might be
more likely to take off. You could have a new type of MX field in
DNS ...
So yet more complexity in addition to the extra layers of the stack.
... if it is set then it defines servers that will use the new protocol,
It defines servers that are purportedly /capable/ of using the new protocol.
Advertising a capability is entirely different than using an advertised
capability.
and you could have backup SMTP servers for legacy transition.
We've been transitioning from SMTP to ESMTP for about 28 years. How
long do you think it will be before we finish transitioning away from
stock SMTP?
Further, how long do you think it will be before we finish transitioning
to your new proposed standard? I'm guessing at least another 30 years
to finish it.
So how long will we have three email protocols; SMTP, ESMTP, and
$Email-protocol-de-jure?
You could change the transport side of things without redefining how
email works, so the same messages are interoperable with both systems.
So now you are proposing putting the SMTP protocol on top of a new
transport instead of TCP.
Old: TCP / SMTP is two layers
New: TCP / HTTP(S) / SMTP is three layers
New: UDP / HTTP(S) / SMTP is three layers
What exactly does moving SMTP from TCP to HTTP(S) provide? What is the
benefit for the added complexity and dependencies?
On the flip side, while I think this is an easier change to make,
I don't think it necessarily offers huge improvements.
So why bother doing it at all?
SMTP still gets the job done,
Yes. SMTP does it's job sufficiently well. Is it the best method?
Doubtful. Is it well supported and widely used? Yes on both accounts.
Note: The same can be said about Ethernet too. Yet here we are 30
years later still using it and enhancing it to be 40,000 times faster
than it originally was. (400,000,000,000 bps vs 10,000,000 bps)
and the new system wouldn't really be a major improvement from a
sysadmin standpoint, so I don't see everybody running out to change.
If there's not a significant* change, why should we bother changing?
Where significant is enough to overcome the work effort and pain to make
the change.
Just changing the transport protocols doesn't provide any anti-spam/etc
benefits.
Nope. So no benefit and additional complexity / overhead. Sounds like
a net loss to me.
SMTP already supports TLS so it isn't any more secure.
Just because (START)TLS is provided doesn't mean that it's used.
There may be some benefit to using a transport that has implicit
(START)TLS encryption.
We could easily revisit SMTPS, which does have implicit TLS encryption
on TCP port 465.
SMTPS is a well establish, but seldom used, way for clients to send
email to their outgoing mail server. Perhaps we could re-use this as a
new server-to-server SMTP transport. Slightly modifying and re-using
prior art is almost always better received than something completely new.
There is also the possibility of working on augmenting SMTP, or more
likely the metadata around it, to make existing SMTP servers to
/require/ STARTTLS when it is known that both the sending and receiving
servers support it.
This latter idea is already somewhat being done by many email
administrators. Admittedly it's sub-optimal and likely manual. But
that sounds like a simpler change. Perhaps publishing a flag in DNS
could indicate that email from my servers should be STARTTLS encrypted
and you should require it on the receiving server (presuming you support
it).
No doubt ...
I have serious doubt.
... if we were starting from scratch this is how it would be done, but
it isn't enough of a reason to change.
DING DING DING!!!
That said, if there were even modest benefits for large mail providers
I could see it taking off simply because it could leverage a lot of
existing libraries and standards, and could have legacy compatibility.
But we already have MANY libraries and MANY standards. What good does
yet another one provide?
Stop focusing on HTTP(S) <any version thereof> for a few minutes.
If you were to create a new protocol from the ground up, why would you
introduce additional layers into the application stack — which represent
additional dependencies and complications — if you didn't have to?
Not everything is a web server. Not everything can benefit from being
forced through a web server.
2. Changing the model around email moving to something more
PKI-based/etc and redefining what an email address is.
This is where you could have revolutionary improvements for
privacy/spam/etc.
What does changing email addresses (user part and / or domain part)
provide functionality that can't be done — as in is not at all possible
to do — with current email addresses?
However, it completely breaks everything that exists with email today,
so it is a hard change.
Changing email addresses doesn't break email as it exists today. /SMTP/
/doesn't/ /care/ what the addresses are.
/Humans/ /using/ /email/ (thus indirectly SMTP) /do/ /care/ what the
addresses are.
Most of the benefits only accrue if you adopt PKI as well, which is
a problem that is hard to scale out to the masses.
PKI is relatively easy to scale. (For a given value.) Hence the
thriving CA industry.
Doing so in a trust worthy manner is the problem. Especially if you
want to avoid a single point of control / failure / GEO political / etc.
If somebody comes up with a way to make PKI take off in a real way,
then I could see something like this taking off eventually. Otherwise,
this stuff is only going to be of interest to the sorts who use gpg
to sign their email, and nobody else.
The biggest issue I see with PKI is /bootstraping/ trust.
If we throw all the trusted CAs out the window, how do you /bootstrap/
encrypted communications? - My personal opinion is DNS.
We already have a very well understood, globally distributed, highly
redundant database with unique keys.
We have methods to authenticate it; DNSSEC.
We have ways to hid it's use; DNSCrypt, DNS-over-TLS (DoT),
DNS-over-HTTPS (DoH), DNS-over-53-over-IPsec (transport mode), and VPNs
(IPsec (tunnel mode), OpenVPN, WireGuard, et al.
So, why not put recipient's public keys in this phenomenal
infrastructure that we already have in place?
Oh, we did. There are already ways to publish S/MIME keys in DNS. I'm
fairly certain that there are ways to publish PGP keys in DNS too.
Non-conflicting ways at that.
We also have ways to publish the public key of a host's public / private
key pair.
The sad thing is that most people don't know that these well defined
methodologies exist and of those that do, few use them.
The long and short is that there are some relatively simple things that
we can do to augment the existing SMTP infrastructure to make it better.
We do not /need/ to do a wholesale replacement and incur all the
complications ~> technical debt that goes along with such a replacement.
--
Grant. . . .
unix || die
