Re: UCSPI-TLS for s6-networking?

2020-11-17 Thread Casper Ti. Vector
On Mon, Nov 16, 2020 at 02:05:27PM +, Laurent Bercot wrote:
>  If I needed to write an SMTP server that supports STARTTLS, the way
> I would do it would be the following:

I have also been thinking about a minimal implementation of STARTTLS MX
(I think this may be the only hard-to-remove instance of STARTTLS, since
the discontinuation of STARTTLS mail submission and retrieval is easily
supported by mail clients, and customer notices can be used to announce
the migration).

In my design, smtpd is a clear-text UCSPI application, which upon the
STARTTLS message exec()s into itself wrapped by s6-tlsd.  States are
passed across exec() through enviroment variables and command line
arguments (the latter only for non-sensitive information).  The client
does what is symmetric to those outlined above.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C



Re: UCSPI-TLS for s6-networking?

2020-11-17 Thread Amitai Schleier

On 16 Nov 2020, at 23:46, Laurent Bercot wrote:


So I'm not holding my breath or keeping my hopes
up that qmail will soon be usable as a user-friendly full-featured MTA
that does not need any patching, that builds easily, that is packaged
in mainstream distributions, and that will actually get mail through
any provider (keywords: SPF, DMARC, DKIM).


Those are indeed some of our goals 
(https://github.com/notqmail/notqmail/wiki#what-are-the-projects-goals) 
and some of the features we'll need to include 
(https://github.com/notqmail/notqmail/wiki#which-features-will-eventually-be-implemented) 
to accomplish those goals. You're right to continue respirating normally 
in the considerable meantime. ;-)  It might be, as you say, an insane 
amount of work. To me, it all feels incrementally reachable from where 
we are, and crazy not to try.


For any given desirable feature, I think about questions like

- Imagine that one or more simple implementations already exist. What 
would our code have had to look like to make this possible?

- What's the smallest possible interface required by this feature?
- How could the new code be fairly convincing at a quick glance?
- How will we automatically test it?

(I imagine these are familiar thoughts for you, too.)

By my lights, it seems possible-to-likely that the many changes notqmail 
needs to make will gradually get easier as we ratchet composability.



hardcoding a dependency to OpenSSL in the qmail-smtpd binary is a big
no-no.

[...]
I'd also like to break up qmail-remote to run its SMTP client under 
tcpclient, because then it could just as easily run under "sslclient 
-y" (if there were one, or the s6-networking equivalent), and then 
STARTTLS support would be simple to add to the client.


Yes, it would be pretty nice. That's one of the things that the 
notqmail

team should be working on. :)


Agreed. I feel fairly strongly that TLS for notqmail must not introduce 
a direct dependency on any crypto library; instead, we should remove any 
remaining custom networking code and expand our existing dependency on 
UCSPI. I've just recently written up one possible path for moving 
stepwise from DJB's qmail-remote to a more featureful one: 
https://github.com/notqmail/notqmail/wiki/Designs#outbound-starttls-auth-ipv6-etc


(I have vague early ideas for DKIM and SRS, mentioned at 
https://schmonz.com/qmail/rejectutils/future and 
https://schmonz.com/qmail/rfilter.)


So that's why I'm wishing s6-networking would provide an independent 
(and almost certainly better factored) implementation of UCSPI-TLS: 
it would give us a boost in our efforts to modernize qmail. But I see 
why you're not wishing to, too.


Well, again, STARTTLS is protocol-specific, so it's difficult to
support in a generic networking package. ucspi-tls sounds like a 
misnomer

to me; it's really ucspi-ssl + starttls-for-qmail.


I'm definitely not looking to wedge any SMTP-specific code into 
s6-networking! Even if I were, I wouldn't expect you to give it a 
moment's consideration. :-p


What I'm looking for in s6-networking is a tiny interface usable by 
_any_ UCSPI client or server, if and when _they_ decide it's time to 
upgrade a plaintext connection to TLS.


UCSPI-TLS is such an interface. (Please ignore the qmail bits of the 
Gifford-and-Brady patch, except insofar as it's useful to you to see how 
UCSPI programs could take advantage.)


notqmail's SMTP client and server would decide when to call it. Other 
UCSPI implementations of network application protocols with delayed 
encryption could do the same. s6-networking would continue to know 
nothing about any particular network application protocol.


s6-tlsclient and s6-tlsserver would continue to encrypt connections 
immediately by default. Given new command-line options, they'd start up 
unencrypted and wait to be told by the application if and when to switch 
to encrypted.


Does this make more sense as a request? I'm hoping so.

- Amitai


Re: UCSPI-TLS for s6-networking?

2020-11-16 Thread Laurent Bercot

fixsmtpio(8) sits in this spot and does some of these things. One way to be 
sure the SMTP server has reset all internal state after STARTTLS is to kill and 
restart it, architecture permitting. fixsmtpio does exactly this. :-)


 Yeah, that would work too. My proposed scheme kills the originally
spawned smtpd and connects to an existing tlsserver spawning its own
smtpd, but if you'd rather not have an existing tlsserver and directly
spawn a tlsd (or sslio or whatever it's called) + smtpd, it's also
valid.



- Port 587 (1) either has always-on TLS or requires STARTTLS before proceeding, 
(2) requires AUTH, (3) allows relaying, (4) other submission-specific minutiae

- Port 25 (1) permits STARTTLS, (2) disallows AUTH, (3) disallows relaying, (4) 
other incoming-specific minutiae

- From qmail's point of view, submission and incoming have been distinct 
services, often provided by distinct programs


 To be honest, I've never even tried submission with qmail. It required
a lot more patching and configuration that I was comfortable with. So
I'm not knowledgeable about all the details. But again, what I wrote was
only an outline, and if you're saying that protocol specifics make it a
bad match, then it can be changed, and the starttls wrapper can spawn
its own tls tunnel + smtpd instead of piggybacking to an existing
service.



I wrote fixsmtpio under the assumption that qmail would never have an active 
upstream, so it'd be cheaper to maintain my own somewhat complex standalone 
program than to carry around yet another patch. That assumption has since 
turned false.


 Yeah, well, color me unenthusiastic. It's a good thing, don't get me
wrong, but it's like, 20 years late, and the amount of work necessary to
bring qmail up to speed with the garbage truck that is e-mail in 2020
is absolutely insane. So I'm not holding my breath or keeping my hopes
up that qmail will soon be usable as a user-friendly full-featured MTA
that does not need any patching, that builds easily, that is packaged
in mainstream distributions, and that will actually get mail through
any provider (keywords: SPF, DMARC, DKIM).



 I now suspect it'd be cheaper to merge UCSPI-TLS logic directly into 
qmail-smtpd and ofmipd and get rid of fixsmtpio. But that puts a lot of weight 
on
ucspi-ssl.


 I also think it's the right thing, because STARTTLS is 
protocol-specific

so its handling belongs in the SMTP server. However, doing this right
requires configurability of the TLS layer executable, because
hardcoding a dependency to OpenSSL in the qmail-smtpd binary is a big
no-no.



I'd also like to break up qmail-remote to run its SMTP client under tcpclient, because 
then it could just as easily run under "sslclient -y" (if there were one, or 
the s6-networking equivalent), and then STARTTLS support would be simple to add to the 
client.


 Yes, it would be pretty nice. That's one of the things that the 
notqmail

team should be working on. :)



So that's why I'm wishing s6-networking would provide an independent (and 
almost certainly better factored) implementation of UCSPI-TLS: it would give us 
a boost in our efforts to modernize qmail. But I see why you're not wishing to, 
too.


 Well, again, STARTTLS is protocol-specific, so it's difficult to
support in a generic networking package. ucspi-tls sounds like a 
misnomer

to me; it's really ucspi-ssl + starttls-for-qmail. I can provide modern
versions of the ucspi-ssl programs, and if you need something special 
for

the starttls part, we can discuss it, but I would first need to know
exactly what your requirements are. :)

--
 Laurent



Re: UCSPI-TLS for s6-networking?

2020-11-16 Thread Amitai Schleier

On 16 Nov 2020, at 15:05, Laurent Bercot wrote:

So, the reason why I never followed the ucspi-tls thing is that I 
simply

don't like opportunistic TLS. A core tenet of my philosophy is that
dynamic stuff is inherently more dangerous than static stuff, and it 
is

harmful to have dynamic stuff just because we can; every addition of
dynamism should be properly justified.


I tend to agree. Every additional runtime state is an invitation to 
misunderstandings. It's also the case that when people and tools have 
come to expect those runtime states, _not_ providing them incurs its own 
costs and risks.



 starttls-wrapper would:


fixsmtpio(8) sits in this spot and does some of these things. One way to 
be sure the SMTP server has reset all internal state after STARTTLS is 
to kill and restart it, architecture permitting. fixsmtpio does exactly 
this. :-)



* on STARTTLS, drop the plaintext connection to qmail-smtpd, open a
secure connection to localhost:587 with the real tls-enabled server,
replay the start of the dialogue with it, then disappear into the
aether, leaving the client to speak with the real server.


Hmm. This doesn't match my initial expectations:

- Port 587 (1) either has always-on TLS or requires STARTTLS before 
proceeding, (2) requires AUTH, (3) allows relaying, (4) other 
submission-specific minutiae


- Port 25 (1) permits STARTTLS, (2) disallows AUTH, (3) disallows 
relaying, (4) other incoming-specific minutiae


- From qmail's point of view, submission and incoming have been distinct 
services, often provided by distinct programs


I get the idea, though, and could try to adjust the design to do more of 
what I'd want.


I wrote fixsmtpio under the assumption that qmail would never have an 
active upstream, so it'd be cheaper to maintain my own somewhat complex 
standalone program than to carry around yet another patch. That 
assumption has since turned false. I now suspect it'd be cheaper to 
merge UCSPI-TLS logic directly into qmail-smtpd and ofmipd and get rid 
of fixsmtpio. But that puts a lot of weight on ucspi-ssl.


I'd also like to break up qmail-remote to run its SMTP client under 
tcpclient, because then it could just as easily run under "sslclient -y" 
(if there were one, or the s6-networking equivalent), and then STARTTLS 
support would be simple to add to the client.


So that's why I'm wishing s6-networking would provide an independent 
(and almost certainly better factored) implementation of UCSPI-TLS: it 
would give us a boost in our efforts to modernize qmail. But I see why 
you're not wishing to, too.


- Amitai


Re: UCSPI-TLS for s6-networking?

2020-11-16 Thread Laurent Bercot



 Hi Amitai,

 Nice to have you here! What took you so long? ;)

 So, the reason why I never followed the ucspi-tls thing is that I 
simply

don't like opportunistic TLS. A core tenet of my philosophy is that
dynamic stuff is inherently more dangerous than static stuff, and it is
harmful to have dynamic stuff just because we can; every addition of
dynamism should be properly justified. And that includes STARTTLS: why
would you start a connection in plain text *then* upgrade to a secure
one, when you can start a TLS connection right away?

 That's why I designed s6-tlsclient and s6-tlsserver to always perform
immediate encryption, not delayed encryption. It works well with IMAP:
it's generally accepted that IMAP servers can (and should) require the
use of imaps as a policy, and that IMAP clients should just perform
imaps connections, starting right away with a TLS exchange, instead of
relying on STARTTLS. Why is it not the case with SMTP?

 (Actually, I have a pretty good idea why it's not the case with SMTP:
the SMTP landscape is a wasteland on top of a landfill, and I'm out of
'land' words but the landfill is contained in a giant garbage truck
that is on fire, and the firemen are all Pennywise the Clown, and the
fire hoses are actually spewing out gasoline.)

 So, the programs in s6-networking that perform the TLS layer, i.e.
s6-tlsc and s6-tlsd, are entirely self-contained, will perform the
TLS handshake, and will just provide a tunnel for the applications to
write into. Same idea as ucspi-ssl, except I wanted to be free of the
horrendous openssl interface, and I wanted to play with bearssl, which
is hands down the best thing that has happened to transport security
since the first release of SSLeay.

 If I needed to write an SMTP server that supports STARTTLS, the way
I would do it would be the following:
 - Use an unpatched SMTP server that does not support it, typically
qmail-smtpd
 - Have a s6-tlsserver process on port 587 (it is really a s6-tcpserver
process that performs the correct s6-tlsd invocation when it gets a
client connection) spawning a qmail-smtpd for native TLS connections
 - Have a regular s6-tcpserver process on port 25 listening to plain
text connections, but invoking something like:
 starttls-wrapper localhost 587 qmail-smtpd
 - Write the starttls-wrapper program.

 starttls-wrapper would:
 * spawn qmail-smtpd and forward the communication between the client
and the qmail-smtpd instance, but record it, and snoop on it looking
for a STARTTLS command
 * depending on a command-line option, if no STARTTLS has been
received by some point, either drop the connection (secure option) or
just stop the recording and continue the plaintext forwarding
(default option)
 * on STARTTLS, drop the plaintext connection to qmail-smtpd, open a
secure connection to localhost:587 with the real tls-enabled server,
replay the start of the dialogue with it, then disappear into the
aether, leaving the client to speak with the real server.

 The benefit of that approach is that it doesn't involve patching at
all, and can be made as secure as you want. The drawback is that it
may interpose yet another forwarding process in the data flow, causing
context switches and thus slowness, but I don't think it's critical.

 The starttls-wrapper program is protocol-specific, but STARTTLS is
already protocol-specific so there's no real difference in the amount
of effort needed to support it.

 I realize it's probably not the answer you were looking for, but it's
the best I can give you. :)

 What do you think?

--
 Laurent



UCSPI-TLS for s6-networking?

2020-11-16 Thread Amitai Schleier
Hi! I just met s6-networking and am wondering whether it might suit some 
of my needs. I see UCSPI tools analogous to those in ucspi-tcp and 
ucspi-ssl, except more composable. That's why I'm here :-)


Way back in the day, Scott Gifford and Charlie Brady designed an 
UCSPI-TLS interface for "delayed encryption" of services that start 
unencrypted -- for instance, an SMTP client and server negotiating 
STARTTLS. Here's an introduction (or maybe reminder): 
https://web.archive.org/web/20150311220932/http://www.suspectclass.com/sgifford/ucspi-tls/ucspi-tls-qmail-howto.html


I think it's a lovely interface. My own TLS and AUTH implementation for 
unpatched qmail-smtpd, qmail-pop3d, and ofmipd 
(https://schmonz.com/qmail/acceptutils) relies on it. At present, the 
only ready-to-run UCSPI-TLS implementation I'm aware of is Erwin 
Hoffmann's ucspi-ssl fork, which includes the "sslserver -n" portion of 
the original implementation but not yet the corresponding "sslclient 
-y".


If my code -- and perhaps notqmail's 
(https://github.com/notqmail/notqmail/wiki/Designs) -- could run equally 
well under s6-networking, that would be really cool. Could UCSPI-TLS be 
made to fit nicely into s6-networking's design? If so, would you be 
willing to consider implementing it?


For reference, here's what I believe is the latest version of Gifford 
and Brady's code, including both client and server implementations for 
William Baxter's ucspi-ssl: 
https://github.com/SuperScript/ucspi-ssl/compare/master...scottgifford:master


Thanks,

- Amitai