Re: httpd/libtls: TLS client certificate revocation checking

[Jack Burton]

On Sun, 2 Apr 2017 06:27:45 +0930
Jack Burton  wrote:
> On Sat, 01 Apr 2017 18:22:17 +
> Bob Beck  wrote:
> > There will be some libtls api additions post 6.1 to get the peer
> > cert in PEM format  
> 
> Thanks Bob. That sounds like exactly what's needed. Happy to wait.

...and your tls_peer_cert_chain_pem() solves things nicely.

Many thanks for that.

Here's a little diff to document it.

Index: lib/libtls/man/tls_conn_version.3
===
RCS file: /cvs/src/lib/libtls/man/tls_conn_version.3,v
retrieving revision 1.4
diff -u -p -r1.4 tls_conn_version.3
--- lib/libtls/man/tls_conn_version.3   28 Jan 2017 00:59:36 -  1.4
+++ lib/libtls/man/tls_conn_version.3   27 May 2017 06:55:41 -
@@ -24,6 +24,7 @@
 .Nm tls_conn_alpn_selected ,
 .Nm tls_conn_servername ,
 .Nm tls_peer_cert_provided ,
+.Nm tls_peer_cert_chain_pem ,
 .Nm tls_peer_cert_contains_name ,
 .Nm tls_peer_cert_issuer ,
 .Nm tls_peer_cert_subject ,
@@ -43,6 +44,11 @@
 .Fn tls_conn_servername "struct tls *ctx"
 .Ft int
 .Fn tls_peer_cert_provided "struct tls *ctx"
+.Ft const uint8_t *
+.Fo tls_peer_cert_chain_pem
+.Fa "struct tls *ctx"
+.Fa "size_t *size"
+.Fc
 .Ft int
 .Fo tls_peer_cert_contains_name
 .Fa "struct tls *ctx"
@@ -89,6 +95,14 @@ checks if the peer of
 .Ar ctx
 has provided a certificate.
 .Pp
+.Fn tls_peer_cert_chain_pem
+returns a string consisting of the PEM encoded certificate chain of the peer
+from
+.Ar ctx .
+The value of
+.Ar size
+is set to the length of the string.
+.Pp
 .Fn tls_peer_cert_contains_name
 checks if the peer of a TLS
 .Ar ctx
@@ -172,6 +186,10 @@ and
 .Fn tls_conn_alpn_selected
 appeared in
 .Ox 6.1 .
+.Pp
+.Fn tls_peer_cert_chain_pem
+appeared in
+.Ox 6.2 .
 .Sh AUTHORS
 .An Bob Beck Aq Mt b...@openbsd.org
 .An Joel Sing Aq Mt js...@openbsd.org

Re: httpd/libtls: TLS client certificate revocation checking

[William Ahern]

On Sat, Apr 01, 2017 at 07:10:35PM +1030, Jack Burton wrote:
> On Fri, 31 Mar 2017 13:03:44 -0700
> William Ahern  wrote:

> > Basically, anything short of passing through the entire certificate
> > is going to be severely limiting and frustrating, to the point of
> > uselessness. And as a practical matter, parsing out and passing
> > through a subset of certificate data is likely going to require more
> > complex code than just passing the entire certificate to the
> > application.
> 
> Thanks William. That's an interesting idea.
> 
> Yes, I can see how having the raw client certificate available to
> fastcgi responders would be useful. That would solve a few more
> problems for my use case too and would provide the ultimate in
> flexibility. And it's not difficult to implement.
> 
> My initial gut feeling was that adding a tls_peer_serial() function to
> libtls would be less intrusive (and more in keeping with the rest of
> the tls_peer_* functions) than adding a tls_peer_cert() along the lines
> you're suggesting.

Converting a cert to a PEM string should be as simple as:

  BIO *bio = BIO_new(BIO_s_mem()); /* See NOTE 1 */
  if (!bio)
goto sslerror;
  if (!PEM_write_bio_X509(bio, crt))
goto sslerror;
  char *pem;
  long pemlen = BIO_get_mem_data(bio, );
  assert(pemlen >= 0); /* < 0 shouldn't be possible */
  ... /* copy pem string */
  BIO_free(bio); /* See NOTE 1 */

compare that to getting the serial:
  
  BIGNUM *bn = BN_new();
  if (!bn)
goto sslerror;
  ASN1_INTEGER *i;
  if (!(i = X509_get_serialNumber(crt))) /* See NOTE 2 */
goto noserial;
  if (!ASN1_INTEGER_to_BN(i, bn))
goto sslerror;
  char *serial = BN_bn2dec(serial);
  if (!serial)
goto sslerror;
  ... /* copy serial string */
  BN_free(bn);


NOTE 1: I usually keep a BIO buffer around as a singleton, using BIO_reset
instead of BIO_free.

NOTE 2: I'd try X509_get0_serialNumber unless the const-ness is troublesome.
I'm cribbing from my own code, originally written for OpenSSL 0.9.8, which
lacks X509_get0_serialNumber.

Re: httpd/libtls: TLS client certificate revocation checking

[Jack Burton]

On Sat, 01 Apr 2017 18:22:17 +
Bob Beck  wrote:
> There will be some libtls api additions post 6.1 to get the peer cert
> in PEM format

Thanks Bob. That sounds like exactly what's needed. Happy to wait.

> In the meantime, testing snaps prior to 6.1 should be the priority.
> not a talkathon.

Agreed. Sorry for the noise.

Re: httpd/libtls: TLS client certificate revocation checking

[Bob Beck]

There will be some libtls api additions post 6.1 to get the peer cert in
PEM format

In the meantime, testing snaps prior to 6.1 should be the priority. not a
talkathon.

On Sat, Apr 1, 2017 at 10:49 Joerg Sonnenberger  wrote:

> On Sat, Apr 01, 2017 at 07:53:05PM +1030, Jack Burton wrote:
> > One common example of that happening is when a cert gets revoked because
> > its private key has been lost/stolen and the user needs a new cert
> > associated with the same identity. An even more common example is when
> > a cert expires & gets renewed.
>
> If you are using certificate revocation, I think you should do the check
> as early as possible. That means in httpd in this case. Nothing later in
> the stack should have to care about expired or revoked certificates --
> it just adds complexity and the danger of someone forgetting about it.
>
> Which mechanisms to support (i.e. CRLs or OCSP) is a completely
> different topic.
>
> Joerg
>
>

Re: httpd/libtls: TLS client certificate revocation checking

[Joerg Sonnenberger]

On Sat, Apr 01, 2017 at 07:53:05PM +1030, Jack Burton wrote:
> One common example of that happening is when a cert gets revoked because
> its private key has been lost/stolen and the user needs a new cert
> associated with the same identity. An even more common example is when
> a cert expires & gets renewed.

If you are using certificate revocation, I think you should do the check
as early as possible. That means in httpd in this case. Nothing later in
the stack should have to care about expired or revoked certificates --
it just adds complexity and the danger of someone forgetting about it.

Which mechanisms to support (i.e. CRLs or OCSP) is a completely
different topic.

Joerg

Re: httpd/libtls: TLS client certificate revocation checking

[Jack Burton]

On Fri, 31 Mar 2017 22:21:38 +0200
Joerg Sonnenberger  wrote:
> On Fri, Mar 31, 2017 at 01:03:44PM -0700, William Ahern wrote:
> > Basically, anything short of passing through the entire certificate
> > is going to be severely limiting and frustrating, to the point of
> > uselessness.  
> 
> Passing down the common name is normally enough, but not doing that
> makes it nearly useless. Speaking from the perspective of using them
> in the wild.

My initial diff passed the DN (from which you can get the CN with a
trivial regex) in TLS_PEER_SUBJECT.

That might be enough in some circumstances, but remember that RFC5280
tells us "A CA MAY issue more than one certificate with the same DN to
the same subject entity"...

One common example of that happening is when a cert gets revoked because
its private key has been lost/stolen and the user needs a new cert
associated with the same identity. An even more common example is when
a cert expires & gets renewed.

The only thing that's always required to be unique is {issuer,
serialNumber}.

So at a minimum the serialNumber & issuer are both still necessary for
unique identification (in addition to the DN plus whatever else for
authorisation)...

...unless you've *already* validated that the cert has not been revoked
(but either checking CRLs or doing non-stapled OCSP also requires
knowing the serialNumber).

In fact, your application's authorisation routines can be quite a bit
simpler if your CA adopts a policy of having a 1:1 mapping of entities
to DNs, varying only the serial number & valid date range when renewing
or revoking/reissuing certs -- but that's a matter of personal choice:
nothing wrong with a CA having a "DNs never get reused" rule, but I
don't think it's reasonable to expect *every* CA to do that, given that
RFC5280 explicitly allows DN reuse for the same entity.

I agree with you that it probably makes sense to continue passing the
DN, even if we pass the whole cert as well, so that fastcgi responders
with simple auth schemes using CAs that do implement a "DNs never get
reused" rule don't need to link against libcrypto just to get the DN,
but I don't think that's a reason to shy away from William's good
suggestion of passing the whole cert.

On the other hand the rest of the TLS_PEER_* fastcgi variables from my
initial diff can probably be left out if we're passing the whole cert
(e.g. if you have the cert it's easy enough to generate the hash
yourself).

Re: httpd/libtls: TLS client certificate revocation checking

[Jack Burton]

On Fri, 31 Mar 2017 13:03:44 -0700
William Ahern  wrote:
> On Thu, Mar 30, 2017 at 10:31:06PM +1030, Jack Burton wrote:
> 
> > Personally, I'm leaning towards either local CRL file checking in
> > httpd (with minimal changes to libtls), or passing through enough
> > data to the let the fastcgi responders take whichever approach they
> > want.
<...>
> My $0.02: client certificates aren't particularly useful unless the
> certificate itself is made available to the application. Knowing that
> a certificate has a valid signature is only part of the equation, and
> not even the most useful part. Unlike server certificates, however,
> there's no single authorization check (i.e. matching the embedded
> domain name to the queried URL) that can be implemented by
> middle-ware. The semantics of embedded data are specific to a service
> and often ad hoc.
> 
> Basically, anything short of passing through the entire certificate
> is going to be severely limiting and frustrating, to the point of
> uselessness. And as a practical matter, parsing out and passing
> through a subset of certificate data is likely going to require more
> complex code than just passing the entire certificate to the
> application.

Thanks William. That's an interesting idea.

Yes, I can see how having the raw client certificate available to
fastcgi responders would be useful. That would solve a few more
problems for my use case too and would provide the ultimate in
flexibility. And it's not difficult to implement.

My initial gut feeling was that adding a tls_peer_serial() function to
libtls would be less intrusive (and more in keeping with the rest of
the tls_peer_* functions) than adding a tls_peer_cert() along the lines
you're suggesting.

But on reflection, adding a tls_peer_cert() would drastically reduce
the likelihood of needing to add anything else to the tls_peer_* family
again in future -- which I'd think would be a big advantage.

Given your point about needing to access the whole cert (I assume by
that you mean you're using some of the less common x509 extensions) for
authorisation (cf. just authentication), it may well be worth doing
that *regardless* of whether or not any common approach to revocation
status checking gets added that far down the stack.

Unless there are any better ideas in the meantime (or someone else does
it first), I'll propose a diff that does just that towards the end of
next week (sorry, will travelling for the next few days & probably
won't have time to site down & write it until then).

Re: httpd/libtls: TLS client certificate revocation checking

[Joerg Sonnenberger]

On Fri, Mar 31, 2017 at 01:03:44PM -0700, William Ahern wrote:
> Basically, anything short of passing through the entire certificate is going
> to be severely limiting and frustrating, to the point of uselessness.

Passing down the common name is normally enough, but not doing that
makes it nearly useless. Speaking from the perspective of using them in
the wild.

Joerg

Re: httpd/libtls: TLS client certificate revocation checking

[William Ahern]

On Thu, Mar 30, 2017 at 10:31:06PM +1030, Jack Burton wrote:

> Personally, I'm leaning towards either local CRL file checking in
> httpd (with minimal changes to libtls), or passing through enough data
> to the let the fastcgi responders take whichever approach they want.

In all my experience with client certificates (whether with browsers, IPSec,
or custom applications), I've usually relied on the ability to embed data in
the client certificate, such as domains, e-mail addresses, UUIDs, etc. That
often entirely removed the need to do database queries from application code
or to maintain and expose any centralized data store (except the CRL, of
course) to network-facing nodes.

In other words, client certificates not only solve the authentication
problem, but can also help solve the authorization problem, and in many
cases solve it completely.

My $0.02: client certificates aren't particularly useful unless the
certificate itself is made available to the application. Knowing that a
certificate has a valid signature is only part of the equation, and not even
the most useful part. Unlike server certificates, however, there's no single
authorization check (i.e. matching the embedded domain name to the queried
URL) that can be implemented by middle-ware. The semantics of embedded data
are specific to a service and often ad hoc.

Basically, anything short of passing through the entire certificate is going
to be severely limiting and frustrating, to the point of uselessness. And as
a practical matter, parsing out and passing through a subset of certificate
data is likely going to require more complex code than just passing the
entire certificate to the application.

httpd/libtls: TLS client certificate revocation checking

[Jack Burton]

Earlier this evening I proposed a patch that adds support for
requiring/verifying TLS client certificates to httpd. See my last post.

But that only solves half the problem (checking that the client cert
was issued by a locally-trusted CA and has not yet expired).

The other (and in my opinion equally important) half is checking that
the client cert has not yet been revoked (without that check, there
seems little point verifying certificate authenticity). There are a
bunch of different ways of going about that.

Assuming that the devs are happy to go down the path of supporting TLS
client certs in httpd (if not, the rest of this post will be pointless,
so just ignore it), I'm looking for some feedback on the most
*desirable* approach to client cert revocation status checking.

My own thoughts (for what they're worth) are:

OCSP stapling, whilst ideal (and already supported) for server certs,
is unlikely to be of much practical use with client certs (since, at
least from my experience, users -- especially non-technical users --
are likely to see the burden on them as too great to justify).

Online OSCP (or worse, online CRL checking) tends to be discouraged,
due to the amount of extra per-request traffic generated. In both cases,
that would involve adding an http client to httpd, which really doesn't
seem like a sensible place to implement such a thing... (complexity,
monoliths, etc. etc.)

But with client certs (as opposed to server certs), it's much more
common for the issuing entity & the verifying entity to be one & the
same. So local methods are probably worth exploring. As far as I can
tell, there are three potentially viable alternatives:

Noting that RFC6960 does not require using HTTP as the transport for
OCSP, talking to a local OSCP responder over a Unix socket (much as
httpd already does with fastcgi responders) might be a possibility.
libtls already provides tls_ocsp_process_response(), so all httpd would
need to do is craft the query, send it to the socket & pass back the
response. That seems to limit complexity in httpd nicely, but creates a
need to write a new OCSP repsonder...

Checking a local CRL file (the approach taken by apache & nginx) would
also be relatively easy to implement, although it would require at least
a minor patch to libtls first, to expose the cert serial number & CRL
URL (or alternatively, just have libtls wrap libssl's local CRL checking
routines?). Even if the issuing & verifying parties are not the same,
this could still work reasonably well -- e.g. with a simple cron job
refreshing a local cached copy of the CRL (still need to fetch the
whole CRL whenever it changes, but that's a lot better than fetching
anything on a per-request basis). Only drawback is that there's a lot
of talk about CRLs being deprecated...

Alternatively, noting that client certs are most often used with
dynamic sites, client cert revocation status checking could perhaps be
left to the fastcgi responder. We're already (if the diff proposed in my
other post gets accepted) passing through TLS_PEER_OCSP_URL &
TLS_PEER_HASH. With a small patch to libtls it would trivial to have
httpd pass through the client cert's CRL URL & serial number too (both
methods require the cert serial number). That would provide the most
flexibility (fastcgi responders could choose to use OSCP, CRLs or
something else; online or against local files) and keep both httpd &
libtls as simple as possible, but does add a bit of work for those
implementing fastcgi responders (although that could be done once as a
separate library, perhaps in ports).

Without modifying libtls, the only way I can see to have a fastcgi
responder do client cert revocation status checking would be to
implement a bespoke protocol for checking a non-standard CRL substitute
file (presumably keyed by TLS_PEER_HASH), which is something I'd really
prefer to avoid given that the problem is obviously of such generic
applicability.

Personally, I'm leaning towards either local CRL file checking in
httpd (with minimal changes to libtls), or passing through enough data
to the let the fastcgi responders take whichever approach they want.

I'm interested in thoughts from the devs -- am I on the right track or
have I missed something obvious? If httpd is to gain support for TLS
client certificate verification (including revocation status checking),
which approach would fit in best with the OpenBSD design philosophy?
Does the bulk of the code "belong" in libtls or in httpd or should as
much as possible be left to the fastcgi responders?

Thanks.