On Sun, Aug 30, 2020 at 05:45:41PM -0500, David Arnold wrote:
> If you prefer this mailing list over github issues, I still want
to ask
> for comments on:
>
> Certificate hot-reloading #12753
> <<https://github.com/openssl/openssl/issues/12753>>
>
> Specifically, my impression is that this topic has died down a
bit and
> from the linked mailing list threads, in my eye, no concrete
conclusion
> was drawn.
>
> I'm not sure how to rank this motion in the context of OpenSSL
> development, but I guess OpenSSL is used to producing ripple
effects,
> so the man-hour argument might be a genuinely valid one.
>
> Please inform my research about this issue with your comments!
This is a worthwhile topic. It has a few interesting aspects:
1. Automatic key+cert reloads upon updates of key+cert chain
PEM
files. This can be tricky when processes start privileged,
load the certs and then drop privs, and are no longer able
to reopen the key + cert chain file.
- Here, for POSIX systems I'd go with an approach where
it is the containing directory that is restricted to
root or similar, and the actual cert files are group
and or world readable. The process can then keep
the directory file descriptor open, and then openat(2)
to periodically check the cert file, reloading when
the metadata changes.
- With non-POSIX systems, or applications that don't
drop privs, the openat(2) is not needed, and one
just checks the cert chain periodically.
- Another option is to use passphrase-protected keys,
and load the secret passphrase at process start from
a separate read-protected file, while the actual
private key + cert chain file is world readable,
with the access control via protecting the passphrase
file.
- In all cases, it is important to keep both the private
key and the cert in the same file, and open it just
once to read both, avoiding races in which the key
and cert are read in a way that results in one or
the other being stale.
2. Having somehow obtained a new key + cert chain, one
now wants to non-disruptively apply them to running
servers. Here there are two potential approaches:
- Hot plug a new pointer into an existing SSL_CTX structure.
While the update itself could be made atomic, the readers
of such pointers might read them more than once to
separately
extract the key and the cert chain, without checking that
they're using the same pointer for both operations.
This is bound to be fragile, though not necessarily
impossible.
- Build a new SSL_CTX, and use it to accept *new*
connections,
while existing connections use whatever SSL_CTX they
started
with. I believe this can work well, because "SSL" handles
increment the reference count of the associated SSL_CTX
when they're created, and decrement it when destroyed.
So when you create a replacement SSL_CTX, you can just
SSL_CTX_free() the old, and it will only actually
be deleted when the last SSL connection tied to that
SSL_CTX is destroyed.
It is true that typical SSL_CTX construction is modestly
expensive (loading CA stores and the like) but some of
that could be handled by sharing and reference-counting
the stores.
So my preferred approach would be to create a new SSL_CTX, and get
new
connections using that. Now in a multi-threaded server, it could
be a
bit tricky to ensure that the SSL_CTX_free() does not happen before
all
threads reading the pointer to the latest SSL_CTX see the new
pointer
installed. Something equivalent to RCU may be needed to ensure
that the
free only happens after the new pointer is visible in all threads.
Designs addressing various parts of this would be cool, provided
they're
well thought out, and not just single-use-case quick hacks.
--
Viktor.
