[
https://issues.apache.org/jira/browse/IGNITE-28867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-28867:
--------------------------------------
Description:
Currently, Ignite node TLS certificates are loaded only once during node
startup and then used to establish secure connections between `node to node`,
`node to thin client`. As a result, updating a certificate requires restarting
the Ignite node, causing unnecessary downtime and requiring additional
administrative effort.
We need to implement hot redeployment of TLS certificates, allowing the Ignite
node to reload the certificates at runtime without requiring a node restart.
Existing TLS sessions should remain unaffected, while all new TLS handshakes
should use the updated credentials.
Basic Steps
1. Introduce a mechanism to trigger certificate reloading (for example, through
`control.sh`).
2. Create a new `SslContext` instance by invoking `SslContextFactory`.
3. Replace the active `SslContext` used for TLS connection establishment. All
newly established connections must use the updated certificates, while existing
connections will continue to operate without interruption.
h2. Design
*Command.* {{control.sh --ssl reload}} is run against any node and reaches
every node of the cluster, servers and clients. {{--dry-run}} does the same
work but puts nothing in use, so the certificates on disk can be verified
before the cluster is touched.
The command carries no certificate and no path: each node re-opens the key and
trust store files at the paths its own {{SslContextFactory}} is configured
with, and takes whatever is in them at that moment.
*Registry instead of point-to-point calls.* A component registers itself in
{{GridInternalSubscriptionProcessor}} once it has set SSL up, so an empty
registry means the node does not use SSL. The command talks only to the
registry; a future SSL transport becomes reloadable by registering.
{code:java}
public interface SslContextReloadable {
boolean reloadSslContext() throws IgniteCheckedException; // rebuild and
apply
boolean checkSslContext() throws IgniteCheckedException; // rebuild only
default X509Certificate servedCertificate() { return null; }
}
{code}
Registered today: communication, discovery, client connector, binary (TCP) REST.
*Applying a new context.* {{AbstractSslContextFactory}} gains {{reload()}}
(rebuild and replace what {{create()}} hands out) and {{build()}} (rebuild
only, used by the dry run). Publishing into the factory is load-bearing:
outbound communication asks the factory per connection, and that is how it
picks the new certificates up. Inbound NIO transports get the context through
{{GridNioSslFilter}}, which creates an {{SSLEngine}} per session, so
established sessions are untouched.
A factory that hands back the context already in use cannot be reloaded. This
is detected by identity rather than by type, and reported as {{NOT reloaded}}
instead of a false success.
*Discovery.* The listening socket is now a plain {{ServerSocket}}, and every
accepted connection is wrapped with the context current at that moment. An
{{SSLServerSocket}} captures the context at bind time, which would have left
discovery reloadable for outbound connections only.
*Check before applying.* For communication and discovery, a rebuilt context
must complete a mutually authenticated TLS handshake in memory before it
replaces the one in use; otherwise the node keeps its current certificates and
the command fails. Client-facing transports are not checked, because a client
legitimately holds a different key and trust store. Only a refused handshake
counts as a failure, so the check cannot become a new way of blocking an
emergency rotation. That handshake also yields the certificate the node now
serves, which the report prints.
*Not covered.* The HTTP REST (Jetty) connector: Ignite does not configure its
TLS at all, it exists only when the user supplies a Jetty XML config, and
Jetty's own {{reload()}} applies unconditionally, so the dry run cannot be
expressed for it. Thin clients and the JDBC/ODBC drivers hold their own
configuration.
h2. Why the context is replaced rather than the key managers
The alternative is to keep one {{SSLContext}} and swap the delegates inside
{{X509ExtendedKeyManager}}/{{X509ExtendedTrustManager}}. Elasticsearch had
exactly that and removed it in elastic/elasticsearch#30509 for FIPS 140-2,
where only SunJSSE managers may be used; Jetty and Kafka also rebuild the
context. Swapping managers additionally keeps the session cache and ticket keys
of the old context, so a resumed session is still served the old certificate
and trust cannot be revoked by a reload.
was:
Currently, Ignite node TLS certificates are loaded only once during node
startup and then used to establish secure connections between `node to node`,
`node to thin client`. As a result, updating a certificate requires restarting
the Ignite node, causing unnecessary downtime and requiring additional
administrative effort.
We need to implement hot redeployment of TLS certificates, allowing the Ignite
node to reload the certificates at runtime without requiring a node restart.
Existing TLS sessions should remain unaffected, while all new TLS handshakes
should use the updated credentials.
Basic Steps
1. Introduce a mechanism to trigger certificate reloading (for example, through
`control.sh`).
2. Create a new `SslContext` instance by invoking `SslContextFactory`.
3. Replace the active `SslContext` used for TLS connection establishment. All
newly established connections must use the updated certificates, while existing
connections will continue to operate without interruption.
> Add control.sh --ssl commands for TLS certificate hot redeployment
> ------------------------------------------------------------------
>
> Key: IGNITE-28867
> URL: https://issues.apache.org/jira/browse/IGNITE-28867
> Project: Ignite
> Issue Type: Task
> Reporter: Mikhail Petrov
> Assignee: Anton Vinogradov
> Priority: Major
> Labels: ise
> Fix For: 2.19
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Currently, Ignite node TLS certificates are loaded only once during node
> startup and then used to establish secure connections between `node to node`,
> `node to thin client`. As a result, updating a certificate requires
> restarting the Ignite node, causing unnecessary downtime and requiring
> additional administrative effort.
> We need to implement hot redeployment of TLS certificates, allowing the
> Ignite node to reload the certificates at runtime without requiring a node
> restart. Existing TLS sessions should remain unaffected, while all new TLS
> handshakes should use the updated credentials.
> Basic Steps
> 1. Introduce a mechanism to trigger certificate reloading (for example,
> through `control.sh`).
> 2. Create a new `SslContext` instance by invoking `SslContextFactory`.
> 3. Replace the active `SslContext` used for TLS connection establishment. All
> newly established connections must use the updated certificates, while
> existing connections will continue to operate without interruption.
> h2. Design
> *Command.* {{control.sh --ssl reload}} is run against any node and reaches
> every node of the cluster, servers and clients. {{--dry-run}} does the same
> work but puts nothing in use, so the certificates on disk can be verified
> before the cluster is touched.
> The command carries no certificate and no path: each node re-opens the key
> and trust store files at the paths its own {{SslContextFactory}} is
> configured with, and takes whatever is in them at that moment.
> *Registry instead of point-to-point calls.* A component registers itself in
> {{GridInternalSubscriptionProcessor}} once it has set SSL up, so an empty
> registry means the node does not use SSL. The command talks only to the
> registry; a future SSL transport becomes reloadable by registering.
> {code:java}
> public interface SslContextReloadable {
> boolean reloadSslContext() throws IgniteCheckedException; // rebuild and
> apply
> boolean checkSslContext() throws IgniteCheckedException; // rebuild only
> default X509Certificate servedCertificate() { return null; }
> }
> {code}
> Registered today: communication, discovery, client connector, binary (TCP)
> REST.
> *Applying a new context.* {{AbstractSslContextFactory}} gains {{reload()}}
> (rebuild and replace what {{create()}} hands out) and {{build()}} (rebuild
> only, used by the dry run). Publishing into the factory is load-bearing:
> outbound communication asks the factory per connection, and that is how it
> picks the new certificates up. Inbound NIO transports get the context through
> {{GridNioSslFilter}}, which creates an {{SSLEngine}} per session, so
> established sessions are untouched.
> A factory that hands back the context already in use cannot be reloaded. This
> is detected by identity rather than by type, and reported as {{NOT reloaded}}
> instead of a false success.
> *Discovery.* The listening socket is now a plain {{ServerSocket}}, and every
> accepted connection is wrapped with the context current at that moment. An
> {{SSLServerSocket}} captures the context at bind time, which would have left
> discovery reloadable for outbound connections only.
> *Check before applying.* For communication and discovery, a rebuilt context
> must complete a mutually authenticated TLS handshake in memory before it
> replaces the one in use; otherwise the node keeps its current certificates
> and the command fails. Client-facing transports are not checked, because a
> client legitimately holds a different key and trust store. Only a refused
> handshake counts as a failure, so the check cannot become a new way of
> blocking an emergency rotation. That handshake also yields the certificate
> the node now serves, which the report prints.
> *Not covered.* The HTTP REST (Jetty) connector: Ignite does not configure its
> TLS at all, it exists only when the user supplies a Jetty XML config, and
> Jetty's own {{reload()}} applies unconditionally, so the dry run cannot be
> expressed for it. Thin clients and the JDBC/ODBC drivers hold their own
> configuration.
> h2. Why the context is replaced rather than the key managers
> The alternative is to keep one {{SSLContext}} and swap the delegates inside
> {{X509ExtendedKeyManager}}/{{X509ExtendedTrustManager}}. Elasticsearch had
> exactly that and removed it in elastic/elasticsearch#30509 for FIPS 140-2,
> where only SunJSSE managers may be used; Jetty and Kafka also rebuild the
> context. Swapping managers additionally keeps the session cache and ticket
> keys of the old context, so a resumed session is still served the old
> certificate and trust cannot be revoked by a reload.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
