[
https://issues.apache.org/jira/browse/IGNITE-29211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-29211:
--------------------------------------
Description:
Administrators can reload TLS certificates with {{control.sh --ssl reload}}.
Add optional automatic hot reload so nodes can read replacements before expiry
without that command. Ignite reads the configured source; it does not issue
certificates.
h3. Schedule
* Hot reload is off by default. Add
{{SslContextFactory.setHotReloadEnabled(boolean)}} and
{{isHotReloadEnabled()}}; changing the setting requires a restart. Cover
enabled factories used by the node's started TCP transports.
* Read certificate chains from the key store actually loaded, including custom
sources. With hot reload enabled, require at least one private key and an X.509
chain for every private key.
* A chain expires when any certificate in it expires. Select the chain with the
earliest expiry. Start attempts with 15% of its lifetime remaining, measured
from the key certificate's validity start. For equal expiries, use the longest
lifetime. A date already due at startup triggers an immediate attempt.
* After an automatic load, read the resulting hot reload date. If it is still
due, record a failed hot reload attempt even though the valid context was
installed. Keep that context and retry. This includes identical certificates,
trust-store-only changes and replacements already within their hot reload
window.
* After a failed automatic attempt, retry with a one-minute base delay,
doubling up to one hour. Add a random delay of up to half the base delay.
Continue retries after expiry.
* After a successful manual reload, reset that factory's failure count and
schedule the next automatic attempt from its loaded certificates. Run
immediately if hot reload is still due. Otherwise, retain the future hot reload
date.
h3. Replacement rules
* Manual and automatic replacements in {{SslContextFactory}} use the same
loading path. Check every certificate in the private-key chains against the
current date. Accept valid certificates with equal, earlier or later expiry,
including identical certificates and trust-store-only changes. The same
validity check applies at startup, including in client applications and the
control script. Expired or not yet valid certificates prevent startup.
* Publish a replacement only after loading and validation succeed. A loading or
validation failure keeps the current context and hot reload date. Established
connections remain open; new connections use the accepted context.
* The manual command reports {{reloaded}} after a successful load, including
when the material has not changed. Verify new TLS connections to confirm the
expected certificates and peer trust.
* Factories extending only {{AbstractSslContextFactory}} keep their own manual
loading behavior.
h3. API and execution
Expose {{getHotReloadTime()}} and {{void hotReload()}} on
{{SslContextFactory}}. Preserve {{final void init()}}. The getter returns an
epoch-millisecond date, or {{0}} if the current certificates were loaded with
hot reload disabled or have not been loaded yet.
{{hotReload()}} checks whether loading is due before calling {{init()}}. It
skips loading when disabled, not initialized or not due. A due call waits for
any active factory load; it does not recheck the date after that wait. Loading
and validation failures throw. After a normal return, the node rereads the date
to decide whether hot reload succeeded.
Run automatic attempts and manual reloads of enabled factories on one executor
per node. Before the hot reload registry starts, reject manual reloads of
enabled factories without loading material. Each factory serializes its loads.
Each node keeps its own schedule; nodes sharing a factory may query the source
in the same window. Startup may load the same factory twice, sequentially.
On shutdown, cancel queued work, interrupt the running executor task and wait
for it to finish. Do not count failures after the executor shuts down. A task
already waiting for a shared factory may still load and install a valid
replacement after shutdown starts. Startup initialization already waiting for
the factory may also proceed. Custom sources must bound load time so shutdown
can finish.
h3. Feedback and checks
* Log failures and the next attempt with the affected transports.
* {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on
the node. Reset a factory's count after a successful manual reload or an
automatic attempt after which hot reload is no longer due. Manual failures and
shutdown cancellation do not increment it.
* Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so
operators can track attempt frequency. Count calls per factory, including
errors and automatic skips. A factory shared by several transports counts once
per attempt. Exclude startup, direct application calls and queued calls
cancelled before execution.
* Test dates, custom sources, identical and trust-only loads, error recovery,
scheduling, shared factories, shutdown and metrics. Verify hot reload through a
new TLS connection and reload through the manual command.
* Document preparation, trust rollout, command results and recovery. Explain
that a failed hot reload can leave newly loaded material in use when hot reload
remains due.
Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command and
certificate inventory are outside this ticket.
[Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]
was:
Administrators can reload TLS certificates with {{control.sh --ssl reload}}.
Add optional automatic renewal so nodes can read replacements before expiry
without that command. Ignite reads the configured source; it does not issue
certificates.
h3. Schedule
* Renewal is off by default. Add
{{SslContextFactory.setRenewalEnabled(boolean)}} and {{isRenewalEnabled()}};
changing the setting requires a restart. Cover enabled factories used by the
node's started TCP transports.
* Read certificate chains from the key store actually loaded, including custom
sources. With renewal enabled, require at least one private key and an X.509
chain for every private key.
* A chain expires when any certificate in it expires. Select the chain with the
earliest expiry. Start attempts with 15% of its lifetime remaining, measured
from the key certificate's validity start. For equal expiries, use the longest
lifetime. A date already due at startup triggers an immediate attempt.
* After an automatic load, read the resulting renewal date. If it is still due,
record a failed renewal attempt even though the valid context was installed.
Keep that context and retry. This includes identical certificates,
trust-store-only changes and replacements already within their renewal window.
* After a failed automatic attempt, retry with a one-minute base delay,
doubling up to one hour. Add a random delay of up to half the base delay.
Continue retries after expiry.
* After a successful manual reload, reset that factory's failure count and
schedule the next automatic attempt from its loaded certificates. Run
immediately if renewal is still due. Otherwise, retain the future renewal date.
h3. Replacement rules
* Manual and automatic replacements in {{SslContextFactory}} use the same
loading path. Check every certificate in the private-key chains against the
current date. Accept valid certificates with equal, earlier or later expiry,
including identical certificates and trust-store-only changes. The same
validity check applies at startup, including in client applications and the
control script. Expired or not yet valid certificates prevent startup.
* Publish a replacement only after loading and validation succeed. A loading or
validation failure keeps the current context and renewal date. Established
connections remain open; new connections use the accepted context.
* The manual command reports {{reloaded}} after a successful load, including
when the material has not changed. Verify new TLS connections to confirm the
expected certificates and peer trust.
* Factories extending only {{AbstractSslContextFactory}} keep their own manual
loading behavior.
h3. API and execution
Expose {{getRenewalTime()}} and {{void renew()}} on {{SslContextFactory}}.
Preserve {{final void init()}}. The getter returns an epoch-millisecond date,
or {{0}} if the current certificates were loaded with renewal disabled or have
not been loaded yet.
{{renew()}} checks whether loading is due before calling {{init()}}. It skips
loading when disabled, not initialized or not due. A due call waits for any
active factory load; it does not recheck the date after that wait. Loading and
validation failures throw. After a normal return, the node rereads the date to
decide whether renewal succeeded.
Run automatic attempts and manual reloads of enabled factories on one executor
per node. Before the renewal registry starts, reject manual reloads of enabled
factories without loading material. Each factory serializes its loads. Each
node keeps its own schedule; nodes sharing a factory may query the source in
the same window. Startup may load the same factory twice, sequentially.
On shutdown, cancel queued work, interrupt the running executor task and wait
for it to finish. Do not count failures after the executor shuts down. A task
already waiting for a shared factory may still load and install a valid
replacement after shutdown starts. Startup initialization already waiting for
the factory may also proceed. Custom sources must bound load time so shutdown
can finish.
h3. Feedback and checks
* Log failures and the next attempt with the affected transports.
* {{ssl.ConsecutiveRenewalFailures}} sums consecutive automatic failures on the
node. Reset a factory's count after a successful manual reload or an automatic
attempt after which renewal is no longer due. Manual failures and shutdown
cancellation do not increment it.
* Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticRenewalAttempts}} so
operators can track attempt frequency. Count calls per factory, including
errors and automatic skips. A factory shared by several transports counts once
per attempt. Exclude startup, direct application calls and queued calls
cancelled before execution.
* Test dates, custom sources, identical and trust-only loads, error recovery,
scheduling, shared factories, shutdown and metrics. Verify renewal through a
new TLS connection and reload through the manual command.
* Document preparation, trust rollout, command results and recovery. Explain
that a failed renewal can leave newly loaded material in use when renewal
remains due.
Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command and
certificate inventory are outside this ticket.
[Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]
Summary: Hot-reload TLS certificates automatically before expiry (was:
Renew TLS certificates automatically before expiry)
> Hot-reload TLS certificates automatically before expiry
> -------------------------------------------------------
>
> Key: IGNITE-29211
> URL: https://issues.apache.org/jira/browse/IGNITE-29211
> Project: Ignite
> Issue Type: Sub-task
> Reporter: Anton Vinogradov
> Assignee: Anton Vinogradov
> Priority: Major
> Labels: ise
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Administrators can reload TLS certificates with {{control.sh --ssl reload}}.
> Add optional automatic hot reload so nodes can read replacements before
> expiry without that command. Ignite reads the configured source; it does not
> issue certificates.
> h3. Schedule
> * Hot reload is off by default. Add
> {{SslContextFactory.setHotReloadEnabled(boolean)}} and
> {{isHotReloadEnabled()}}; changing the setting requires a restart. Cover
> enabled factories used by the node's started TCP transports.
> * Read certificate chains from the key store actually loaded, including
> custom sources. With hot reload enabled, require at least one private key and
> an X.509 chain for every private key.
> * A chain expires when any certificate in it expires. Select the chain with
> the earliest expiry. Start attempts with 15% of its lifetime remaining,
> measured from the key certificate's validity start. For equal expiries, use
> the longest lifetime. A date already due at startup triggers an immediate
> attempt.
> * After an automatic load, read the resulting hot reload date. If it is still
> due, record a failed hot reload attempt even though the valid context was
> installed. Keep that context and retry. This includes identical certificates,
> trust-store-only changes and replacements already within their hot reload
> window.
> * After a failed automatic attempt, retry with a one-minute base delay,
> doubling up to one hour. Add a random delay of up to half the base delay.
> Continue retries after expiry.
> * After a successful manual reload, reset that factory's failure count and
> schedule the next automatic attempt from its loaded certificates. Run
> immediately if hot reload is still due. Otherwise, retain the future hot
> reload date.
> h3. Replacement rules
> * Manual and automatic replacements in {{SslContextFactory}} use the same
> loading path. Check every certificate in the private-key chains against the
> current date. Accept valid certificates with equal, earlier or later expiry,
> including identical certificates and trust-store-only changes. The same
> validity check applies at startup, including in client applications and the
> control script. Expired or not yet valid certificates prevent startup.
> * Publish a replacement only after loading and validation succeed. A loading
> or validation failure keeps the current context and hot reload date.
> Established connections remain open; new connections use the accepted context.
> * The manual command reports {{reloaded}} after a successful load, including
> when the material has not changed. Verify new TLS connections to confirm the
> expected certificates and peer trust.
> * Factories extending only {{AbstractSslContextFactory}} keep their own
> manual loading behavior.
> h3. API and execution
> Expose {{getHotReloadTime()}} and {{void hotReload()}} on
> {{SslContextFactory}}. Preserve {{final void init()}}. The getter returns an
> epoch-millisecond date, or {{0}} if the current certificates were loaded with
> hot reload disabled or have not been loaded yet.
> {{hotReload()}} checks whether loading is due before calling {{init()}}. It
> skips loading when disabled, not initialized or not due. A due call waits for
> any active factory load; it does not recheck the date after that wait.
> Loading and validation failures throw. After a normal return, the node
> rereads the date to decide whether hot reload succeeded.
> Run automatic attempts and manual reloads of enabled factories on one
> executor per node. Before the hot reload registry starts, reject manual
> reloads of enabled factories without loading material. Each factory
> serializes its loads. Each node keeps its own schedule; nodes sharing a
> factory may query the source in the same window. Startup may load the same
> factory twice, sequentially.
> On shutdown, cancel queued work, interrupt the running executor task and wait
> for it to finish. Do not count failures after the executor shuts down. A task
> already waiting for a shared factory may still load and install a valid
> replacement after shutdown starts. Startup initialization already waiting for
> the factory may also proceed. Custom sources must bound load time so shutdown
> can finish.
> h3. Feedback and checks
> * Log failures and the next attempt with the affected transports.
> * {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on
> the node. Reset a factory's count after a successful manual reload or an
> automatic attempt after which hot reload is no longer due. Manual failures
> and shutdown cancellation do not increment it.
> * Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so
> operators can track attempt frequency. Count calls per factory, including
> errors and automatic skips. A factory shared by several transports counts
> once per attempt. Exclude startup, direct application calls and queued calls
> cancelled before execution.
> * Test dates, custom sources, identical and trust-only loads, error recovery,
> scheduling, shared factories, shutdown and metrics. Verify hot reload through
> a new TLS connection and reload through the manual command.
> * Document preparation, trust rollout, command results and recovery. Explain
> that a failed hot reload can leave newly loaded material in use when hot
> reload remains due.
> Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command
> and certificate inventory are outside this ticket.
> [Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)