[
https://issues.apache.org/jira/browse/IGNITE-29211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-29211:
--------------------------------------
Description:
Administrators can reload TLS certificates with {{control.sh --ssl reload}}.
Add optional automatic hot reload so nodes can read replacements before expiry
without that command. Ignite reads the configured source; it does not issue
certificates.
h3. Schedule
* Hot reload is off by default. Add
{{SslContextFactory.setHotReloadEnabled(boolean)}} and
{{isHotReloadEnabled()}}; changing the setting requires a restart. Cover
enabled factories used by the node's started TCP transports.
* Read certificate chains from the key store actually loaded, including custom
sources. With hot reload enabled, require at least one private key with an
X.509 chain. Ignore keys without such a chain, as TLS does.
* A chain expires when any certificate in it expires. Start attempts
{{hotReloadBeforeExpiry}} before the earliest chain expiry, 7 days by default.
A date already due at startup triggers an immediate attempt.
* After an automatic load, read the resulting hot reload date. If it is still
due, record a failed hot reload attempt even though the context was installed.
Keep that context and retry. This includes identical certificates,
trust-store-only changes and replacements that expire within
{{hotReloadBeforeExpiry}}.
* After a failed automatic attempt, retry every hour. Continue retries after
expiry.
* After a successful manual reload, reset that factory's failure count and
schedule the next automatic attempt from its loaded certificates. Run
immediately if hot reload is still due. Otherwise, retain the future hot reload
date.
h3. Replacement rules
* Manual and automatic replacements in {{SslContextFactory}} use the same
loading path. Do not reject own certificates solely because they have expired
or are not yet valid, including at startup. Accept equal, earlier or later
expiry, identical certificates and trust-store-only changes. Peers decide
whether to accept certificates during TLS connections using their configured
trust managers.
* Publish a replacement only after loading and context creation succeed. A
loading or context creation failure keeps the current context and hot reload
date. Established connections remain open; new connections use the accepted
context.
* The manual command reports {{reloaded}} after a successful load, including
when the material has not changed. Verify new TLS connections to confirm the
expected certificates and peer trust.
* Factories extending only {{AbstractSslContextFactory}} keep their own manual
loading behavior.
h3. API and execution
Expose {{getHotReloadTime()}} and {{void hotReload()}} on
{{SslContextFactory}}. Add {{setHotReloadBeforeExpiry(long)}} and
{{getHotReloadBeforeExpiry()}} in milliseconds; reject values that are not
positive. Preserve {{final void init()}}. The getter returns an
epoch-millisecond date, or {{0}} if the current certificates were loaded with
hot reload disabled or have not been loaded yet.
{{hotReload()}} checks whether loading is due before calling {{init()}}. It
skips loading when disabled, not initialized or not due. A due call waits for
any active factory load; it does not recheck the date after that wait. Loading
or context creation failures throw. After a normal return, the node rereads the
date to decide whether hot reload succeeded.
Run automatic attempts and manual reloads of enabled factories on one executor
per node. Before the hot reload registry starts, reject manual reloads of
enabled factories without loading material. Each factory serializes its loads.
Each node keeps its own schedule; nodes sharing a factory may query the source
in the same window. Startup may load the same factory twice, sequentially.
On shutdown, cancel queued work, interrupt the running executor task and wait
for it to finish. Do not count failures after the executor shuts down. A task
already waiting for a shared factory may still load and install a replacement
after shutdown starts. Startup initialization already waiting for the factory
may also proceed. Custom sources must bound load time so shutdown can finish.
h3. Feedback and checks
* Log with the affected transports: each scheduled attempt time, the start and
success of each attempt at INFO, and failures with the reason and the retry
time at WARN.
* {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on
the node. Reset a factory's count after a successful manual reload or an
automatic attempt after which hot reload is no longer due. Manual failures and
shutdown cancellation do not increment it.
* Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so
operators can track attempt frequency. Count calls per factory, including
errors and automatic skips. A factory shared by several transports counts once
per attempt. Exclude startup, direct application calls and queued calls
cancelled before execution.
* Test dates, loading expired and not yet valid certificates, peer trust
checks, custom sources, identical and trust-only loads, error recovery,
scheduling, shared factories, shutdown and metrics. Verify hot reload through a
new TLS connection and reload through the manual command.
* Document preparation, trust rollout, command results and recovery. Explain
that a failed hot reload can leave newly loaded material in use when hot reload
remains due.
Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command and
certificate inventory are outside this ticket.
[Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]
was:
Administrators can reload TLS certificates with {{control.sh --ssl reload}}.
Add optional automatic hot reload so nodes can read replacements before expiry
without that command. Ignite reads the configured source; it does not issue
certificates.
h3. Schedule
* Hot reload is off by default. Add
{{SslContextFactory.setHotReloadEnabled(boolean)}} and
{{isHotReloadEnabled()}}; changing the setting requires a restart. Cover
enabled factories used by the node's started TCP transports.
* Read certificate chains from the key store actually loaded, including custom
sources. With hot reload enabled, require at least one private key and an X.509
chain for every private key.
* A chain expires when any certificate in it expires. Select the chain with the
earliest expiry. Start attempts with 15% of its lifetime remaining, measured
from the key certificate's validity start. For equal expiries, use the longest
lifetime. A date already due at startup triggers an immediate attempt.
* After an automatic load, read the resulting hot reload date. If it is still
due, record a failed hot reload attempt even though the context was installed.
Keep that context and retry. This includes identical certificates,
trust-store-only changes and replacements already within their hot reload
window.
* After a failed automatic attempt, retry with a one-minute base delay,
doubling up to one hour. Add a random delay of up to half the base delay.
Continue retries after expiry.
* After a successful manual reload, reset that factory's failure count and
schedule the next automatic attempt from its loaded certificates. Run
immediately if hot reload is still due. Otherwise, retain the future hot reload
date.
h3. Replacement rules
* Manual and automatic replacements in {{SslContextFactory}} use the same
loading path. Do not reject own certificates solely because they have expired
or are not yet valid, including at startup. Accept equal, earlier or later
expiry, identical certificates and trust-store-only changes. Peers decide
whether to accept certificates during TLS connections using their configured
trust managers.
* Publish a replacement only after loading and context creation succeed. A
loading or context creation failure keeps the current context and hot reload
date. Established connections remain open; new connections use the accepted
context.
* The manual command reports {{reloaded}} after a successful load, including
when the material has not changed. Verify new TLS connections to confirm the
expected certificates and peer trust.
* Factories extending only {{AbstractSslContextFactory}} keep their own manual
loading behavior.
h3. API and execution
Expose {{getHotReloadTime()}} and {{void hotReload()}} on
{{SslContextFactory}}. Preserve {{final void init()}}. The getter returns an
epoch-millisecond date, or {{0}} if the current certificates were loaded with
hot reload disabled or have not been loaded yet.
{{hotReload()}} checks whether loading is due before calling {{init()}}. It
skips loading when disabled, not initialized or not due. A due call waits for
any active factory load; it does not recheck the date after that wait. Loading
or context creation failures throw. After a normal return, the node rereads the
date to decide whether hot reload succeeded.
Run automatic attempts and manual reloads of enabled factories on one executor
per node. Before the hot reload registry starts, reject manual reloads of
enabled factories without loading material. Each factory serializes its loads.
Each node keeps its own schedule; nodes sharing a factory may query the source
in the same window. Startup may load the same factory twice, sequentially.
On shutdown, cancel queued work, interrupt the running executor task and wait
for it to finish. Do not count failures after the executor shuts down. A task
already waiting for a shared factory may still load and install a replacement
after shutdown starts. Startup initialization already waiting for the factory
may also proceed. Custom sources must bound load time so shutdown can finish.
h3. Feedback and checks
* Log failures and the next attempt with the affected transports.
* {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on
the node. Reset a factory's count after a successful manual reload or an
automatic attempt after which hot reload is no longer due. Manual failures and
shutdown cancellation do not increment it.
* Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so
operators can track attempt frequency. Count calls per factory, including
errors and automatic skips. A factory shared by several transports counts once
per attempt. Exclude startup, direct application calls and queued calls
cancelled before execution.
* Test dates, loading expired and not yet valid certificates, peer trust
checks, custom sources, identical and trust-only loads, error recovery,
scheduling, shared factories, shutdown and metrics. Verify hot reload through a
new TLS connection and reload through the manual command.
* Document preparation, trust rollout, command results and recovery. Explain
that a failed hot reload can leave newly loaded material in use when hot reload
remains due.
Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command and
certificate inventory are outside this ticket.
[Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]
> Hot-reload TLS certificates automatically before expiry
> -------------------------------------------------------
>
> Key: IGNITE-29211
> URL: https://issues.apache.org/jira/browse/IGNITE-29211
> Project: Ignite
> Issue Type: Sub-task
> Reporter: Anton Vinogradov
> Assignee: Anton Vinogradov
> Priority: Major
> Labels: ise
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Administrators can reload TLS certificates with {{control.sh --ssl reload}}.
> Add optional automatic hot reload so nodes can read replacements before
> expiry without that command. Ignite reads the configured source; it does not
> issue certificates.
> h3. Schedule
> * Hot reload is off by default. Add
> {{SslContextFactory.setHotReloadEnabled(boolean)}} and
> {{isHotReloadEnabled()}}; changing the setting requires a restart. Cover
> enabled factories used by the node's started TCP transports.
> * Read certificate chains from the key store actually loaded, including
> custom sources. With hot reload enabled, require at least one private key
> with an X.509 chain. Ignore keys without such a chain, as TLS does.
> * A chain expires when any certificate in it expires. Start attempts
> {{hotReloadBeforeExpiry}} before the earliest chain expiry, 7 days by
> default. A date already due at startup triggers an immediate attempt.
> * After an automatic load, read the resulting hot reload date. If it is still
> due, record a failed hot reload attempt even though the context was
> installed. Keep that context and retry. This includes identical certificates,
> trust-store-only changes and replacements that expire within
> {{hotReloadBeforeExpiry}}.
> * After a failed automatic attempt, retry every hour. Continue retries after
> expiry.
> * After a successful manual reload, reset that factory's failure count and
> schedule the next automatic attempt from its loaded certificates. Run
> immediately if hot reload is still due. Otherwise, retain the future hot
> reload date.
> h3. Replacement rules
> * Manual and automatic replacements in {{SslContextFactory}} use the same
> loading path. Do not reject own certificates solely because they have expired
> or are not yet valid, including at startup. Accept equal, earlier or later
> expiry, identical certificates and trust-store-only changes. Peers decide
> whether to accept certificates during TLS connections using their configured
> trust managers.
> * Publish a replacement only after loading and context creation succeed. A
> loading or context creation failure keeps the current context and hot reload
> date. Established connections remain open; new connections use the accepted
> context.
> * The manual command reports {{reloaded}} after a successful load, including
> when the material has not changed. Verify new TLS connections to confirm the
> expected certificates and peer trust.
> * Factories extending only {{AbstractSslContextFactory}} keep their own
> manual loading behavior.
> h3. API and execution
> Expose {{getHotReloadTime()}} and {{void hotReload()}} on
> {{SslContextFactory}}. Add {{setHotReloadBeforeExpiry(long)}} and
> {{getHotReloadBeforeExpiry()}} in milliseconds; reject values that are not
> positive. Preserve {{final void init()}}. The getter returns an
> epoch-millisecond date, or {{0}} if the current certificates were loaded with
> hot reload disabled or have not been loaded yet.
> {{hotReload()}} checks whether loading is due before calling {{init()}}. It
> skips loading when disabled, not initialized or not due. A due call waits for
> any active factory load; it does not recheck the date after that wait.
> Loading or context creation failures throw. After a normal return, the node
> rereads the date to decide whether hot reload succeeded.
> Run automatic attempts and manual reloads of enabled factories on one
> executor per node. Before the hot reload registry starts, reject manual
> reloads of enabled factories without loading material. Each factory
> serializes its loads. Each node keeps its own schedule; nodes sharing a
> factory may query the source in the same window. Startup may load the same
> factory twice, sequentially.
> On shutdown, cancel queued work, interrupt the running executor task and wait
> for it to finish. Do not count failures after the executor shuts down. A task
> already waiting for a shared factory may still load and install a replacement
> after shutdown starts. Startup initialization already waiting for the factory
> may also proceed. Custom sources must bound load time so shutdown can finish.
> h3. Feedback and checks
> * Log with the affected transports: each scheduled attempt time, the start
> and success of each attempt at INFO, and failures with the reason and the
> retry time at WARN.
> * {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on
> the node. Reset a factory's count after a successful manual reload or an
> automatic attempt after which hot reload is no longer due. Manual failures
> and shutdown cancellation do not increment it.
> * Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so
> operators can track attempt frequency. Count calls per factory, including
> errors and automatic skips. A factory shared by several transports counts
> once per attempt. Exclude startup, direct application calls and queued calls
> cancelled before execution.
> * Test dates, loading expired and not yet valid certificates, peer trust
> checks, custom sources, identical and trust-only loads, error recovery,
> scheduling, shared factories, shutdown and metrics. Verify hot reload through
> a new TLS connection and reload through the manual command.
> * Document preparation, trust rollout, command results and recovery. Explain
> that a failed hot reload can leave newly loaded material in use when hot
> reload remains due.
> Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command
> and certificate inventory are outside this ticket.
> [Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)