[ 
https://issues.apache.org/jira/browse/IGNITE-29211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-29211:
--------------------------------------
    Description: 
Administrators can reload TLS certificates with {{control.sh --ssl reload}}. 
Add optional automatic hot reload so nodes can read replacements before expiry 
without that command. Ignite reads the configured source; it does not issue 
certificates.

h3. Schedule

* Hot reload is off by default. Add 
{{SslContextFactory.setHotReloadEnabled(boolean)}} and 
{{isHotReloadEnabled()}}; changing the setting requires a restart. Cover 
enabled factories used by the node's started TCP transports.
* Read certificate chains from the key store actually loaded, including custom 
sources. Compute the date from private keys with an X.509 chain and ignore 
other keys, as TLS does. If there is no such key, log a warning and schedule no 
hot reload until a reload brings one.
* A chain expires when any certificate in it expires. Start attempts 
{{hotReloadBeforeExpiryMs}} before the earliest chain expiry, 7 days by 
default. A date already due at startup triggers an immediate attempt.
* After an automatic load, read the resulting hot reload date. If it is still 
due, record a failed hot reload attempt even though the context was installed. 
Keep that context and retry. This includes identical certificates, 
trust-store-only changes and replacements that expire within 
{{hotReloadBeforeExpiryMs}}.
* After a failed automatic attempt, retry every hour. Continue retries after 
expiry.
* After a successful manual reload, reset that factory's failure count and 
schedule the next automatic attempt from its loaded certificates. Run 
immediately if hot reload is still due. Otherwise, retain the future hot reload 
date.

h3. Replacement rules

* Manual and automatic replacements in {{SslContextFactory}} use the same 
loading path. Do not reject own certificates solely because they have expired 
or are not yet valid, including at startup. Accept equal, earlier or later 
expiry, identical certificates and trust-store-only changes. Peers decide 
whether to accept certificates during TLS connections using their configured 
trust managers.
* Publish a replacement only after loading and context creation succeed. A 
loading or context creation failure keeps the current context and hot reload 
date. Established connections remain open; new connections use the accepted 
context.
* The manual command reports {{reloaded}} after a successful load, including 
when the material has not changed. Verify new TLS connections to confirm the 
expected certificates and peer trust.
* Factories extending only {{AbstractSslContextFactory}} keep their own manual 
loading behavior.

h3. API and execution

Expose {{getHotReloadTime()}} and {{void hotReload()}} on 
{{SslContextFactory}}. Add {{setHotReloadBeforeExpiryMs(long)}} and 
{{getHotReloadBeforeExpiryMs()}} in milliseconds; reject values that are not 
positive. Preserve {{final void init()}}. The getter returns an 
epoch-millisecond date, or {{0}} if the current certificates were loaded with 
hot reload disabled, have not been loaded yet, or have no private key with an 
X.509 chain.

{{hotReload()}} checks whether loading is due before calling {{init()}}. It 
skips loading when disabled, not initialized or not due. A due call waits for 
any active factory load; it does not recheck the date after that wait. Loading 
or context creation failures throw. After a normal return, the node rereads the 
date to decide whether hot reload succeeded.

Run automatic attempts and manual reloads of enabled factories on one executor 
per node. Before the hot reload registry starts, reject manual reloads of 
enabled factories without loading material. Each factory serializes its loads. 
Each node keeps its own schedule; nodes sharing a factory may query the source 
in the same window. Startup may load the same factory twice, sequentially.

On shutdown, cancel queued work and interrupt the running executor task without 
waiting for it. Do not count failures or schedule attempts after the executor 
shuts down. The interrupted task may still finish its load after the node 
stops. Startup initialization already waiting for the factory may also proceed. 
Custom sources must bound load time, because a blocked load delays other hot 
reloads and manual reloads.

h3. Feedback and checks

* Log with the affected transports: each scheduled attempt time, the start and 
success of each attempt at INFO, and failures with the reason and the retry 
time at WARN.
* {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on 
the node. Reset a factory's count after a successful manual reload or an 
automatic attempt after which hot reload is no longer due. Manual failures and 
shutdown cancellation do not increment it.
* Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so 
operators can track attempt frequency. Count calls per factory, including 
errors and automatic skips. A factory shared by several transports counts once 
per attempt. Exclude startup, direct application calls and queued calls 
cancelled before execution.
* Test dates, loading expired and not yet valid certificates, peer trust 
checks, custom sources, identical and trust-only loads, error recovery, 
scheduling, shared factories, shutdown and metrics. Verify hot reload through a 
new TLS connection and reload through the manual command.
* Document preparation, trust rollout, command results and recovery. Explain 
that a failed hot reload can leave newly loaded material in use when hot reload 
remains due.

Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command and 
certificate inventory are outside this ticket.

[Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]

  was:
Administrators can reload TLS certificates with {{control.sh --ssl reload}}. 
Add optional automatic hot reload so nodes can read replacements before expiry 
without that command. Ignite reads the configured source; it does not issue 
certificates.

h3. Schedule

* Hot reload is off by default. Add 
{{SslContextFactory.setHotReloadEnabled(boolean)}} and 
{{isHotReloadEnabled()}}; changing the setting requires a restart. Cover 
enabled factories used by the node's started TCP transports.
* Read certificate chains from the key store actually loaded, including custom 
sources. Compute the date from private keys with an X.509 chain and ignore 
other keys, as TLS does. If there is no such key, log a warning and schedule no 
hot reload until a reload brings one.
* A chain expires when any certificate in it expires. Start attempts 
{{hotReloadBeforeExpiryMs}} before the earliest chain expiry, 7 days by 
default. A date already due at startup triggers an immediate attempt.
* After an automatic load, read the resulting hot reload date. If it is still 
due, record a failed hot reload attempt even though the context was installed. 
Keep that context and retry. This includes identical certificates, 
trust-store-only changes and replacements that expire within 
{{hotReloadBeforeExpiryMs}}.
* After a failed automatic attempt, retry every hour. Continue retries after 
expiry.
* After a successful manual reload, reset that factory's failure count and 
schedule the next automatic attempt from its loaded certificates. Run 
immediately if hot reload is still due. Otherwise, retain the future hot reload 
date.

h3. Replacement rules

* Manual and automatic replacements in {{SslContextFactory}} use the same 
loading path. Do not reject own certificates solely because they have expired 
or are not yet valid, including at startup. Accept equal, earlier or later 
expiry, identical certificates and trust-store-only changes. Peers decide 
whether to accept certificates during TLS connections using their configured 
trust managers.
* Publish a replacement only after loading and context creation succeed. A 
loading or context creation failure keeps the current context and hot reload 
date. Established connections remain open; new connections use the accepted 
context.
* The manual command reports {{reloaded}} after a successful load, including 
when the material has not changed. Verify new TLS connections to confirm the 
expected certificates and peer trust.
* Factories extending only {{AbstractSslContextFactory}} keep their own manual 
loading behavior.

h3. API and execution

Expose {{getHotReloadTime()}} and {{void hotReload()}} on 
{{SslContextFactory}}. Add {{setHotReloadBeforeExpiryMs(long)}} and 
{{getHotReloadBeforeExpiryMs()}} in milliseconds; reject values that are not 
positive. Preserve {{final void init()}}. The getter returns an 
epoch-millisecond date, or {{0}} if the current certificates were loaded with 
hot reload disabled, have not been loaded yet, or have no private key with an 
X.509 chain.

{{hotReload()}} checks whether loading is due before calling {{init()}}. It 
skips loading when disabled, not initialized or not due. A due call waits for 
any active factory load; it does not recheck the date after that wait. Loading 
or context creation failures throw. After a normal return, the node rereads the 
date to decide whether hot reload succeeded.

Run automatic attempts and manual reloads of enabled factories on one executor 
per node. Before the hot reload registry starts, reject manual reloads of 
enabled factories without loading material. Each factory serializes its loads. 
Each node keeps its own schedule; nodes sharing a factory may query the source 
in the same window. Startup may load the same factory twice, sequentially.

On shutdown, cancel queued work, interrupt the running executor task and wait 
for it to finish. Do not count failures after the executor shuts down. A task 
already waiting for a shared factory may still load and install a replacement 
after shutdown starts. Startup initialization already waiting for the factory 
may also proceed. Custom sources must bound load time so shutdown can finish.

h3. Feedback and checks

* Log with the affected transports: each scheduled attempt time, the start and 
success of each attempt at INFO, and failures with the reason and the retry 
time at WARN.
* {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on 
the node. Reset a factory's count after a successful manual reload or an 
automatic attempt after which hot reload is no longer due. Manual failures and 
shutdown cancellation do not increment it.
* Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so 
operators can track attempt frequency. Count calls per factory, including 
errors and automatic skips. A factory shared by several transports counts once 
per attempt. Exclude startup, direct application calls and queued calls 
cancelled before execution.
* Test dates, loading expired and not yet valid certificates, peer trust 
checks, custom sources, identical and trust-only loads, error recovery, 
scheduling, shared factories, shutdown and metrics. Verify hot reload through a 
new TLS connection and reload through the manual command.
* Document preparation, trust rollout, command results and recovery. Explain 
that a failed hot reload can leave newly loaded material in use when hot reload 
remains due.

Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command and 
certificate inventory are outside this ticket.

[Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]


> Hot-reload TLS certificates automatically before expiry
> -------------------------------------------------------
>
>                 Key: IGNITE-29211
>                 URL: https://issues.apache.org/jira/browse/IGNITE-29211
>             Project: Ignite
>          Issue Type: Sub-task
>            Reporter: Anton Vinogradov
>            Assignee: Anton Vinogradov
>            Priority: Major
>              Labels: ise
>          Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Administrators can reload TLS certificates with {{control.sh --ssl reload}}. 
> Add optional automatic hot reload so nodes can read replacements before 
> expiry without that command. Ignite reads the configured source; it does not 
> issue certificates.
> h3. Schedule
> * Hot reload is off by default. Add 
> {{SslContextFactory.setHotReloadEnabled(boolean)}} and 
> {{isHotReloadEnabled()}}; changing the setting requires a restart. Cover 
> enabled factories used by the node's started TCP transports.
> * Read certificate chains from the key store actually loaded, including 
> custom sources. Compute the date from private keys with an X.509 chain and 
> ignore other keys, as TLS does. If there is no such key, log a warning and 
> schedule no hot reload until a reload brings one.
> * A chain expires when any certificate in it expires. Start attempts 
> {{hotReloadBeforeExpiryMs}} before the earliest chain expiry, 7 days by 
> default. A date already due at startup triggers an immediate attempt.
> * After an automatic load, read the resulting hot reload date. If it is still 
> due, record a failed hot reload attempt even though the context was 
> installed. Keep that context and retry. This includes identical certificates, 
> trust-store-only changes and replacements that expire within 
> {{hotReloadBeforeExpiryMs}}.
> * After a failed automatic attempt, retry every hour. Continue retries after 
> expiry.
> * After a successful manual reload, reset that factory's failure count and 
> schedule the next automatic attempt from its loaded certificates. Run 
> immediately if hot reload is still due. Otherwise, retain the future hot 
> reload date.
> h3. Replacement rules
> * Manual and automatic replacements in {{SslContextFactory}} use the same 
> loading path. Do not reject own certificates solely because they have expired 
> or are not yet valid, including at startup. Accept equal, earlier or later 
> expiry, identical certificates and trust-store-only changes. Peers decide 
> whether to accept certificates during TLS connections using their configured 
> trust managers.
> * Publish a replacement only after loading and context creation succeed. A 
> loading or context creation failure keeps the current context and hot reload 
> date. Established connections remain open; new connections use the accepted 
> context.
> * The manual command reports {{reloaded}} after a successful load, including 
> when the material has not changed. Verify new TLS connections to confirm the 
> expected certificates and peer trust.
> * Factories extending only {{AbstractSslContextFactory}} keep their own 
> manual loading behavior.
> h3. API and execution
> Expose {{getHotReloadTime()}} and {{void hotReload()}} on 
> {{SslContextFactory}}. Add {{setHotReloadBeforeExpiryMs(long)}} and 
> {{getHotReloadBeforeExpiryMs()}} in milliseconds; reject values that are not 
> positive. Preserve {{final void init()}}. The getter returns an 
> epoch-millisecond date, or {{0}} if the current certificates were loaded with 
> hot reload disabled, have not been loaded yet, or have no private key with an 
> X.509 chain.
> {{hotReload()}} checks whether loading is due before calling {{init()}}. It 
> skips loading when disabled, not initialized or not due. A due call waits for 
> any active factory load; it does not recheck the date after that wait. 
> Loading or context creation failures throw. After a normal return, the node 
> rereads the date to decide whether hot reload succeeded.
> Run automatic attempts and manual reloads of enabled factories on one 
> executor per node. Before the hot reload registry starts, reject manual 
> reloads of enabled factories without loading material. Each factory 
> serializes its loads. Each node keeps its own schedule; nodes sharing a 
> factory may query the source in the same window. Startup may load the same 
> factory twice, sequentially.
> On shutdown, cancel queued work and interrupt the running executor task 
> without waiting for it. Do not count failures or schedule attempts after the 
> executor shuts down. The interrupted task may still finish its load after the 
> node stops. Startup initialization already waiting for the factory may also 
> proceed. Custom sources must bound load time, because a blocked load delays 
> other hot reloads and manual reloads.
> h3. Feedback and checks
> * Log with the affected transports: each scheduled attempt time, the start 
> and success of each attempt at INFO, and failures with the reason and the 
> retry time at WARN.
> * {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on 
> the node. Reset a factory's count after a successful manual reload or an 
> automatic attempt after which hot reload is no longer due. Manual failures 
> and shutdown cancellation do not increment it.
> * Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so 
> operators can track attempt frequency. Count calls per factory, including 
> errors and automatic skips. A factory shared by several transports counts 
> once per attempt. Exclude startup, direct application calls and queued calls 
> cancelled before execution.
> * Test dates, loading expired and not yet valid certificates, peer trust 
> checks, custom sources, identical and trust-only loads, error recovery, 
> scheduling, shared factories, shutdown and metrics. Verify hot reload through 
> a new TLS connection and reload through the manual command.
> * Document preparation, trust rollout, command results and recovery. Explain 
> that a failed hot reload can leave newly loaded material in use when hot 
> reload remains due.
> Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command 
> and certificate inventory are outside this ticket.
> [Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to