[ 
https://issues.apache.org/jira/browse/IGNITE-29211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-29211:
--------------------------------------
    Description: 
Administrators can reload TLS certificates with {{control.sh --ssl reload}}. 
Add optional automatic hot reload so nodes can read replacements before expiry 
without that command. Ignite reads the configured source; it does not issue 
certificates.

h3. Schedule

* Hot reload is off by default. Add 
{{SslContextFactory.setHotReloadEnabled(boolean)}} and 
{{isHotReloadEnabled()}}; changing the setting requires a restart. Cover 
enabled factories used by the node's started TCP transports.
* Read certificate chains from the key store actually loaded, including custom 
sources. With hot reload enabled, require at least one private key and an X.509 
chain for every private key.
* A chain expires when any certificate in it expires. Select the chain with the 
earliest expiry. Start attempts with 15% of its lifetime remaining, measured 
from the key certificate's validity start. For equal expiries, use the longest 
lifetime. A date already due at startup triggers an immediate attempt.
* After an automatic load, read the resulting hot reload date. If it is still 
due, record a failed hot reload attempt even though the valid context was 
installed. Keep that context and retry. This includes identical certificates, 
trust-store-only changes and replacements already within their hot reload 
window.
* After a failed automatic attempt, retry with a one-minute base delay, 
doubling up to one hour. Add a random delay of up to half the base delay. 
Continue retries after expiry.
* After a successful manual reload, reset that factory's failure count and 
schedule the next automatic attempt from its loaded certificates. Run 
immediately if hot reload is still due. Otherwise, retain the future hot reload 
date.

h3. Replacement rules

* Manual and automatic replacements in {{SslContextFactory}} use the same 
loading path. Check every certificate in the private-key chains against the 
current date. Accept valid certificates with equal, earlier or later expiry, 
including identical certificates and trust-store-only changes. The same 
validity check applies at startup, including in client applications and the 
control script. Expired or not yet valid certificates prevent startup.
* Publish a replacement only after loading and validation succeed. A loading or 
validation failure keeps the current context and hot reload date. Established 
connections remain open; new connections use the accepted context.
* The manual command reports {{reloaded}} after a successful load, including 
when the material has not changed. Verify new TLS connections to confirm the 
expected certificates and peer trust.
* Factories extending only {{AbstractSslContextFactory}} keep their own manual 
loading behavior.

h3. API and execution

Expose {{getHotReloadTime()}} and {{void hotReload()}} on 
{{SslContextFactory}}. Preserve {{final void init()}}. The getter returns an 
epoch-millisecond date, or {{0}} if the current certificates were loaded with 
hot reload disabled or have not been loaded yet.

{{hotReload()}} checks whether loading is due before calling {{init()}}. It 
skips loading when disabled, not initialized or not due. A due call waits for 
any active factory load; it does not recheck the date after that wait. Loading 
and validation failures throw. After a normal return, the node rereads the date 
to decide whether hot reload succeeded.

Run automatic attempts and manual reloads of enabled factories on one executor 
per node. Before the hot reload registry starts, reject manual reloads of 
enabled factories without loading material. Each factory serializes its loads. 
Each node keeps its own schedule; nodes sharing a factory may query the source 
in the same window. Startup may load the same factory twice, sequentially.

On shutdown, cancel queued work, interrupt the running executor task and wait 
for it to finish. Do not count failures after the executor shuts down. A task 
already waiting for a shared factory may still load and install a valid 
replacement after shutdown starts. Startup initialization already waiting for 
the factory may also proceed. Custom sources must bound load time so shutdown 
can finish.

h3. Feedback and checks

* Log failures and the next attempt with the affected transports.
* {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on 
the node. Reset a factory's count after a successful manual reload or an 
automatic attempt after which hot reload is no longer due. Manual failures and 
shutdown cancellation do not increment it.
* Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so 
operators can track attempt frequency. Count calls per factory, including 
errors and automatic skips. A factory shared by several transports counts once 
per attempt. Exclude startup, direct application calls and queued calls 
cancelled before execution.
* Test dates, custom sources, identical and trust-only loads, error recovery, 
scheduling, shared factories, shutdown and metrics. Verify hot reload through a 
new TLS connection and reload through the manual command.
* Document preparation, trust rollout, command results and recovery. Explain 
that a failed hot reload can leave newly loaded material in use when hot reload 
remains due.

Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command and 
certificate inventory are outside this ticket.

[Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]

  was:
Administrators can reload TLS certificates with {{control.sh --ssl reload}}. 
Add optional automatic renewal so nodes can read replacements before expiry 
without that command. Ignite reads the configured source; it does not issue 
certificates.

h3. Schedule

* Renewal is off by default. Add 
{{SslContextFactory.setRenewalEnabled(boolean)}} and {{isRenewalEnabled()}}; 
changing the setting requires a restart. Cover enabled factories used by the 
node's started TCP transports.
* Read certificate chains from the key store actually loaded, including custom 
sources. With renewal enabled, require at least one private key and an X.509 
chain for every private key.
* A chain expires when any certificate in it expires. Select the chain with the 
earliest expiry. Start attempts with 15% of its lifetime remaining, measured 
from the key certificate's validity start. For equal expiries, use the longest 
lifetime. A date already due at startup triggers an immediate attempt.
* After an automatic load, read the resulting renewal date. If it is still due, 
record a failed renewal attempt even though the valid context was installed. 
Keep that context and retry. This includes identical certificates, 
trust-store-only changes and replacements already within their renewal window.
* After a failed automatic attempt, retry with a one-minute base delay, 
doubling up to one hour. Add a random delay of up to half the base delay. 
Continue retries after expiry.
* After a successful manual reload, reset that factory's failure count and 
schedule the next automatic attempt from its loaded certificates. Run 
immediately if renewal is still due. Otherwise, retain the future renewal date.

h3. Replacement rules

* Manual and automatic replacements in {{SslContextFactory}} use the same 
loading path. Check every certificate in the private-key chains against the 
current date. Accept valid certificates with equal, earlier or later expiry, 
including identical certificates and trust-store-only changes. The same 
validity check applies at startup, including in client applications and the 
control script. Expired or not yet valid certificates prevent startup.
* Publish a replacement only after loading and validation succeed. A loading or 
validation failure keeps the current context and renewal date. Established 
connections remain open; new connections use the accepted context.
* The manual command reports {{reloaded}} after a successful load, including 
when the material has not changed. Verify new TLS connections to confirm the 
expected certificates and peer trust.
* Factories extending only {{AbstractSslContextFactory}} keep their own manual 
loading behavior.

h3. API and execution

Expose {{getRenewalTime()}} and {{void renew()}} on {{SslContextFactory}}. 
Preserve {{final void init()}}. The getter returns an epoch-millisecond date, 
or {{0}} if the current certificates were loaded with renewal disabled or have 
not been loaded yet.

{{renew()}} checks whether loading is due before calling {{init()}}. It skips 
loading when disabled, not initialized or not due. A due call waits for any 
active factory load; it does not recheck the date after that wait. Loading and 
validation failures throw. After a normal return, the node rereads the date to 
decide whether renewal succeeded.

Run automatic attempts and manual reloads of enabled factories on one executor 
per node. Before the renewal registry starts, reject manual reloads of enabled 
factories without loading material. Each factory serializes its loads. Each 
node keeps its own schedule; nodes sharing a factory may query the source in 
the same window. Startup may load the same factory twice, sequentially.

On shutdown, cancel queued work, interrupt the running executor task and wait 
for it to finish. Do not count failures after the executor shuts down. A task 
already waiting for a shared factory may still load and install a valid 
replacement after shutdown starts. Startup initialization already waiting for 
the factory may also proceed. Custom sources must bound load time so shutdown 
can finish.

h3. Feedback and checks

* Log failures and the next attempt with the affected transports.
* {{ssl.ConsecutiveRenewalFailures}} sums consecutive automatic failures on the 
node. Reset a factory's count after a successful manual reload or an automatic 
attempt after which renewal is no longer due. Manual failures and shutdown 
cancellation do not increment it.
* Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticRenewalAttempts}} so 
operators can track attempt frequency. Count calls per factory, including 
errors and automatic skips. A factory shared by several transports counts once 
per attempt. Exclude startup, direct application calls and queued calls 
cancelled before execution.
* Test dates, custom sources, identical and trust-only loads, error recovery, 
scheduling, shared factories, shutdown and metrics. Verify renewal through a 
new TLS connection and reload through the manual command.
* Document preparation, trust rollout, command results and recovery. Explain 
that a failed renewal can leave newly loaded material in use when renewal 
remains due.

Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command and 
certificate inventory are outside this ticket.

[Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]

        Summary: Hot-reload TLS certificates automatically before expiry  (was: 
Renew TLS certificates automatically before expiry)

> Hot-reload TLS certificates automatically before expiry
> -------------------------------------------------------
>
>                 Key: IGNITE-29211
>                 URL: https://issues.apache.org/jira/browse/IGNITE-29211
>             Project: Ignite
>          Issue Type: Sub-task
>            Reporter: Anton Vinogradov
>            Assignee: Anton Vinogradov
>            Priority: Major
>              Labels: ise
>          Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Administrators can reload TLS certificates with {{control.sh --ssl reload}}. 
> Add optional automatic hot reload so nodes can read replacements before 
> expiry without that command. Ignite reads the configured source; it does not 
> issue certificates.
> h3. Schedule
> * Hot reload is off by default. Add 
> {{SslContextFactory.setHotReloadEnabled(boolean)}} and 
> {{isHotReloadEnabled()}}; changing the setting requires a restart. Cover 
> enabled factories used by the node's started TCP transports.
> * Read certificate chains from the key store actually loaded, including 
> custom sources. With hot reload enabled, require at least one private key and 
> an X.509 chain for every private key.
> * A chain expires when any certificate in it expires. Select the chain with 
> the earliest expiry. Start attempts with 15% of its lifetime remaining, 
> measured from the key certificate's validity start. For equal expiries, use 
> the longest lifetime. A date already due at startup triggers an immediate 
> attempt.
> * After an automatic load, read the resulting hot reload date. If it is still 
> due, record a failed hot reload attempt even though the valid context was 
> installed. Keep that context and retry. This includes identical certificates, 
> trust-store-only changes and replacements already within their hot reload 
> window.
> * After a failed automatic attempt, retry with a one-minute base delay, 
> doubling up to one hour. Add a random delay of up to half the base delay. 
> Continue retries after expiry.
> * After a successful manual reload, reset that factory's failure count and 
> schedule the next automatic attempt from its loaded certificates. Run 
> immediately if hot reload is still due. Otherwise, retain the future hot 
> reload date.
> h3. Replacement rules
> * Manual and automatic replacements in {{SslContextFactory}} use the same 
> loading path. Check every certificate in the private-key chains against the 
> current date. Accept valid certificates with equal, earlier or later expiry, 
> including identical certificates and trust-store-only changes. The same 
> validity check applies at startup, including in client applications and the 
> control script. Expired or not yet valid certificates prevent startup.
> * Publish a replacement only after loading and validation succeed. A loading 
> or validation failure keeps the current context and hot reload date. 
> Established connections remain open; new connections use the accepted context.
> * The manual command reports {{reloaded}} after a successful load, including 
> when the material has not changed. Verify new TLS connections to confirm the 
> expected certificates and peer trust.
> * Factories extending only {{AbstractSslContextFactory}} keep their own 
> manual loading behavior.
> h3. API and execution
> Expose {{getHotReloadTime()}} and {{void hotReload()}} on 
> {{SslContextFactory}}. Preserve {{final void init()}}. The getter returns an 
> epoch-millisecond date, or {{0}} if the current certificates were loaded with 
> hot reload disabled or have not been loaded yet.
> {{hotReload()}} checks whether loading is due before calling {{init()}}. It 
> skips loading when disabled, not initialized or not due. A due call waits for 
> any active factory load; it does not recheck the date after that wait. 
> Loading and validation failures throw. After a normal return, the node 
> rereads the date to decide whether hot reload succeeded.
> Run automatic attempts and manual reloads of enabled factories on one 
> executor per node. Before the hot reload registry starts, reject manual 
> reloads of enabled factories without loading material. Each factory 
> serializes its loads. Each node keeps its own schedule; nodes sharing a 
> factory may query the source in the same window. Startup may load the same 
> factory twice, sequentially.
> On shutdown, cancel queued work, interrupt the running executor task and wait 
> for it to finish. Do not count failures after the executor shuts down. A task 
> already waiting for a shared factory may still load and install a valid 
> replacement after shutdown starts. Startup initialization already waiting for 
> the factory may also proceed. Custom sources must bound load time so shutdown 
> can finish.
> h3. Feedback and checks
> * Log failures and the next attempt with the affected transports.
> * {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on 
> the node. Reset a factory's count after a successful manual reload or an 
> automatic attempt after which hot reload is no longer due. Manual failures 
> and shutdown cancellation do not increment it.
> * Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so 
> operators can track attempt frequency. Count calls per factory, including 
> errors and automatic skips. A factory shared by several transports counts 
> once per attempt. Exclude startup, direct application calls and queued calls 
> cancelled before execution.
> * Test dates, custom sources, identical and trust-only loads, error recovery, 
> scheduling, shared factories, shutdown and metrics. Verify hot reload through 
> a new TLS connection and reload through the manual command.
> * Document preparation, trust rollout, command results and recovery. Explain 
> that a failed hot reload can leave newly loaded material in use when hot 
> reload remains due.
> Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command 
> and certificate inventory are outside this ticket.
> [Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to