[ 
https://issues.apache.org/jira/browse/IGNITE-29211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18145424#comment-18145424
 ] 

Anton Vinogradov commented on IGNITE-29211:
-------------------------------------------

[Ignite PR Checker|https://ignite-pr-checker.is-a.dev/ignite?pr=13786] verdict 
for PR 13786 · RunAll build 
[9499819|https://ci2.ignite.apache.org/build/9499819] · tested 
[a5cfeb9|https://github.com/apache/ignite/pull/13786/commits/a5cfeb9d91d7da63fa63dcfdd85fb6d409ddc6ad]
 · 149 suites ran, 0 reused

(!) *This run doesn't cover the PR fully:*
- 2 suites have no reliable result (non-zero exit code; execution timeout · 
non-zero exit code)

Everything below is what it did manage to say.

(!) *Broken suites* (no reliable result):
- \[Check Code Style Ducktests\]: non-zero exit code
- Thin client: Node.js: execution timeout · non-zero exit code

(?) *No blockers found — but the run above can't prove the PR is clean.* 113 
failing tests are not blamed on this PR. Re-run after you handle what is listed 
above.

_What each label means and what to do: [verdict 
glossary|https://github.com/anton-vinogradov/ignite-pr-checker/blob/main/docs/features.md#verdict-glossary]._

♻️ Settled after 2 auto re-run waves: #1 — 5 suites that failed mid-run; #2 — 3 
broken suites.

> Hot-reload TLS certificates automatically before expiry
> -------------------------------------------------------
>
>                 Key: IGNITE-29211
>                 URL: https://issues.apache.org/jira/browse/IGNITE-29211
>             Project: Ignite
>          Issue Type: Sub-task
>            Reporter: Anton Vinogradov
>            Assignee: Anton Vinogradov
>            Priority: Major
>              Labels: ise
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Administrators can reload TLS certificates with {{control.sh --ssl reload}}. 
> Add optional automatic hot reload so nodes can read replacements before 
> expiry without that command. Ignite reads the configured source; it does not 
> issue certificates.
> h3. Schedule
> * Hot reload is off by default. Add 
> {{SslContextFactory.setHotReloadEnabled(boolean)}} and 
> {{isHotReloadEnabled()}}; changing the setting requires a restart. Cover 
> enabled factories used by the node's started TCP transports.
> * Read certificate chains from the key store actually loaded, including 
> custom sources. With hot reload enabled, require at least one private key and 
> an X.509 chain for every private key.
> * A chain expires when any certificate in it expires. Select the chain with 
> the earliest expiry. Start attempts with 15% of its lifetime remaining, 
> measured from the key certificate's validity start. For equal expiries, use 
> the longest lifetime. A date already due at startup triggers an immediate 
> attempt.
> * After an automatic load, read the resulting hot reload date. If it is still 
> due, record a failed hot reload attempt even though the context was 
> installed. Keep that context and retry. This includes identical certificates, 
> trust-store-only changes and replacements already within their hot reload 
> window.
> * After a failed automatic attempt, retry with a one-minute base delay, 
> doubling up to one hour. Add a random delay of up to half the base delay. 
> Continue retries after expiry.
> * After a successful manual reload, reset that factory's failure count and 
> schedule the next automatic attempt from its loaded certificates. Run 
> immediately if hot reload is still due. Otherwise, retain the future hot 
> reload date.
> h3. Replacement rules
> * Manual and automatic replacements in {{SslContextFactory}} use the same 
> loading path. Do not reject own certificates solely because they have expired 
> or are not yet valid, including at startup. Accept equal, earlier or later 
> expiry, identical certificates and trust-store-only changes. Peers decide 
> whether to accept certificates during TLS connections using their configured 
> trust managers.
> * Publish a replacement only after loading and context creation succeed. A 
> loading or context creation failure keeps the current context and hot reload 
> date. Established connections remain open; new connections use the accepted 
> context.
> * The manual command reports {{reloaded}} after a successful load, including 
> when the material has not changed. Verify new TLS connections to confirm the 
> expected certificates and peer trust.
> * Factories extending only {{AbstractSslContextFactory}} keep their own 
> manual loading behavior.
> h3. API and execution
> Expose {{getHotReloadTime()}} and {{void hotReload()}} on 
> {{SslContextFactory}}. Preserve {{final void init()}}. The getter returns an 
> epoch-millisecond date, or {{0}} if the current certificates were loaded with 
> hot reload disabled or have not been loaded yet.
> {{hotReload()}} checks whether loading is due before calling {{init()}}. It 
> skips loading when disabled, not initialized or not due. A due call waits for 
> any active factory load; it does not recheck the date after that wait. 
> Loading or context creation failures throw. After a normal return, the node 
> rereads the date to decide whether hot reload succeeded.
> Run automatic attempts and manual reloads of enabled factories on one 
> executor per node. Before the hot reload registry starts, reject manual 
> reloads of enabled factories without loading material. Each factory 
> serializes its loads. Each node keeps its own schedule; nodes sharing a 
> factory may query the source in the same window. Startup may load the same 
> factory twice, sequentially.
> On shutdown, cancel queued work, interrupt the running executor task and wait 
> for it to finish. Do not count failures after the executor shuts down. A task 
> already waiting for a shared factory may still load and install a replacement 
> after shutdown starts. Startup initialization already waiting for the factory 
> may also proceed. Custom sources must bound load time so shutdown can finish.
> h3. Feedback and checks
> * Log failures and the next attempt with the affected transports.
> * {{ssl.ConsecutiveHotReloadFailures}} sums consecutive automatic failures on 
> the node. Reset a factory's count after a successful manual reload or an 
> automatic attempt after which hot reload is no longer due. Manual failures 
> and shutdown cancellation do not increment it.
> * Add {{ssl.ManualReloadAttempts}} and {{ssl.AutomaticHotReloadAttempts}} so 
> operators can track attempt frequency. Count calls per factory, including 
> errors and automatic skips. A factory shared by several transports counts 
> once per attempt. Exclude startup, direct application calls and queued calls 
> cancelled before execution.
> * Test dates, loading expired and not yet valid certificates, peer trust 
> checks, custom sources, identical and trust-only loads, error recovery, 
> scheduling, shared factories, shutdown and metrics. Verify hot reload through 
> a new TLS connection and reload through the manual command.
> * Document preparation, trust rollout, command results and recovery. Explain 
> that a failed hot reload can leave newly loaded material in use when hot 
> reload remains due.
> Depends on IGNITE-29210. IGNITE-29212 adds HTTPS. A separate status command 
> and certificate inventory are outside this ticket.
> [Implementation in PR #13786|https://github.com/apache/ignite/pull/13786]



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to