[
https://issues.apache.org/jira/browse/HDDS-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-17316:
------------------------------
Epic Link: HDDS-17321
> Cap the wait between retries when loading the initial secret key from SCM
> -------------------------------------------------------------------------
>
> Key: HDDS-17316
> URL: https://issues.apache.org/jira/browse/HDDS-17316
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Security
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
>
> At startup, OM and Datanode load the current secret key from SCM in
> {{DefaultSecretKeySignerClient.loadInitialSecretKey}}. While SCM has not
> initialized secret keys yet ({{SECRET_KEY_NOT_INITIALIZED}}), it makes up to
> 100 attempts with exponential backoff that restarts every 10 retries. The
> base wait goes 4s, 8s, ..., 1024s, then 2s, randomized by +/-50%, so a single
> wait can be up to about 25 minutes and the total about 5.7 hours. The code
> comment ("1 2 4 8 ... 512", "around 200min") understates both.
> So once SCM finishes initializing its keys, an OM or Datanode can stay stuck
> for many more minutes before trying again.
> Proposed fix: keep the exponential backoff but cap the base wait at 16s (4s,
> 8s, 16s, 16s, ...), so a single wait is at most 23s and the total for 100
> attempts is around 26 minutes.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
