[
https://issues.apache.org/jira/browse/HDDS-9217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-9217:
---------------------------------
Labels: pull-request-available (was: )
> Refine certificate renewer service to avoid it scheduled ahead of time
> ----------------------------------------------------------------------
>
> Key: HDDS-9217
> URL: https://issues.apache.org/jira/browse/HDDS-9217
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sammi Chen
> Assignee: Sammi Chen
> Priority: Major
> Labels: pull-request-available
>
> Here the first rotation is om3 is delayed by ~3mins. The rotation should
> happen around 08:26:12, but actually the new certificate start time is
> 08:29:32.
> bash-4.2$ ozone admin cert list -c 1000 --role=datanode | grep om3
> 10394014160981 Fri Aug 25 08:16:12 UTC 2023 Fri Aug 25 08:36:12 UTC 2023
>
> CN=om3,OU=8ca275d2-c634-4700-a8e3-4bd0bfcb12dd,O=CID-8b191bcb-7415-4bcb-9be0-c0f01f6ac078
>
> [email protected],OU=8ca275d2-c634-4700-a8e3-4bd0bfcb12dd,O=CID-8b191bcb-7415-4bcb-9be0-c0f01f6ac078
> 11193839930971 Fri Aug 25 08:29:32 UTC 2023 Fri Aug 25 08:49:32 UTC 2023
>
> CN=om3,OU=8ca275d2-c634-4700-a8e3-4bd0bfcb12dd,O=CID-8b191bcb-7415-4bcb-9be0-c0f01f6ac078
>
> [email protected],OU=8ca275d2-c634-4700-a8e3-4bd0bfcb12dd,O=CID-8b191bcb-7415-4bcb-9be0-c0f01f6ac078
> 11793471401494 Fri Aug 25 08:39:32 UTC 2023 Fri Aug 25 08:59:32 UTC 2023
>
> CN=om3,OU=8ca275d2-c634-4700-a8e3-4bd0bfcb12dd,O=CID-8b191bcb-7415-4bcb-9be0-c0f01f6ac078
>
> [email protected],OU=8ca275d2-c634-4700-a8e3-4bd0bfcb12dd,O=CID-8b191bcb-7415-4bcb-9be0-c0f01f6ac078
> 12393664601601 Fri Aug 25 08:49:32 UTC 2023 Fri Aug 25 09:09:32 UTC 2023
>
> CN=om3,OU=8ca275d2-c634-4700-a8e3-4bd0bfcb12dd,O=CID-8b191bcb-7415-4bcb-9be0-c0f01f6ac078
>
> [email protected],OU=8ca275d2-c634-4700-a8e3-4bd0bfcb12dd,O=CID-8b191bcb-7415-4bcb-9be0-c0f01f6ac078
> Here is the key logs of this OM3 cert rotation. From the log, we can see that
> on 08:26:12, the rotation task executed, but it found the certificate was
> still outside of the renew grace period by PT0.000025S, so it exited the task
> this time. Next try happened after 3m20s, this time the certificate is
> renewed.
> sammi@SAMMICHEN-MB0 ozonesecure-ha % cat om3.log| grep
> security.OMCertificateClient | grep "Current certificate"
> 2023-08-25 08:26:12,000 [om-CertificateRenewerService] INFO
> security.OMCertificateClient: Current certificate 10394014160981 hasn't
> entered the renew grace period. Remaining period is PT0.000025S.
> 2023-08-25 08:29:32,065 [om-CertificateRenewerService] INFO
> security.OMCertificateClient: Current certificate 10394014160981 needs to be
> renewed remaining grace period PT0S. Forced renewal due to root ca rotation:
> false.
> 2023-08-25 08:32:52,066 [om-CertificateRenewerService] INFO
> security.OMCertificateClient: Current certificate 11193839930971 hasn't
> entered the renew grace period. Remaining period is PT6M39.93403S.
> 2023-08-25 08:36:12,076 [om-CertificateRenewerService] INFO
> security.OMCertificateClient: Current certificate 11193839930971 hasn't
> entered the renew grace period. Remaining period is PT3M19.924957S.
> 2023-08-25 08:39:32,068 [om-CertificateRenewerService] INFO
> security.OMCertificateClient: Current certificate 11193839930971 needs to be
> renewed remaining grace period PT0S. Forced renewal due to root ca rotation:
> false.
> 2023-08-25 08:42:52,069 [om-CertificateRenewerService] INFO
> security.OMCertificateClient: Current certificate 11793471401494 hasn't
> entered the renew grace period. Remaining period is PT6M39.930225S.
> 2023-08-25 08:46:12,082 [om-CertificateRenewerService] INFO
> security.OMCertificateClient: Current certificate 11793471401494 hasn't
> entered the renew grace period. Remaining period is PT3M19.917761S.
> 2023-08-25 08:49:32,083 [om-CertificateRenewerService] INFO
> security.OMCertificateClient: Current certificate 11793471401494 needs to be
> renewed remaining grace period PT0S. Forced renewal due to root ca rotation:
> false.
>
> In the cert rotation implementation, the renew task is scheduled like this
> this.executorService.scheduleAtFixedRate(
> new CertificateRenewerService(false, () -> {
> }),
> timeBeforeGracePeriod, interval, TimeUnit.MILLISECONDS);
> The timeBeforeGracePeriod is the time should be passed until certificate's
> renew grace period reached moment, and interval is 1/3 of renew grace period.
> It looks like the Java started the task PT0.000025S ahead of time, so that
> the first task exited because of grace period is not yet reached.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
