[
https://issues.apache.org/jira/browse/HDDS-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17743323#comment-17743323
]
István Fajth commented on HDDS-9016:
------------------------------------
After I was able to reproduce the issue with some additional logging added, I
figured out what is causing this problem, and why it is intermittent.
In a reproduction:
Leader OM log:
{code}
2023-07-14 21:56:20,854 [RootCaRotationPoller] INFO
client.RootCaRotationPoller: Some root CAs are not known to the client out of
the root CAs known to the SCMs. Root CA Cert ids known to the client: 1, 2.
Root CA Cert ids from SCM not known by the client: 3
{code}
Client side command run:
{code}
21:56:05.299 INFO Running command 'ozone sh key get
/rootca-volume/rootca-bucket/rootca-key /tmp/key-80823 2>&1'.
21:56:21.808 INFO ${rc} = 255
21:56:21.808 INFO ${output} = java.util.concurrent.ExecutionException:
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNAVAILABLE: io
exception
{code}
This happens because the rootCA certificate is rotated during the client is
trying to communicate with the cluster, and the client side is able to handle a
failure happens because of this, while there is also a similar problem due to
the timing of how the services starts to use the new rootCA certificate.
This will be fixed by the combination of HDDS-8958 and HDDS-9015
> Failed to Read data from previously created key
> -----------------------------------------------
>
> Key: HDDS-9016
> URL: https://issues.apache.org/jira/browse/HDDS-9016
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Attila Doroszlai
> Assignee: Sammi Chen
> Priority: Major
>
> {code:title=https://github.com/adoroszlai/ozone-build-results/blob/master/2023/07/14/24239/acceptance-HA-secure/output.log}
> Root-Ca-Rotation-Client-Checks :: Generate data
> ==============================================================================
> Create a volume and bucket | PASS |
> ------------------------------------------------------------------------------
> Create key | PASS |
> ------------------------------------------------------------------------------
> Read data from previously created key | FAIL |
> 255 != 0
> {code}
> CC [~pifta], [~sgal]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
