[
https://issues.apache.org/jira/browse/HDDS-8140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Rose updated HDDS-8140:
-----------------------------
Description:
This message was observed in the logs on startup for 250 containers. The same
message occurred on 3/6 datanodes for the same set of containers on each node:
{code}
2023-02-16 16:43:05,799 WARN
org.apache.hadoop.ozone.container.common.impl.ContainerSet: Adding container
64079 to missing container set.
{code}
This is a byproduct of HDDS-935, which is an old and involved change. In that
change there is a TODO in
[{{ContainerStateMachine#persistContainerSet}}|https://github.com/apache/ozone/blob/7f22916889b7bf39cdb31e5943cae5768f368198/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java#L279]
that says there will be a race if container delete does not go through Ratis.
When container deletion was implemented after that change, it did not go
through Ratis so the race may happen. We need to revisit this area of the code.
was:
This message was observed in the logs on startup for 250 containers. The same
message occurred on 3/6 datanodes for the same set of containers on each node:
{code}
2023-02-16 16:43:05,799 WARN
org.apache.hadoop.ozone.container.common.impl.ContainerSet: Adding container
64079 to missing container set.
{code}
This is a byproduct of HDDS-935, which is an old and involved change. In that
change there is a TODO in
[{{ContainerStateMachine#persistContainerSet}}|https://github.com/apache/ozone/blob/7f22916889b7bf39cdb31e5943cae5768f368198/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java#L279]
that says there will be a race if block delete does not go through Ratis. When
block deletion was implemented after that change, it did not go through Ratis
so the race may happen. We need to revisit this area of the code.
> Startup warning about adding containers to missing container set
> ----------------------------------------------------------------
>
> Key: HDDS-8140
> URL: https://issues.apache.org/jira/browse/HDDS-8140
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ethan Rose
> Priority: Major
>
> This message was observed in the logs on startup for 250 containers. The same
> message occurred on 3/6 datanodes for the same set of containers on each node:
> {code}
> 2023-02-16 16:43:05,799 WARN
> org.apache.hadoop.ozone.container.common.impl.ContainerSet: Adding container
> 64079 to missing container set.
> {code}
> This is a byproduct of HDDS-935, which is an old and involved change. In that
> change there is a TODO in
> [{{ContainerStateMachine#persistContainerSet}}|https://github.com/apache/ozone/blob/7f22916889b7bf39cdb31e5943cae5768f368198/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java#L279]
> that says there will be a race if container delete does not go through
> Ratis. When container deletion was implemented after that change, it did not
> go through Ratis so the race may happen. We need to revisit this area of the
> code.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
