[
https://issues.apache.org/jira/browse/HDDS-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Rose updated HDDS-6235:
-----------------------------
Summary: Empty KeyValueContainers are replicated without chunks directory
(was: Empty KeyValueContainers cannot be replicated)
> Empty KeyValueContainers are replicated without chunks directory
> ----------------------------------------------------------------
>
> Key: HDDS-6235
> URL: https://issues.apache.org/jira/browse/HDDS-6235
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode
> Affects Versions: 1.0.0, 1.1.0, 1.2.0
> Reporter: Ethan Rose
> Assignee: Ethan Rose
> Priority: Major
>
> An empty KeyValueContainer will have an empty chunks directory.
> TarContainerPacker#pack recurses into directories adding files into
> containers, but if the chunks directory is empty, it will not be included in
> the tar. The receiver will unpack the tar successfully, but the resulting
> container will not have a chunks directory. After this, the container will
> not be able to replicated further, as the tar packing step requires all
> container pieces to be present on disk. This issue is more likely to occur
> due to HDDS-5359, which causes many empty containers to be tracked by SCM
> indefinitely.
> Since the issue only affects empty containers, there does not appear to be
> any data loss risk, even though the container scanner may detect it as
> "corruption". The issue may manifest as a container continuously attempting
> to be replicated and failing, or the container being marked unhealthy by the
> background container scanner (if it is enabled).
> This Jira will fix the issue with the tar packer, and also add a repair step
> on datanode startup to create the chunks directory for containers that do not
> have one. This step should be a quick addition to datanode startup that
> already iterates all the containers, and should not impact startup time.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
