[
https://issues.apache.org/jira/browse/HDFS-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDFS-15303:
-----------------------------------
Issue Type: New Feature (was: Improvement)
> Provide a tool that can validate/fix the block file placement in DataNode
> data directories
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-15303
> URL: https://issues.apache.org/jira/browse/HDFS-15303
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Istvan Fajth
> Assignee: Istvan Fajth
> Priority: Minor
>
> We recently run into an issue where during an upgrade from Hadoop2 to
> Hadoop3, the filesystem under DataNode data directories was left in an
> intermediate state, and part of the directories were in Hadoop2 format other
> parts in Hadoop3 format.
> At first we had to rollback the upgrade, and after the rollback we started to
> see FileNotFoundExceptions for particular block files.
> The exception was logged on the DataNodes and failed the jobs as well
> sometimes. HDFS NameNode did not show any missing blocks, and we found the
> block files and meta files also in the DataNode's data directories, but at a
> different location.
> This was the point when we realized that something went wrong during the
> rollback, and some of the data directories had blocks placed according to
> Hadoop3 rules, while other were placed according to Hadoop2 rules. We suspect
> a possible premature DataNode shutdown or an unknown failure during the
> rollback, but at the point when we realized what is the issue and could check
> into things, we already ran out of the logs that would have been able to show
> us the cause.
> This JIRA is to suggest two new commands that can help administrators in this
> situation, to validate the data directory and ensure that blocks are placed
> correctly according to the rules, and to fix the data directory layout if
> needed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
