[ https://issues.apache.org/jira/browse/HDFS-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wei-Chiu Chuang updated HDFS-15303: ----------------------------------- Issue Type: New Feature (was: Improvement) > Provide a tool that can validate/fix the block file placement in DataNode > data directories > ------------------------------------------------------------------------------------------ > > Key: HDFS-15303 > URL: https://issues.apache.org/jira/browse/HDFS-15303 > Project: Hadoop HDFS > Issue Type: New Feature > Reporter: Istvan Fajth > Assignee: Istvan Fajth > Priority: Minor > > We recently run into an issue where during an upgrade from Hadoop2 to > Hadoop3, the filesystem under DataNode data directories was left in an > intermediate state, and part of the directories were in Hadoop2 format other > parts in Hadoop3 format. > At first we had to rollback the upgrade, and after the rollback we started to > see FileNotFoundExceptions for particular block files. > The exception was logged on the DataNodes and failed the jobs as well > sometimes. HDFS NameNode did not show any missing blocks, and we found the > block files and meta files also in the DataNode's data directories, but at a > different location. > This was the point when we realized that something went wrong during the > rollback, and some of the data directories had blocks placed according to > Hadoop3 rules, while other were placed according to Hadoop2 rules. We suspect > a possible premature DataNode shutdown or an unknown failure during the > rollback, but at the point when we realized what is the issue and could check > into things, we already ran out of the logs that would have been able to show > us the cause. > This JIRA is to suggest two new commands that can help administrators in this > situation, to validate the data directory and ensure that blocks are placed > correctly according to the rules, and to fix the data directory layout if > needed. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org