[
https://issues.apache.org/jira/browse/HDFS-7645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14360657#comment-14360657
]
Kihwal Lee commented on HDFS-7645:
----------------------------------
I think you want {{clearTrash()}} to be called when a rolling upgrade is
finalized. That is if {{inProgress}} is not true, clear all trash. For regular
or downgrade start-ups, if the rolling upgrade is already aborted/finallized,
the trash will get cleared once the datanode registers with the namenode. So we
don't have to anything special on start-up.
As for breaking test cases, when there is a layout change during rolling
upgrade, the old blocks are moved to {{previous}}. The current code do this by
restoring trash and going through {{doUpgrade()}}. In order to keep this
behavior, {{doTransition()}} needs to check for layout change (but ignore ctime
change) and does restore, before the check for calling {{doUpgrade()}}. I
tried the following and it seems to work.
{code:java}
if (this.layoutVersion > HdfsConstants.DATANODE_LAYOUT_VERSION) {
int restored = restoreBlockFilesFromTrash(getTrashRootDir(sd));
LOG.info("Restored " + restored + " block files from trash " +
"before the layout upgrade. These blocks will be moved to " +
"the previous directory during the upgrade");
}
{code}
> Rolling upgrade is restoring blocks from trash multiple times
> -------------------------------------------------------------
>
> Key: HDFS-7645
> URL: https://issues.apache.org/jira/browse/HDFS-7645
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.6.0
> Reporter: Nathan Roberts
> Assignee: Keisuke Ogiwara
> Attachments: HDFS-7645.01.patch, HDFS-7645.02.patch,
> HDFS-7645.03.patch
>
>
> When performing an HDFS rolling upgrade, the trash directory is getting
> restored twice when under normal circumstances it shouldn't need to be
> restored at all. iiuc, the only time these blocks should be restored is if we
> need to rollback a rolling upgrade.
> On a busy cluster, this can cause significant and unnecessary block churn
> both on the datanodes, and more importantly in the namenode.
> The two times this happens are:
> 1) restart of DN onto new software
> {code}
> private void doTransition(DataNode datanode, StorageDirectory sd,
> NamespaceInfo nsInfo, StartupOption startOpt) throws IOException {
> if (startOpt == StartupOption.ROLLBACK && sd.getPreviousDir().exists()) {
> Preconditions.checkState(!getTrashRootDir(sd).exists(),
> sd.getPreviousDir() + " and " + getTrashRootDir(sd) + " should not
> " +
> " both be present.");
> doRollback(sd, nsInfo); // rollback if applicable
> } else {
> // Restore all the files in the trash. The restored files are retained
> // during rolling upgrade rollback. They are deleted during rolling
> // upgrade downgrade.
> int restored = restoreBlockFilesFromTrash(getTrashRootDir(sd));
> LOG.info("Restored " + restored + " block files from trash.");
> }
> {code}
> 2) When heartbeat response no longer indicates a rollingupgrade is in progress
> {code}
> /**
> * Signal the current rolling upgrade status as indicated by the NN.
> * @param inProgress true if a rolling upgrade is in progress
> */
> void signalRollingUpgrade(boolean inProgress) throws IOException {
> String bpid = getBlockPoolId();
> if (inProgress) {
> dn.getFSDataset().enableTrash(bpid);
> dn.getFSDataset().setRollingUpgradeMarker(bpid);
> } else {
> dn.getFSDataset().restoreTrash(bpid);
> dn.getFSDataset().clearRollingUpgradeMarker(bpid);
> }
> }
> {code}
> HDFS-6800 and HDFS-6981 were modifying this behavior making it not completely
> clear whether this is somehow intentional.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
