[
https://issues.apache.org/jira/browse/HDFS-9230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14961561#comment-14961561
]
Andrew Wang commented on HDFS-9230:
-----------------------------------
For hardlink upgrades, you could check the link count to see if a file in
previous is still referenced in current. This is similar in cost to du.
> Report space overhead of unfinalized upgrade/rollingUpgrade
> -----------------------------------------------------------
>
> Key: HDFS-9230
> URL: https://issues.apache.org/jira/browse/HDFS-9230
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: HDFS
> Reporter: Xiaoyu Yao
>
> DataNodes do not delete block files during upgrades to allow rollback. This
> is often confusing to administrators since they sometimes delete files before
> finalize upgrade but don't see the DFS used space reduce.
> Ideally, HDFS should report the un-finalized upgrade overhead along with its
> message on NN UI "Upgrade in progress. Not yet finalized." Or, this can be
> improve with better NN UI message and document that space won't be reclaimed
> for deletion until upgrade is finalized.
> For non-rolling upgrade, it is not easy to track this due to hard link. Say
> NN initialized upgrade at T1, the block files on DNs that exist before T1 are
> still under 'current' directory but is just a hard link to 'previous'
> directory. When those files are deleted after T1 due to deletion, the block
> file usage on DN won't get deleted until upgrade is finalized.
> So we need to book keeping files created before T1 but deleted after T1 as
> the un-finalized upgrade overhead here.
> For rolling upgrade, it is relative easy to track space overhead as we are
> not using hard link.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
