[
https://issues.apache.org/jira/browse/HDFS-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinwei Qin updated HDFS-8710:
------------------------------
Attachment: HDFS-8710.001.patch
Attached the patch to review.
> Always read DU value from the cached "dfsUsed" file on datanode startup
> -----------------------------------------------------------------------
>
> Key: HDFS-8710
> URL: https://issues.apache.org/jira/browse/HDFS-8710
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Xinwei Qin
> Assignee: Xinwei Qin
> Attachments: HDFS-8710.001.patch
>
>
> Currently, DataNode will cache DU value in "dfsUsed" file termly. When
> DataNode starts or restarts, it will read in the cached DU value from
> "dfsUsed" file if the value is less than 600 seconds old, otherwise, it will
> run DU command, which is a very time-consuming operation(may up to dozens of
> minutes) when DataNode has huge number of blocks.
> Since slight imprecision of dfsUsed is not critical, and the DU value will be
> updated every 600 seconds (the default DU interval) after DataNode started,
> we can always read DU value from the cached file (Regardless of whether this
> value is less than 600 seconds old or not) and skip DU operation on DataNode
> startup to significantly shorten the startup time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
