Rushabh S Shah created HDFS-7928:
------------------------------------
Summary: Scanning blocks from disk during rolling upgrade startup
takes a lot of time if disks are busy
Key: HDFS-7928
URL: https://issues.apache.org/jira/browse/HDFS-7928
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Affects Versions: 2.6.0
Reporter: Rushabh S Shah
Assignee: Rushabh S Shah
We observed this issue in rolling upgrade to 2.6.x on one of our cluster.
One of the disks was very busy and it took long time to scan that disk compared
to other disks.
Seeing the sar (System Activity Reporter) data we saw that the particular disk
was very busy performing IO operations.
Requesting for an improvement during datanode rolling upgrade.
During shutdown, we can persist the whole volume map on the disk and let the
datanode read that file and create the volume map during startup after rolling
upgrade.
This will not require the datanode process to scan all the disk and read the
block.
This will significantly improve the datanode startup time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
