[
https://issues.apache.org/jira/browse/HDFS-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310587#comment-14310587
]
Leitao Guo commented on HDFS-7692:
----------------------------------
[~eddyxu], thanks for your review and comments!
When running tests after patching, I got "NullPointerException" at this line,
so I add the check of "null != datanode.getConf()" here.
{code} Executors.newFixedThreadPool(null != datanode.getConf() ? datanode
493 .getConf().getInt(
494 DFSConfigKeys.DFS_DATANODE_DIRECTORYSCAN_THREADS_KEY,
495
DFSConfigKeys.DFS_DATANODE_DIRECTORYSCAN_THREADS_DEFAULT)
496 : dataDirs.size()); {code}
I will update the patch according to your comments, thanks!
> DataStorage#addStorageLocations(...) should support MultiThread to speedup
> the upgrade of block pool at multi storage directories.
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-7692
> URL: https://issues.apache.org/jira/browse/HDFS-7692
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.5.2
> Reporter: Leitao Guo
> Assignee: Leitao Guo
> Attachments: HDFS-7692.01.patch
>
>
> {code:title=DataStorage#addStorageLocations(...)|borderStyle=solid}
> for (StorageLocation dataDir : dataDirs) {
> File root = dataDir.getFile();
> ... ...
> bpStorage.recoverTransitionRead(datanode, nsInfo, bpDataDirs,
> startOpt);
> addBlockPoolStorage(bpid, bpStorage);
> ... ...
> successVolumes.add(dataDir);
> }
> {code}
> In the above code the storage directories will be analyzed one by one, which
> is really time consuming when upgrading HDFS with datanodes have dozens of
> large volumes. MultiThread dataDirs analyzing should be supported here to
> speedup upgrade.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
