[
https://issues.apache.org/jira/browse/HDFS-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HDFS-7596:
--------------------------------
Attachment: HDFS-7596.02.patch
Thanks for the review [~lars_francke].
The .002 patch fixes failed test case {{TestBlockManager#testSafeModeIBR}} and
addresses part of Lars' feedback.
{code}
excessStorages = new HashMap<String, DatanodeStorageInfo>(storageMap);
{code}
This is deliberate to simplify back-porting the patch if needed (branch-2.6 is
required to support Java 6).
Differing chunks between the two patches for quick reference.
{code}
> @@ -416,6 +416,46 @@ public void updateHeartbeatState(StorageReport[]
> reports, long cacheCapacity,
33,36c33
< + final String storageId = report.getStorage().getStorageID();
< + if (excessStorages.get(storageId) != null) {
< + excessStorages.remove(storageId);
< + }
---
> + excessStorages.remove(report.getStorage().getStorageID());
And
> @@ -571,11 +571,13 @@ public void testSafeModeIBR() throws Exception {
> reset(node);
> bm.getDatanodeManager().registerDatanode(nodeReg);
> verify(node).updateRegInfo(nodeReg);
> - assertEquals(0, ds.getBlockReportCount()); // ready for report again
> // send block report, should be processed after restart
> reset(node);
> bm.processReport(node, new DatanodeStorage(ds.getStorageID()),
> - new BlockListAsLongs(null, null));
> + new BlockListAsLongs(null, null));
> + // Reinitialize as registration with empty storage list pruned
> + // node.storageMap.
> + ds = node.getStorageInfos()[0];
{code}
> NameNode should prune dead storages from storageMap
> ---------------------------------------------------
>
> Key: HDFS-7596
> URL: https://issues.apache.org/jira/browse/HDFS-7596
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.6.0
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments: HDFS-7596.01.patch, HDFS-7596.02.patch
>
>
> The NameNode must be able to prune storages that are no longer reported by
> the DataNode and that have no blocks associated. These stale storages can
> skew the balancer behavior.
> Detailed discussion on HDFS-7575.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
