[
https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963662#comment-14963662
]
Brahma Reddy Battula commented on HDFS-8647:
--------------------------------------------
[~mingma]
{quote}BlockManager should swap addedNode and delNodeHint.,{quote}
yes you are correct..
{quote}After the fix, the only failed unit test left is
{{TestBalancer#testBalancerWithPinnedBlocks}}. Can you please investigate if it
is related to the patch?{quote}
.{code}cluster = new
MiniDFSCluster.Builder(conf).numDataNodes(capacities.length)
.hosts(new String[]{"localhost", "localhost"})
.racks(new String[]{"rack0",
"rack1"}).simulatedCapacities(capacities).build(){code}
2 DNs are started with "rack1". Ideally we should not create 2 DNs with the
same hostname.And Pinning depends on favoredNodes.DFSClient#create(..) only
uses host:port, if favoredNodes is created by {{new InetSocketAddress(ip,
port)}}
DFSClient will attempt a reverse lookup locally to get host:port, instead of
sending ip:port directly to NameNode.
.
MiniDFSCluster use fake hostname "host1.foo.com" to start DataNodes.DFSClient
doesn't use StaticMapping. So if DFSClient do reverse lookup, "127.0.0.1:8020"
becomes "localhost:8020".
{quote}{{public}} can be removed from {{public DatanodeStorageInfo
chooseReplicaToDelete}}.{quote}
it's overidden in test {{TestDnfencing#chooseReplicaToDelete}} .hence I make
this method visible for test.
Remaning changes are done and uploaded the patch....Hope I address all of your
comments..
> Abstract BlockManager's rack policy into BlockPlacementPolicy
> -------------------------------------------------------------
>
> Key: HDFS-8647
> URL: https://issues.apache.org/jira/browse/HDFS-8647
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Ming Ma
> Assignee: Brahma Reddy Battula
> Attachments: HDFS-8647-001.patch, HDFS-8647-002.patch,
> HDFS-8647-003.patch, HDFS-8647-004.patch, HDFS-8647-004.patch,
> HDFS-8647-005.patch, HDFS-8647-006.patch, HDFS-8647-007.patch
>
>
> Sometimes we want to have namenode use alternative block placement policy
> such as upgrade domains in HDFS-7541.
> BlockManager has built-in assumption about rack policy in functions such as
> useDelHint, blockHasEnoughRacks. That means when we have new block placement
> policy, we need to modify BlockManager to account for the new policy. Ideally
> BlockManager should ask BlockPlacementPolicy object instead. That will allow
> us to provide new BlockPlacementPolicy without changing BlockManager.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
