[
https://issues.apache.org/jira/browse/HDFS-14268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767869#comment-16767869
]
Ayush Saxena commented on HDFS-14268:
-------------------------------------
Thanx [~elgoiri] for the patch. The overall patch seems good.
IMO Regarding the test, If it is getting affected by HDFS-14226. we can push
this after that.
Gave a look at the failure.
{noformat}
expected:<...dundancyBlockGroups=[0], CorruptBlockGroups...> but
was:<...dundancyBlockGroups=[2], CorruptBlockGroups...>
{noformat}
The failure seems to say that number of LowRedundancy blocks reported by the
namenode is 0 but by the router is 2. Hence the failure. The thing that catches
my eye over here is: In router for getECBlockStats. We merge the result from
all the namespaces in {{ErasureCoding.java}}.
{code:java}
Map<FederationNamespaceInfo, ECBlockGroupStats> allStats =
rpcClient.invokeConcurrent(
nss, method, true, false, ECBlockGroupStats.class);
// Merge the stats from all the namespaces
{code}
So the result (Low Redundancy 2) is the aggregate from both nameservice. If
either of the two nameservice has 2 as Low redundancy Block it shall give us 2.
But it the test at {{TestRouterRPCMultipleDestination}} we are comparing this
router merged result with a single namervice which we chose randomly.
{code:java}
String ns = cluster.getRandomNameservice();
this.setNs(ns);
this.setNamenode(cluster.getNamenode(ns, null));
{code}
Might be the nameservice with which we compare doesn't had the block. The other
one had???
Since in RouterRpc only one destination is there. This works. But in Multiple
case / is mounted to two.
> RBF: Fix the location of the DNs in getDatanodeReport()
> -------------------------------------------------------
>
> Key: HDFS-14268
> URL: https://issues.apache.org/jira/browse/HDFS-14268
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Íñigo Goiri
> Assignee: Íñigo Goiri
> Priority: Major
> Attachments: HDFS-14268-HDFS-13891.000.patch,
> HDFS-14268-HDFS-13891.001.patch, HDFS-14268-HDFS-13891.002.patch
>
>
> When getting all the DNs in the federation, the Router queries each of the
> subclusters and aggregates them assigning the subcluster id to the location.
> This query uses a {{HashSet}} which provides a "random" order for the results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
