[
https://issues.apache.org/jira/browse/HDFS-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yanghuafeng updated HDFS-13845:
-------------------------------
Description:
When we use the default MountTableResolver to resolve the path, we cannot get
the destination paths for the default DestinationOrder.HASH.
{code:java}
// Some comments here
private static PathLocation buildLocation(
......
List<RemoteLocation> locations = new LinkedList<>();
for (RemoteLocation oneDst : entry.getDestinations()) {
String nsId = oneDst.getNameserviceId();
String dest = oneDst.getDest();
String newPath = dest;
if (!newPath.endsWith(Path.SEPARATOR) && !remainingPath.isEmpty()) {
newPath += Path.SEPARATOR;
}
newPath += remainingPath;
RemoteLocation remoteLocation = new RemoteLocation(nsId, newPath, path);
locations.add(remoteLocation);
}
DestinationOrder order = entry.getDestOrder();
return new PathLocation(srcPath, locations, order);
}
{code}
The default order will be hash, but the HashFirstResolver will not be invoked
to order the location.
It is ambiguous for the MountTableResolver that we will see the HASH order in
the web ui for multi-destinations path but we cannot get the result.
In my opinion, the MountTableResolver will be a simple resolver to implement 1
to 1 not including the 1 to n destinations. So we should check the
buildLocation. If the entry has multi destinations, we should reject it.
was:
When we use the default MountTableResolver to resolve the path, we cannot get
the destination paths for the default DestinationOrder.HASH.
{code:java}
// Some comments here
private static PathLocation buildLocation(
......
List<RemoteLocation> locations = new LinkedList<>();
for (RemoteLocation oneDst : entry.getDestinations()) {
String nsId = oneDst.getNameserviceId();
String dest = oneDst.getDest();
String newPath = dest;
if (!newPath.endsWith(Path.SEPARATOR) && !remainingPath.isEmpty()) {
newPath += Path.SEPARATOR;
}
newPath += remainingPath;
RemoteLocation remoteLocation = new RemoteLocation(nsId, newPath, path);
locations.add(remoteLocation);
}
DestinationOrder order = entry.getDestOrder();
return new PathLocation(srcPath, locations, order);
}
{code}
The default order will be hash, but the HashFirstResolver will not be invoked
to order the location.
It is ambiguous for the MountTableResolver that we will see the HASH order in
the web ui for multi-destinations path but we cannot get the result.
In my opinion, the MountTableResolver will be a simple resolver to implement 1
to 1 not including the 1 to n destinations. So we should check the
addMountTable and updateMountTable when we use MountTableResolver. If the
entry has multi destinations, we should reject it.
> RBF: The default MountTableResolver cannot get multi-destination path for the
> DestinationOrder.
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-13845
> URL: https://issues.apache.org/jira/browse/HDFS-13845
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: federation, hdfs
> Affects Versions: 3.0.0, 3.1.0, 2.9.1
> Reporter: yanghuafeng
> Assignee: yanghuafeng
> Priority: Major
>
> When we use the default MountTableResolver to resolve the path, we cannot get
> the destination paths for the default DestinationOrder.HASH.
> {code:java}
> // Some comments here
> private static PathLocation buildLocation(
> ......
> List<RemoteLocation> locations = new LinkedList<>();
> for (RemoteLocation oneDst : entry.getDestinations()) {
> String nsId = oneDst.getNameserviceId();
> String dest = oneDst.getDest();
> String newPath = dest;
> if (!newPath.endsWith(Path.SEPARATOR) && !remainingPath.isEmpty()) {
> newPath += Path.SEPARATOR;
> }
> newPath += remainingPath;
> RemoteLocation remoteLocation = new RemoteLocation(nsId, newPath, path);
> locations.add(remoteLocation);
> }
> DestinationOrder order = entry.getDestOrder();
> return new PathLocation(srcPath, locations, order);
> }
> {code}
> The default order will be hash, but the HashFirstResolver will not be invoked
> to order the location.
> It is ambiguous for the MountTableResolver that we will see the HASH order in
> the web ui for multi-destinations path but we cannot get the result.
> In my opinion, the MountTableResolver will be a simple resolver to implement
> 1 to 1 not including the 1 to n destinations. So we should check the
> buildLocation. If the entry has multi destinations, we should reject it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
