Janus Chow created HDFS-15669:
---------------------------------
Summary: [HDFS] [RBF] Incorrect GetQuota caused by different
implementation of HashSet
Key: HDFS-15669
URL: https://issues.apache.org/jira/browse/HDFS-15669
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Janus Chow
In the method of Quota#getQuotaUsage, the result could be different on
different versions of Java.
The trace route is as follows:
{code:java}
Quota#getQuotaUsage
- Quota#getValidQuotaLocations
- Quota#getQuotaRemoteLocations
- RouterQuotaManager#getPaths{code}
In RouterQuotaManager#getPaths, the path is stored in a _HashSet_. And in
Quota#getValidQuotaLocations, the values in HashSet would be used to check if
the paths are parent-child relation. So paths in HashSet could affect the
result of getValidQuotaLocations as follows:
{code:java}
// Situation.1 Locations in HashSet
[ns0->/testdir7, ns0->/testdir7/subdir, ns1->/testdir8]
// Situation.1 getQuota results
{ns0->/testdir7= 10 6 100 100 , ns1->/testdir8= 10 8 100 100}
// Situation.2 Locations in HashSet
[ns0->/testdir7/subdir, ns1->/testdir8, ns0->/testdir7]
// Situation.2 getQuota results
{ns0->/testdir7= 10 8 100 100 , ns0->/testdir7/subdir= 10 6 100 100 ,
ns1->/testdir8= 10 8 100 100 }{code}
Situation.1 and Situation.2 happen when the underlying implementation of
HashSet is different, that is Situation.2 happens using Java-7, and Situation.2
happens after Java-8.
This problem can be solved when we do sorting on the results of
Quota#_getQuotaRemoteLocations_.
{code:java}
/**
* Get all quota remote locations across subclusters under given
* federation path.
* @param path Federation path.
* @return List of quota remote locations.
* @throws IOException
*/
private List<RemoteLocation> getQuotaRemoteLocations(String path)
throws IOException {
List<RemoteLocation> locations = new LinkedList<>();
RouterQuotaManager manager = this.router.getQuotaManager();
if (manager != null) {
Set<String> childrenPaths = manager.getPaths(path);
for (String childPath : childrenPaths) {
locations.addAll(rpcServer.getLocationsForPath(childPath, true, false));
}
}
++ Collections.sort(locations);
return locations;
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
