[
https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959388#comment-16959388
]
Jinglun edited comment on HDFS-14908 at 10/25/19 3:29 AM:
----------------------------------------------------------
Hi [~hexiaoqiao], thanks your nice comments ! And sorry for my late response.
Your demo is much simpler and it needs a minor change. When super user calls
listOpenFiles, the parameter path might end with a slash, like
'/user/hdfs_admin/demo/'. So before calling the code below we must normalize
the path first. The normalize part would introduce some overhead.
{code:java}
(fullPathName.startsWith(path) && (fullPathName.equals(path) ||
fullPathName.charAt(path.length() - 1) == Path.SEPARATOR_CHAR)){code}
When doing the random tests, I added a new method startsWithAndCharAt() with
the normalize part to monitor the demo.
{code:java}
public static boolean startsWithAndCharAt(String path, String parent) {
if (path.length() > 1 && path.charAt(path.length() - 1) == '/') {
path = path.substring(0, path.length() - 1);
}
return path.startsWith(parent) && (path.equals(parent)
|| path.charAt(parent.length() - 1) == '/');
}
{code}
And here is the result:
*Case 1:*
path starts with parent and neither path nor parent end with '/'
||Time||100,000,000||
|isParent|7,888ms|
|startsWithAndCharAt|8,850ms|
|startsWith|7,877ms|
*Case 2:*
path doesn't start with parent and neither path nor parent end with '/'
||Time||10,000,000,000||
|isParent|2,391ms|
|startsWithAndCharAt|2,362ms|
|startsWith|2,384ms|
*Case 4:*
path starts with parent and both path and parent end with '/'
||Time||100,000,000||
|isParent|7,882ms|
|startsWithAndCharAt|11,118ms|
|startsWith|7,803ms|
Test commands are:
{quote}java -Xmx512m Test 100000000 case1
java -Xmx512m Test 10000000000 case2
java -Xmx512m Test 100000000 case4
{quote}
Test file is TestV3.java
was (Author: lijinglun):
Hi [~hexiaoqiao], thanks your nice comments ! And sorry for my late response.
Your demo is much simpler and it needs a minor change. When super user calls
listOpenFiles, the parameter path might end with a slash, like
'/user/hdfs_admin/demo/'. So before calling the code below we must normalize
the path first. The normalize part would introduce some overhead.
(fullPathName.startsWith(path) && (fullPathName.equals(path) ||
fullPathName.charAt(path.length() - 1) == Path.SEPARATOR_CHAR))
When doing the random tests, I added a new method startsWithAndCharAt() with
the normalize part to monitor the demo.
{code:java}
public static boolean startsWithAndCharAt(String path, String parent) {
if (path.length() > 1 && path.charAt(path.length() - 1) == '/') {
path = path.substring(0, path.length() - 1);
}
return path.startsWith(parent) && (path.equals(parent)
|| path.charAt(parent.length() - 1) == '/');
}
{code}
And here is the result:
*Case 1:*
path starts with parent and neither path nor parent end with '/'
||Time||100,000,000||
|isParent|7,888ms|
|startsWithAndCharAt|8,850ms|
|startsWith|7,877ms|
*Case 2:*
path doesn't start with parent and neither path nor parent end with '/'
||Time||10,000,000,000||
|isParent|2,391ms|
|startsWithAndCharAt|2,362ms|
|startsWith|2,384ms|
*Case 4:*
path starts with parent and both path and parent end with '/'
||Time||100,000,000||
|isParent|7,882ms|
|startsWithAndCharAt|11,118ms|
|startsWith|7,803ms|
Test commands are:
{quote}java -Xmx512m Test 100000000 case1
java -Xmx512m Test 10000000000 case2
java -Xmx512m Test 100000000 case4
{quote}
Test file is TestV3.java
> LeaseManager should check parent-child relationship when filter open files.
> ---------------------------------------------------------------------------
>
> Key: HDFS-14908
> URL: https://issues.apache.org/jira/browse/HDFS-14908
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.1.0, 3.0.1
> Reporter: Jinglun
> Assignee: Jinglun
> Priority: Minor
> Attachments: HDFS-14908.001.patch, HDFS-14908.002.patch,
> HDFS-14908.003.patch, Test.java, TestV2.java, TestV3.java
>
>
> Now when doing listOpenFiles(), LeaseManager only checks whether the filter
> path is the prefix of the open files. We should check whether the filter path
> is the parent/ancestor of the open files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
