[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

[Jinglun (Jira)]

    [ 
https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959388#comment-16959388
 ] 

Jinglun commented on HDFS-14908:
--------------------------------

Hi [~hexiaoqiao], thanks your nice comments ! And sorry for my late response. 
Your demo is much simpler and it needs a minor change. When super user calls 
listOpenFiles, the parameter path might end with a slash, like 
'/user/hdfs_admin/demo/'. So before calling the code below we must normalize 
the path first. The normalize part would introduce some overhead.
(fullPathName.startsWith(path) && (fullPathName.equals(path) || 
fullPathName.charAt(path.length() - 1) == Path.SEPARATOR_CHAR))
When doing the random tests, I added a new method startsWithAndCharAt() with 
the normalize part to monitor the demo.

 
{code:java}
  public static boolean startsWithAndCharAt(String path, String parent) {
    if (path.length() > 1 && path.charAt(path.length() - 1) == '/') {
      path = path.substring(0, path.length() - 1);
    }
    return path.startsWith(parent) && (path.equals(parent)
        || path.charAt(parent.length() - 1) == '/');
  }

{code}
And here is the result:

*Case 1:* 

path starts with parent and neither path nor parent end with '/'

 
||Time||100,000,000||
|isParent|7,888ms|
|startsWithAndCharAt|8,850ms|
|startsWith|7,877ms|

*Case 2:*

path doesn't start with parent and neither path nor parent end with '/'
||Time||10,000,000,000||
|isParent|2,391ms|
|startsWithAndCharAt|2,362ms|
|startsWith|2,384ms|

*Case 4:*

path starts with parent and both path and parent end with '/'
||Time||100,000,000||
|isParent|7,882ms|
|startsWithAndCharAt|11,118ms|
|startsWith|7,803ms|

Test commands are:
{quote}java -Xmx512m Test 100000000 case1

java -Xmx512m Test 10000000000 case2

java -Xmx512m Test 100000000 case4
{quote}
Test file is TestV3.java

> LeaseManager should check parent-child relationship when filter open files.
> ---------------------------------------------------------------------------
>
>                 Key: HDFS-14908
>                 URL: https://issues.apache.org/jira/browse/HDFS-14908
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 3.1.0, 3.0.1
>            Reporter: Jinglun
>            Assignee: Jinglun
>            Priority: Minor
>         Attachments: HDFS-14908.001.patch, HDFS-14908.002.patch, 
> HDFS-14908.003.patch, Test.java, TestV2.java
>
>
> Now when doing listOpenFiles(), LeaseManager only checks whether the filter 
> path is the prefix of the open files. We should check whether the filter path 
> is the parent/ancestor of the open files.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org