[jira] [Updated] (HDFS-14718) HttpFS: Sort LISTSTATUS response by key names as WebHDFS does
[ https://issues.apache.org/jira/browse/HDFS-14718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDFS-14718: -- Description: *Example* See description of HDFS-14665 for an example of LISTSTATUS. *Analysis* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? - I looked into the Git history. It seems it's simply because WebHDFS uses TreeMap from the beginning; and HttpFS uses LinkedHashMap from the beginning. It is not only limited to LISTSTATUS, but ALL other request's JSON serialization. Now the real question: Could/Should we replace ALL LinkedHashMap into TreeMap in HttpFS serialization in FSOperations class? was: Example: see description of HDFS-14665. *Root cause* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? - I looked into the Git history. It seems it's simply because WebHDFS uses TreeMap from the beginning; and HttpFS uses LinkedHashMap from the beginning. Now the real question: Could/Should we replace ALL LinkedHashMap into TreeMap in HttpFS serialization in FSOperations class? > HttpFS: Sort LISTSTATUS response by key names as WebHDFS does > - > > Key: HDFS-14718 > URL: https://issues.apache.org/jira/browse/HDFS-14718 > Project: Hadoop HDFS > Issue Type: Improvement > Components: httpfs >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > *Example* > See description of HDFS-14665 for an example of LISTSTATUS. > *Analysis* > WebHDFS is [using a > TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] > to serialize HdfsFileStatus, while HttpFS [uses a > LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] > to serialize FileStatus. > *Questions* > Why the difference? Is this intentional? > - I looked into the Git history. It seems it's simply because WebHDFS uses > TreeMap from the beginning; and HttpFS uses LinkedHashMap from the beginning. > It is not only limited to LISTSTATUS, but ALL other request's JSON > serialization. > Now the real question: Could/Should we replace ALL LinkedHashMap into TreeMap > in HttpFS serialization in FSOperations class? -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14718) HttpFS: Sort LISTSTATUS response by key names as WebHDFS does
[ https://issues.apache.org/jira/browse/HDFS-14718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDFS-14718: -- Description: Example: see description of HDFS-14665. *Root cause* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? - I looked into the Git history. It seems it's simply because WebHDFS uses TreeMap from the beginning; and HttpFS uses LinkedHashMap from the beginning. Now the real question: Could/Should we replace ALL LinkedHashMap into TreeMap in HttpFS serialization in FSOperations class? was: Example: see description of HDFS-14665. *Root cause* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? I looked into the Git history. It seems it's simply because WebHDFS uses TreeMap from the beginning; and HttpFS uses LinkedHashMap from the beginning. > HttpFS: Sort LISTSTATUS response by key names as WebHDFS does > - > > Key: HDFS-14718 > URL: https://issues.apache.org/jira/browse/HDFS-14718 > Project: Hadoop HDFS > Issue Type: Improvement > Components: httpfs >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > Example: see description of HDFS-14665. > *Root cause* > WebHDFS is [using a > TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] > to serialize HdfsFileStatus, while HttpFS [uses a > LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] > to serialize FileStatus. > *Questions* > Why the difference? Is this intentional? > - I looked into the Git history. It seems it's simply because WebHDFS uses > TreeMap from the beginning; and HttpFS uses LinkedHashMap from the beginning. > Now the real question: Could/Should we replace ALL LinkedHashMap into TreeMap > in HttpFS serialization in FSOperations class? -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14718) HttpFS: Sort LISTSTATUS response by key names as WebHDFS does
[ https://issues.apache.org/jira/browse/HDFS-14718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDFS-14718: -- Description: Example: see description of HDFS-14665. *Root cause* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? I looked into the Git history. It seems it's simply because WebHDFS uses TreeMap from the beginning; and HttpFS uses LinkedHashMap from the beginning. was: Example: see description of HDFS-14665. *Root cause* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? e.g. for some performance concerns on HttpFS? > HttpFS: Sort LISTSTATUS response by key names as WebHDFS does > - > > Key: HDFS-14718 > URL: https://issues.apache.org/jira/browse/HDFS-14718 > Project: Hadoop HDFS > Issue Type: Improvement > Components: httpfs >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > Example: see description of HDFS-14665. > *Root cause* > WebHDFS is [using a > TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] > to serialize HdfsFileStatus, while HttpFS [uses a > LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] > to serialize FileStatus. > *Questions* > Why the difference? Is this intentional? > I looked into the Git history. It seems it's simply because WebHDFS uses > TreeMap from the beginning; and HttpFS uses LinkedHashMap from the beginning. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14718) HttpFS: Sort LISTSTATUS response by key names as WebHDFS does
[ https://issues.apache.org/jira/browse/HDFS-14718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDFS-14718: -- Description: Example: see description of HDFS-14665. *Root cause* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? e.g. for some performance concerns on HttpFS? was: Example: see description of HDFS-14665. *Root cause* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? e.g. for some performance concern on HttpFS? > HttpFS: Sort LISTSTATUS response by key names as WebHDFS does > - > > Key: HDFS-14718 > URL: https://issues.apache.org/jira/browse/HDFS-14718 > Project: Hadoop HDFS > Issue Type: Improvement > Components: httpfs >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > Example: see description of HDFS-14665. > *Root cause* > WebHDFS is [using a > TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] > to serialize HdfsFileStatus, while HttpFS [uses a > LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] > to serialize FileStatus. > *Questions* > Why the difference? Is this intentional? e.g. for some performance concerns > on HttpFS? -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14718) HttpFS: Sort LISTSTATUS response by key names as WebHDFS does
[ https://issues.apache.org/jira/browse/HDFS-14718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDFS-14718: -- Description: Example: see description of HDFS-14665. *Root cause* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? e.g. for some performance concern on HttpFS? was: Example: see description of HDFS-14665. Root cause: WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. Questions: Why the difference? Is this intentional? e.g. for some performance concern on HttpFS? > HttpFS: Sort LISTSTATUS response by key names as WebHDFS does > - > > Key: HDFS-14718 > URL: https://issues.apache.org/jira/browse/HDFS-14718 > Project: Hadoop HDFS > Issue Type: Improvement > Components: httpfs >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > Example: see description of HDFS-14665. > *Root cause* > WebHDFS is [using a > TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] > to serialize HdfsFileStatus, while HttpFS [uses a > LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] > to serialize FileStatus. > *Questions* > Why the difference? Is this intentional? e.g. for some performance concern on > HttpFS? -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14718) HttpFS: Sort LISTSTATUS response by key names as WebHDFS does
[ https://issues.apache.org/jira/browse/HDFS-14718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDFS-14718: -- Description: Example: see description of HDFS-14665. *Root cause* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? e.g. for some performance concern on HttpFS? was: Example: see description of HDFS-14665. *Root cause* WebHDFS is [using a TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] to serialize HdfsFileStatus, while HttpFS [uses a LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] to serialize FileStatus. *Questions* Why the difference? Is this intentional? e.g. for some performance concern on HttpFS? > HttpFS: Sort LISTSTATUS response by key names as WebHDFS does > - > > Key: HDFS-14718 > URL: https://issues.apache.org/jira/browse/HDFS-14718 > Project: Hadoop HDFS > Issue Type: Improvement > Components: httpfs >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > Example: see description of HDFS-14665. > *Root cause* > WebHDFS is [using a > TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] > to serialize HdfsFileStatus, while HttpFS [uses a > LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] > to serialize FileStatus. > *Questions* > Why the difference? Is this intentional? e.g. for some performance concern on > HttpFS? -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14718) HttpFS: Sort LISTSTATUS response by key names as WebHDFS does
[ https://issues.apache.org/jira/browse/HDFS-14718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDFS-14718: -- Summary: HttpFS: Sort LISTSTATUS response by key names as WebHDFS does (was: HttpFS: Sort LISTSTATUS response by key names) > HttpFS: Sort LISTSTATUS response by key names as WebHDFS does > - > > Key: HDFS-14718 > URL: https://issues.apache.org/jira/browse/HDFS-14718 > Project: Hadoop HDFS > Issue Type: Improvement > Components: httpfs >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > Example: see description of HDFS-14665. > Root cause: > WebHDFS is [using a > TreeMap|https://github.com/apache/hadoop/blob/99bf1dc9eb18f9b4d0338986d1b8fd2232f1232f/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java#L120] > to serialize HdfsFileStatus, while HttpFS [uses a > LinkedHashMap|https://github.com/apache/hadoop/blob/6fcc5639ae32efa5a5d55a6b6cf23af06fc610c3/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java#L107] > to serialize FileStatus. > Questions: > Why the difference? Is this intentional? e.g. for some performance concern on > HttpFS? -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
