[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15124670#comment-15124670
]
Akira AJISAKA commented on HDFS-6565:
-
Hi [~sztanko], I don't think replacing jetty with jackson itself is the cause.
I'm thinking the cause is that the patch does not re-use {{ObjectMapper}}
instance.
Filed HDFS-9724.
> Use jackson instead jetty json in hdfs-client
> -
>
> Key: HDFS-6565
> URL: https://issues.apache.org/jira/browse/HDFS-6565
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Haohui Mai
>Assignee: Akira AJISAKA
> Fix For: 2.7.0
>
> Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch,
> HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch
>
>
> hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121913#comment-15121913 ] Demeter Sztanko commented on HDFS-6565: --- We are experiencing performance degradation of webhdfs, when listing directories with large amount of files (>1000). Our profiling is indicating that most of resources are spent within Jackson library. Do you think replacing jetty with jackson could be the cause of this? > Use jackson instead jetty json in hdfs-client > - > > Key: HDFS-6565 > URL: https://issues.apache.org/jira/browse/HDFS-6565 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Haohui Mai >Assignee: Akira AJISAKA > Fix For: 2.7.0 > > Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, > HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch > > > hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347038#comment-14347038 ] Hudson commented on HDFS-6565: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2072 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2072/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Fix For: 2.7.0 Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346711#comment-14346711 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #122 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/122/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Fix For: 2.7.0 Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346721#comment-14346721 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-Yarn-trunk #856 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/856/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Fix For: 2.7.0 Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346926#comment-14346926 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2054 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2054/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Fix For: 2.7.0 Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346935#comment-14346935 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #113 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/113/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Fix For: 2.7.0 Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347001#comment-14347001 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #122 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/122/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Fix For: 2.7.0 Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344770#comment-14344770
]
Hadoop QA commented on HDFS-6565:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12702086/HDFS-6565-003.patch
against trunk revision d1c6acc.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-hdfs-project/hadoop-hdfs
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9708//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9708//console
This message is automatically generated.
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345761#comment-14345761
]
Haohui Mai commented on HDFS-6565:
--
Can you please update the patch to remove the {{toTokenList()}} methods? Other
than that the v3 patch looks pretty good to me.
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345758#comment-14345758 ] Haohui Mai commented on HDFS-6565: -- bq. Okay. I suppose that should be documented. Would you mind adding @InterfaceAudience.Private to the class? There are consensus on assuming unannotated class are Private-Unstable. Please see http://marc.info/?t=14060662831r=1w=2 Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345942#comment-14345942
]
Haohui Mai commented on HDFS-6565:
--
As a result the following methods are no longer used:
{code}
public static String toJsonString(final Token? extends TokenIdentifier[]
tokens
private static ListToken? toTokenList(final List? objects)
{code}
Can you please remove these methods as well?
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch,
HDFS-6565-004.patch, HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346071#comment-14346071
]
Hadoop QA commented on HDFS-6565:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12702263/HDFS-6565-004.patch
against trunk revision e17e5ba.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9713//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9713//console
This message is automatically generated.
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch,
HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346272#comment-14346272 ] Akira AJISAKA commented on HDFS-6565: - Thanks Haohui! Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Fix For: 2.7.0 Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346216#comment-14346216 ] Haohui Mai commented on HDFS-6565: -- +1. I'm committing this. Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346229#comment-14346229 ] Hudson commented on HDFS-6565: -- FAILURE: Integrated in Hadoop-trunk-Commit #7249 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7249/]) HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. (wheat9: rev e2262d3d18c6d5c2aa20f96920104dc07271b869) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Fix For: 2.7.0 Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346267#comment-14346267
]
Hadoop QA commented on HDFS-6565:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12702293/HDFS-6565-005.patch
against trunk revision e17e5ba.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode
org.apache.hadoop.hdfs.server.namenode.TestFileTruncate
The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9715//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9715//console
This message is automatically generated.
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Fix For: 2.7.0
Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch,
HDFS-6565-004.patch, HDFS-6565-005.patch, HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344400#comment-14344400 ] Akira AJISAKA commented on HDFS-6565: - The test failure looks unrelated to the patch. I ran the test locally and it passed. Use jackson instead jetty json in hdfs-client - Key: HDFS-6565 URL: https://issues.apache.org/jira/browse/HDFS-6565 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Akira AJISAKA Attachments: HDFS-6565-002.patch, HDFS-6565.patch hdfs-client should use Jackson instead of jetty to parse JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344353#comment-14344353
]
Hadoop QA commented on HDFS-6565:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12702006/HDFS-6565-002.patch
against trunk revision 5d0bae5.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9701//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9701//console
This message is automatically generated.
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565-002.patch, HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344675#comment-14344675
]
Haohui Mai commented on HDFS-6565:
--
bq. It's a public API. I'm thinking deleting it will break compatibility.
The class is not intended to be used other classes other than
{{WebHdfsFileSystem}} and {{NameNodeWebHdfsMethods}}. I think that it is fine
to remove it.
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344585#comment-14344585
]
Akira AJISAKA commented on HDFS-6565:
-
Thanks [~wheat9] for the review.
bq. Looks like the function can be deleted.
It's a public API. I'm thinking deleting it will break compatibility.
{quote}
Simply make it like:
{code}
} catch (IOException ignored) {
}
{code}
{quote}
I'll refrect that in the next patch.
bq. There are many changes like this – it seems that they are unnecessary?
It is necessary. See my previous
[comment|https://issues.apache.org/jira/browse/HDFS-6565?focusedCommentId=14041409page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14041409].
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565-002.patch, HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344679#comment-14344679
]
Akira AJISAKA commented on HDFS-6565:
-
bq. The class is not intended to be used other classes other than
WebHdfsFileSystem and NameNodeWebHdfsMethods.
Okay. I suppose that should be documented. Would you mind adding
{{@InterfaceAudience.Private}} to the class?
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565-002.patch, HDFS-6565-003.patch, HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344555#comment-14344555
]
Haohui Mai commented on HDFS-6565:
--
Looks pretty good to me.
{code}
public static ListToken? toTokenList(final Map?, ? json) throws
IOException {
{code}
Looks like the function can be deleted.
{code}
+} catch (IOException e) {
+ // can not reach here
+}
{code}
Simply make it like:
{code}
} catch (IOException ignored) {
}
{code}
{code}
+final long len = ((Number) m.get(length)).longValue();
{code}
There are many changes like this -- it seems that they are unnecessary?
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565-002.patch, HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041409#comment-14041409
]
Akira AJISAKA commented on HDFS-6565:
-
Attaching a patch to remove jetty json library from JsonUtil and
WebHdfsFileSystem.
The way Jackson parse JSON number is different from jetty json:
* Jackson: number - Integer, Long, or BigInteger (smallest applicable)
* jetty json: number - Long
so I changed the code for parsing JSON number
{code}
(Long) m.get(blockId) // doesn't work if m.get(blockId) is Integer
{code}
to
{code}
((Number) m.get(blockId)).longValue() // support all classes extends Number
{code}
In addition, the way Jackson parse JSON array is different from jetty json:
* Jackson: array - ArrayListObject
* jetty json: array - Object[]
so I changed the code for parsing JSON array
{code}
(Object[]) m.get(locatedBlocks)
{code}
to
{code}
(ListObject) m.get(locatedBlocks)
{code}
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041608#comment-14041608
]
Hadoop QA commented on HDFS-6565:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652077/HDFS-6565.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7217//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7217//console
This message is automatically generated.
Use jackson instead jetty json in hdfs-client
-
Key: HDFS-6565
URL: https://issues.apache.org/jira/browse/HDFS-6565
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Haohui Mai
Assignee: Akira AJISAKA
Attachments: HDFS-6565.patch
hdfs-client should use Jackson instead of jetty to parse JSON.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
