[jira] [Commented] (HDFS-10871) DiskBalancerWorkItem should not import jackson relocated by htrace
[
https://issues.apache.org/jira/browse/HDFS-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515298#comment-15515298
]
Hudson commented on HDFS-10871:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10477 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10477/])
HDFS-10871. DiskBalancerWorkItem should not import jackson relocated by
(aengineer: rev d85d9b2e7b4df04335111f4e3ce21a2fda39aee9)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/diskbalancer/planner/MoveStep.java
* (edit) hadoop-hdfs-project/hadoop-hdfs-client/pom.xml
* (edit)
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancerWorkItem.java
> DiskBalancerWorkItem should not import jackson relocated by htrace
> --
>
> Key: HDFS-10871
> URL: https://issues.apache.org/jira/browse/HDFS-10871
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: 3.0.0-alpha1
>Reporter: Masatake Iwasaki
>Assignee: Manoj Govindassamy
> Fix For: 3.0.0-alpha2
>
> Attachments: HDFS-10871.001.patch
>
>
> Compiling trunk against upstream htrace fails since it does not bundle the
> {{org.apache.htrace.fasterxml.jackson.annotation.JsonInclude}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10871) DiskBalancerWorkItem should not import jackson relocated by htrace
[
https://issues.apache.org/jira/browse/HDFS-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515127#comment-15515127
]
Anu Engineer commented on HDFS-10871:
-
[~manojg] Thanks for fixing this issue. I will commit this shortly
> DiskBalancerWorkItem should not import jackson relocated by htrace
> --
>
> Key: HDFS-10871
> URL: https://issues.apache.org/jira/browse/HDFS-10871
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: 3.0.0-alpha1
>Reporter: Masatake Iwasaki
>Assignee: Manoj Govindassamy
> Attachments: HDFS-10871.001.patch
>
>
> Compiling trunk against upstream htrace fails since it does not bundle the
> {{org.apache.htrace.fasterxml.jackson.annotation.JsonInclude}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10871) DiskBalancerWorkItem should not import jackson relocated by htrace
[
https://issues.apache.org/jira/browse/HDFS-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515054#comment-15515054
]
Manoj Govindassamy commented on HDFS-10871:
---
Thanks for the review [~anu]. Manually verified the fix against latest upstream
htrace.
> DiskBalancerWorkItem should not import jackson relocated by htrace
> --
>
> Key: HDFS-10871
> URL: https://issues.apache.org/jira/browse/HDFS-10871
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: 3.0.0-alpha1
>Reporter: Masatake Iwasaki
>Assignee: Manoj Govindassamy
> Attachments: HDFS-10871.001.patch
>
>
> Compiling trunk against upstream htrace fails since it does not bundle the
> {{org.apache.htrace.fasterxml.jackson.annotation.JsonInclude}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10871) DiskBalancerWorkItem should not import jackson relocated by htrace
[
https://issues.apache.org/jira/browse/HDFS-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505251#comment-15505251
]
Hadoop QA commented on HDFS-10871:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
6s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
56s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 60m
20s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 90m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HDFS-10871 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12829294/HDFS-10871.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux 0e565a8ec9d7 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 98bdb51 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/16800/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client
hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/16800/console |
| Powered by |
[jira] [Commented] (HDFS-10871) DiskBalancerWorkItem should not import jackson relocated by htrace
[
https://issues.apache.org/jira/browse/HDFS-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505000#comment-15505000
]
Anu Engineer commented on HDFS-10871:
-
[~manojg] Thanks for fixing this so quickly. +1, pending Jenkins.
> DiskBalancerWorkItem should not import jackson relocated by htrace
> --
>
> Key: HDFS-10871
> URL: https://issues.apache.org/jira/browse/HDFS-10871
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: 3.0.0-alpha1
>Reporter: Masatake Iwasaki
>Assignee: Manoj Govindassamy
> Attachments: HDFS-10871.001.patch
>
>
> Compiling trunk against upstream htrace fails since it does not bundle the
> {{org.apache.htrace.fasterxml.jackson.annotation.JsonInclude}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10871) DiskBalancerWorkItem should not import jackson relocated by htrace
[
https://issues.apache.org/jira/browse/HDFS-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504814#comment-15504814
]
Anu Engineer commented on HDFS-10871:
-
Thanks for taking care of this issue. I like proposal 1. Since It is cleaner
and we will need to share json classes between client and server anyway.
> DiskBalancerWorkItem should not import jackson relocated by htrace
> --
>
> Key: HDFS-10871
> URL: https://issues.apache.org/jira/browse/HDFS-10871
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Affects Versions: 3.0.0-alpha1
>Reporter: Masatake Iwasaki
>Assignee: Manoj Govindassamy
>
> Compiling trunk against upstream htrace fails since it does not bundle the
> {{org.apache.htrace.fasterxml.jackson.annotation.JsonInclude}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10871) DiskBalancerWorkItem should not import jackson relocated by htrace
[
https://issues.apache.org/jira/browse/HDFS-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504793#comment-15504793
]
Manoj Govindassamy commented on HDFS-10871:
---
I am able to recreate the problem with htrace upstream. Now, below are the fix
proposals. Both these fixes seem to solve the problem for the latest upstream
htrace.
1. Change the dependency library
* Add {{com.fasterxml.jackson.core}} as a dependent artifact in
{{hadoop-hdfs-project/hadoop-hdfs-client/}} project. Remember,
{{hadoop-hdfs-project/}} project is already dependent on
{{com.fasterxml.jackson.core}}
** hadoop-hdfs-project/hadoop-hdfs-client/pom.xml {code}
109
110
111 com.fasterxml.jackson.core
112 jackson-annotations
113
114{code}
* In {{DiskBalancerWorkItem}} and {{MoveStep}}, replace the import
{{org.apache.htrace.fasterxml.jackson}} with {{com.fasterxml.jackson}} {code}
-import org.apache.htrace.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonInclude;
{code}
* No other {{JsonInclude}} annotation changes are needed
2. Make use of {{org.codehaus.jackson.map.annotate}} instead of
{{org.apache.htrace.fasterxml.jackson}}
* {{DiskBalancerWorkItem}} Object Mapper can have an explicit serialization
inclusion instead of class level annotation {code}
diff --git
a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancerWorkItem.java
b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancerWorkItem.java
index
592a89facf16bb3d046e0f87c83f571c2d68443a..edd2801c072968f5a2dc46941e847bd8cba9157a
100644
---
a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancerWorkItem.java
+++
b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancerWorkItem.java
@@ -19,12 +19,13 @@
package org.apache.hadoop.hdfs.server.datanode;
+import com.fasterxml.jackson.annotation.JsonInclude;
import com.google.common.base.Preconditions;
import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;
-import org.apache.htrace.fasterxml.jackson.annotation.JsonInclude;
import org.codehaus.jackson.map.ObjectMapper;
import org.codehaus.jackson.map.ObjectReader;
+import org.codehaus.jackson.map.annotate.JsonSerialize.Inclusion;
import java.io.IOException;
@@ -33,7 +34,6 @@
*/
@InterfaceAudience.Private
@InterfaceStability.Unstable
-@JsonInclude(JsonInclude.Include.NON_DEFAULT)
public class DiskBalancerWorkItem {
private static final ObjectMapper MAPPER = new ObjectMapper();
private static final ObjectReader READER =
@@ -52,6 +52,13 @@
private long bandwidth;
/**
+ * Initialization block for static members
+ */
+ static {
+MAPPER.setSerializationInclusion(Inclusion.NON_DEFAULT);
+ }
+
+ /**
* Empty constructor for Json serialization.
*/
public DiskBalancerWorkItem() {
{code}
* {{MoveStep}} annotation can be removed as there are no Object Mappers for the
class. Wondering if from/to json strings is necessary for this class at all ?
{code}
diff --git
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/diskbalancer/planner/MoveStep.java
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/diskbalancer/planner/MoveStep.java
index
b5f68fd8ad3ee8a803f039c422c23145935e589d..97fd650808d2b3e29c5d8132c755ef9096fc68de
100644
---
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/diskbalancer/planner/MoveStep.java
+++
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/diskbalancer/planner/MoveStep.java
@@ -17,20 +17,10 @@
package org.apache.hadoop.hdfs.server.diskbalancer.planner;
import org.apache.hadoop.hdfs.server.diskbalancer.datamodel.DiskBalancerVolume;
import org.apache.hadoop.util.StringUtils;
-import org.apache.htrace.fasterxml.jackson.annotation.JsonInclude;
-/**
- * Ignore fields with default values. In most cases Throughtput, diskErrors
- * tolerancePercent and bandwidth will be the system defaults.
- * So we will avoid serializing them into JSON.
- */
-@JsonInclude(JsonInclude.Include.NON_DEFAULT)
/**
* Move step is a step that planner can execute that will move data from one
* volume to another.
{code}
[~eddyxu], [~anu] any thoughts on the above proposals ?
> DiskBalancerWorkItem should not import jackson relocated by htrace
>
[jira] [Commented] (HDFS-10871) DiskBalancerWorkItem should not import jackson relocated by htrace
[
https://issues.apache.org/jira/browse/HDFS-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503945#comment-15503945
]
Masatake Iwasaki commented on HDFS-10871:
-
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hadoop-hdfs-client: Compilation failure: Compilation failure:
[ERROR]
/home/iwasakims/srcs/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancerWorkItem.java:[25,54]
package org.apache.htrace.fasterxml.jackson.annotation does not exist
[ERROR]
/home/iwasakims/srcs/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancerWorkItem.java:[36,2]
cannot find symbol
{noformat}
> DiskBalancerWorkItem should not import jackson relocated by htrace
> --
>
> Key: HDFS-10871
> URL: https://issues.apache.org/jira/browse/HDFS-10871
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 3.0.0-alpha1
>Reporter: Masatake Iwasaki
>
> Compiling trunk against upstream htrace fails since it does not bundle the
> {{org.apache.htrace.fasterxml.jackson.annotation.JsonInclude}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
