[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14519838#comment-14519838
]
stack commented on HBASE-13149:
---
Excellent release note [~jerryhe]
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Assignee: Jerry He
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 1.2.0
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517655#comment-14517655
]
Jerry He commented on HBASE-13149:
--
Added ReleaseNotes. Hope it is not too much or too little.
Feel free to comment on it or update it.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Assignee: Jerry He
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 1.2.0
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14515861#comment-14515861
]
Jerry He commented on HBASE-13149:
--
Thanks for resolving this [~ndimiduk]!
The test failures in the various branches are not related:
On HBase-TRUNK, HBase-1.2 and HBase-1.1: TestHBaseFsck, which has been
reported by HBASE-13574
On branch-1.1: TestSnapshotCloneIndependence, which I have seen occasional
failure in the builds previously.
There is no objection to this commit so far.
I will put a ReleaseNotes to warn any possible incompatibility issue and
workarounds.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Assignee: Jerry He
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 1.2.0
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14513294#comment-14513294
]
Hudson commented on HBASE-13149:
FAILURE: Integrated in HBase-1.1 #431 (See
[https://builds.apache.org/job/HBase-1.1/431/])
HBASE-13149 HBase MR is broken on Hadoop 2.5+ Yarn (Jerry He) (ndimiduk: rev
555679d8e607109368d0e03448c5fd2cf71b7bc0)
* pom.xml
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Assignee: Jerry He
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 1.2.0
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14513326#comment-14513326
]
Hudson commented on HBASE-13149:
FAILURE: Integrated in HBase-TRUNK #6415 (See
[https://builds.apache.org/job/HBase-TRUNK/6415/])
HBASE-13149 HBase MR is broken on Hadoop 2.5+ Yarn (Jerry He) (ndimiduk: rev
9ae023fcab15d0e628bcb21c40f4043890a1feb8)
* pom.xml
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Assignee: Jerry He
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 1.2.0
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14513274#comment-14513274
]
Hudson commented on HBASE-13149:
FAILURE: Integrated in HBase-1.2 #24 (See
[https://builds.apache.org/job/HBase-1.2/24/])
HBASE-13149 HBase MR is broken on Hadoop 2.5+ Yarn (Jerry He) (ndimiduk: rev
39b7a55580206a3b6f6ec5a3f39b1aca2ea50ac0)
* pom.xml
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Assignee: Jerry He
Priority: Blocker
Fix For: 2.0.0, 1.1.0, 1.2.0
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364135#comment-14364135
]
Enis Soztutar commented on HBASE-13149:
---
bq. As for client API compatibility... Good point. I do find that a stretch,
though, since this just happens to be a library that we ship with and that a
user might have compiled against and that might break compilation now. If that
is the only concern I honestly think we can make the change.
+1.
It is our API that we are making promises for source compatibility, not
other's API coming through as a dependency. I thought the dependency
compatibility is for that, and hence my argument for dropping support for it
in minor versions.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362788#comment-14362788
]
Lars Hofhansl commented on HBASE-13149:
---
So it's not Dependency Compatibility we're talking about. I agree.
As for client API compatibility... Good point. I do find that a stretch,
though, since this just happens to be a library that we ship with and that a
user _might_ have compiled against and that _might_ break compilation now. If
that is the only concern I honestly think we can make the change.
Do we know for sure this is the only situation where things are potentially
broken?
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362621#comment-14362621
]
Lars Hofhansl commented on HBASE-13149:
---
Just asked on the mailing list, but now think I should have asked here.
What *precisely* will break when upgrade Jackson to 1.9? As long as we do not
break client-server, server-server, file-formats, etc, we're good. IMHO.
Specifically we *can* break client binary compatibility. The section about
Dependency Compatibility is about not pulling in anything that breaks any of
guarantees we just so carefully stated (my main worry there was to pull in an
incompatible version of Hadoop that breaks the wire-protocol).
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362697#comment-14362697
]
Sean Busbey commented on HBASE-13149:
-
{quote}
What precisely will break when upgrade Jackson to 1.9? As long as we do not
break client-server, server-server, file-formats, etc, we're good. IMHO.
Specifically we can break client binary compatibility. The section about
Dependency Compatibility is about not pulling in anything that breaks any of
guarantees we just so carefully stated (my main worry there was to pull in an
incompatible version of Hadoop that breaks the wire-protocol).
{quote}
Some downstream applications that compiled against HBase 1.0.0 will fail to
compile against HBase 1.1.0. Specifically, it will be those applications that
relied on the hbase-client artifact and used things that were present in
jackson 1.8.8 that are no longer in 1.9 (e.g. the breaking changes mentioned in
the release notes, or those that show up in the source compatibility tab of
the jackson 1.8 to 1.9 compatibility report).
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362703#comment-14362703
]
Lars Hofhansl commented on HBASE-13149:
---
That's fine, no? Since there is no requirement to upgrade the HBase client for
a minor version... Clients can stay at the old version with the old Jackson.
I think we're fine making this change, unless there are other effects.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362711#comment-14362711
]
Sean Busbey commented on HBASE-13149:
-
Our promise of client API compatibility on minor release versions makes me
think that I should be able to recompile my client against a new minor version
without any issue.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359155#comment-14359155
]
Andrew Purtell commented on HBASE-13149:
bq. I see this issue as a test of the projects resolve on semvar.
I feel like this is an over the top statement. We are limited on what we can
realistically promise by how we ourselves are affected by minor bumps of
dependencies. Backing ourselves into a corner based on policy would be a first
for this community and, IMHO, an unwelcome departure from earlier practice.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355362#comment-14355362
]
Enis Soztutar commented on HBASE-13149:
---
bq. Our compatibility guide currently says we will only cause a breaking
dependency change on major version. The comparison Jerry He did above shows
that Jackson 1.8 - Jackson 1.9 is a breaking dependency change, unless we're
missing something in the jackson library that scopes which packages / classes
should be relied on by downstream clients.
I think the changes are in the internals of Jackson from what I see in the
report for jackson-core. I do not realistically expect us to do a full sweep of
every dependency compat story, so we should rely on what our dependencies use
for versioning. Assuming that jackson (or other deps) uses semver or something
similar going from 1.8 to 1.9 should be fine.
I do not think we should corner ourselves into cannot upgrade dependencies
situation (I think [~ndimiduk] started a thread on this some time ago). From
what I see, inconveniencing our users with not upgrading this lib in 1.1 is far
worse than upgrading it, but possibly causing some minor (if any) inconvenience
in case the user is relying on that lib explicitly.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355519#comment-14355519
]
Enis Soztutar commented on HBASE-13149:
---
Ok, let's continue on dev@, and come back here.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355413#comment-14355413
]
stack commented on HBASE-13149:
---
Nice report [~busbey] if damning. I like the list of possible solns. Inventive
(e.g. 1.1 = 2.0).
I see this issue as a test of the projects resolve on semvar. Can we surface on
dev list? The issues discussed in here are too large for a single issue (and
besides the discussion is quality)
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355439#comment-14355439
]
Sean Busbey commented on HBASE-13149:
-
{quote}
I think the changes are in the internals of Jackson from what I see in the
report for jackson-core.
{quote}
I've attached a single report for everything, which makes it easier to spot
things that are broken.
Additionally, the release notes pointed out by [~jinghe] earlier makes it clear
that there are breaking changes:
{quote}
Here is more info on the jackson releases:
http://svn.codehaus.org/jackson/trunk/release-notes/VERSION
You can see in this Release Notes the following for 1.9:
{noformat}
Potential backwards compatibility issues (compared to 1.8.x):
* Removed 'org.codehaus.jackson.annotate.JsonClass, JsonKeyClass and
JsonContentClass (deprecated since 1.1)
* Move TokenBufferDeserializer to separate class (from inside StdDeserializer)
{noformat}
{quote}
This will break anyone relying on the old annotation approach to using JSON
with POJOs. Semver makes no allowance for removing just deprecated stuff on
minor version and the [jackson versioning statement doesn't carve it
out|https://github.com/FasterXML/jackson-docs#on-jackson-versioning]
{quote}
I do not realistically expect us to do a full sweep of every dependency compat
story, so we should rely on what our dependencies use for versioning. Assuming
that jackson (or other deps) uses semver or something similar going from 1.8 to
1.9 should be fine.
{quote}
I am willing to do such a check so long as we are going to make downstream
promises. I believe I can also automate it. I think the above notes on jackson
make it clear that they did not follow semver for 1.8 - 1.9, regardless of
what their stated policy is.
{quote}
I do not think we should corner ourselves into cannot upgrade dependencies
situation (I think Nick Dimiduk started a thread on this some time ago). From
what I see, inconveniencing our users with not upgrading this lib in 1.1 is far
worse than upgrading it, but possibly causing some minor (if any) inconvenience
in case the user is relying on that lib explicitly.
{quote}
This is fine, provided we update our compatibility promises to reflect this
reality. There is pain in either direction, we just need to set expectations
for downstream folks. Note that options #2 and #3 I provided above will meet
the goal of fixing this by updating the dependency while having a compatibility
promise that is realistic for downstream folks.
If we are going to set a precedent that's different from our current
compatibility promises for dependencies, we should get a wider audience on dev@.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
jackson_1.8_to_1.9_compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354486#comment-14354486
]
Hadoop QA commented on HBASE-13149:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703591/HBASE-13149-master.patch
against master branch at commit 9b53a1c214f87bd0a38b12e8d308e7e3bde6de84.
ATTACHMENT ID: 12703591
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.io.TestHeapSize
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.activemq.JmsQueueBrowserTest.testReceiveBrowseReceive(JmsQueueBrowserTest.java:60)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13152//console
This message is automatically generated.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353623#comment-14353623
]
Enis Soztutar commented on HBASE-13149:
---
+1 for changing the jackson version in master and branch-1. For 1.0 and 0.98 we
can document this in the book (there is a section about hadoop versions, we can
add it there).
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353740#comment-14353740
]
Sean Busbey commented on HBASE-13149:
-
From reading the compatibility comparison between Jackson 1.8 and 1.9,
updating it in branch-1 would violate our promise not to break any
dependencies on minor version releases.
Can we find a version of jackson that doesn't break yarn, us, and our
compatibility promise for dependencies on branch-1?
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353496#comment-14353496
]
Sean Busbey commented on HBASE-13149:
-
+1 on updating it in 2.0, with a release note.
Shouldn't the RunJar isolation protect us on the client side when launching our
tools? Do we do something that prevents the use of runjar?
IIRC the runjar isolation is only on 2.6. For versions that don't have it, we
can give folks instructions on manually replacing the impacted library (as we
do for replacing the hadoop jars) with a note that they'll need to update their
application to ensure it also uses the correct version.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353433#comment-14353433
]
Jerry He commented on HBASE-13149:
--
Hi, folks
We need a fix here.
Should we at least upgrade to jackson 1.9.13 on the 2.0 branch? And maybe even
the 1.1+ branch? We build on Hadoop 2.5.1 by default on these branches.
We'll document clearly for 0.98.
Any suggestions?
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353914#comment-14353914
]
Enis Soztutar commented on HBASE-13149:
---
Running Hbase with jackson 1.8 with Hadoop-2.5+ does not work at all, so I
don't know how compatibility is in the picture.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354255#comment-14354255
]
Jerry He commented on HBASE-13149:
--
Here is more info on the jackson releases:
http://svn.codehaus.org/jackson/trunk/release-notes/VERSION
You can see in this Release Notes the following for 1.9:
{noformat}
Potential backwards compatibility issues (compared to 1.8.x):
* Removed 'org.codehaus.jackson.annotate.JsonClass, JsonKeyClass and
JsonContentClass (deprecated since 1.1)
* Move TokenBufferDeserializer to separate class (from inside StdDeserializer)
{noformat}
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354000#comment-14354000
]
Jerry He commented on HBASE-13149:
--
If the jackson 1.8 appears before jackson 1.9 in the classpath when the job is
submitted, we will have a problem.
Did a little more investigation. 'hbase' script puts hbaes lib/* in front of
hadoop classpath. hbase lib/* is also in front of user provided
HBASE_CLASSPATH.
But hadoop RunJar (or 'hadoop jar') puts hadoop jars and dependency jars in
front of user jars (unless HADOOP_USER_CLASSPATH_FIRST is specified').
Therefore using 'hadoop jar' to run hbase MR class works.
Well, it is a messy classpath issue. The hbase lib jars will need to appear
AFTER the hadoop lib jars.
We probably can not ask user to use 'hadoop jar' to run HBase MR always.
Also it turns out that hbase-server jar only expose limited main classes via
org.apache.hadoop.hbase.mapreduce.Driver.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354170#comment-14354170
]
Sean Busbey commented on HBASE-13149:
-
{quote}
I used 'hbase classname'.
'hbase classpath' would show the jackson jars.
{quote}
That's true, but we don't need the whole of the classpath. It looks like the
reference guide already tells folks that they should be using {{hadoop jar}} to
run everything against HBase (ref [section on the
classpath|http://hbase.apache.org/book.html#hbase.mapreduce.classpath] and
[section on bundled
jobs|http://hbase.apache.org/book.html#_bundled_hbase_mapreduce_jobs]).
How about we just add this failure to the troubleshooting section and reference
the earlier guide sections?
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354135#comment-14354135
]
Enis Soztutar commented on HBASE-13149:
---
bq. This is not true. While I think this issue shows a gap in our testing of
the 1.0.0 release
Most of my tests are done with Hadoop-2.6.0, but I had already replaced the
jackson jar from the installation.
bq. I don't think that gap is large enough to say it doesn't work at all. So
far as we know right now, HBase with a YARN based app (including our bundled MR
tools) doesn't work at all.
Yes, HBase standalone works. But MR support is a big part of HBase.
I think for 0.98 and 1.0 we can treat this as a documentation issue. For 1.1 we
should change the dependency version. Not sure why we cannot bump up a minor
version of a dependency while we are also bumping up a minor version.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354158#comment-14354158
]
Jerry He commented on HBASE-13149:
--
Hi, Sean
I used 'hbase classname'.
'hbase classpath' would show the jackson jars.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354108#comment-14354108
]
Sean Busbey commented on HBASE-13149:
-
{quote}
Running Hbase with jackson 1.8 with Hadoop-2.5+ does not work at all, so I
don't know how compatibility is in the picture.
{quote}
This is not true. While I think this issue shows a gap in our testing of the
1.0.0 release, I don't think that gap is large enough to say it doesn't work at
all. So far as we know right now, HBase with a YARN based app (including our
bundled MR tools) doesn't work at all. Downstream folks who don't care about
running MR or other YARN apps against HBase should work just fine.
This is still a substantial problem, since it impacts folks who want to use our
bundled tools, those who want to run MR directly, those who want to use Hive
(atleast Hive on MR), Sqoop, and possibly Spark (in on-yarn mode).
The way I see it we have a few options that keeps us honest wrt our
compatibility promises, in roughly order of my decreasing preference:
# Find a solution that fixes _just_ the problem at hand: running Hadoop 2.5+
YARN based apps against HBase
#* This might break operational compatibility, but we could do this without
qualms on a minor version.
# Move branch-1 to be 2.0
#* We could fix this in branch-1, make it 2.0.0 and move master to 3.0.0. It
would be simple and in line with our current promises. It would also do a great
job of desensitizing us to major version number increments.
# Change our compatibility promise for dependencies to patch-only
#* I don't like this option because we just made our promises and I think not
offering dependency compat at minor versions will be a substantial
dis-incentive to upgrade
# Take a mulligan on jackson specifically
#* We could site this testing gap and the severity of the breakage in a big
release note. But I really don't like this option because we just made our
compatibility promises. It sets a dangerous precedent that erodes the trust
downstream folks can place in our compatibility promises.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354120#comment-14354120
]
Sean Busbey commented on HBASE-13149:
-
{quote}
If the jackson 1.8 appears before jackson 1.9 in the classpath when the job is
submitted, we will have a problem.
Did a little more investigation. 'hbase' script puts hbaes lib/* in front of
hadoop classpath. hbase lib/* is also in front of user provided HBASE_CLASSPATH.
But hadoop RunJar (or 'hadoop jar') puts hadoop jars and dependency jars in
front of user jars (unless HADOOP_USER_CLASSPATH_FIRST is specified').
Therefore using 'hadoop jar' to run hbase MR class works.
Well, it is a messy classpath issue. The hbase lib jars will need to appear
AFTER the hadoop lib jars.
We probably can not ask user to use 'hadoop jar' to run HBase MR always.
Also it turns out that hbase-server jar only expose limited main classes via
org.apache.hadoop.hbase.mapreduce.Driver.
{quote}
So it sounds like we can document a work around that sets HADOOP_CLASSPATH
equal to the output of {{hbase mapredcp}} and then uses {{hadoop jar}} to run
things. Do I have that right?
Actually, looking at the output of {{hbase mapredcp}}, I don't see jackson
listed at all. Could you include the command you're running to start the job?
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13149) HBase MR is broken on Hadoop 2.5+ Yarn
[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354161#comment-14354161
]
Sean Busbey commented on HBASE-13149:
-
{quote}
For 1.1 we should change the dependency version. Not sure why we cannot bump up
a minor version of a dependency while we are also bumping up a minor version.
{quote}
Our compatibility guide currently says we will only cause a breaking dependency
change on major version. The comparison [~jinghe] did above shows that Jackson
1.8 - Jackson 1.9 is a breaking dependency change, unless we're missing
something in the jackson library that scopes which packages / classes should be
relied on by downstream clients.
HBase MR is broken on Hadoop 2.5+ Yarn
--
Key: HBASE-13149
URL: https://issues.apache.org/jira/browse/HBASE-13149
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
Reporter: Jerry He
Priority: Critical
Attachments: HBASE-13149-0.98.patch,
jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
Running the server MR tools is not working on Yarn version 2.5+.
Running org.apache.hadoop.hbase.mapreduce.Export:
{noformat}
Exception in thread main java.lang.NoSuchMethodError:
org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
at
org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
at
org.apache.hadoop.yarn.util.timeline.TimelineUtils.clinit(TimelineUtils.java:47)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.init(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.init(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
{noformat}
The problem seems to be the jackson jar version. HADOOP-10104 updated
jackson version to 1.9.13. YARN-2092 reported a problem as well.
HBase is using jackson 1.8.8. This version of the jar in the classpath seem
to cause the problem.
Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
