[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-13149:
---------------------------------
Release Note:
In HBase 1.1.0 and above we have upgraded the version of Jackson dependencies
(jackson-core-asl, jackson-mapper-asl, jackson-jaxrs and jackson-xc) from 1.8.8
to 1.9.13. This is to follow the upgrade to Jackson 1.9.13 in Hadoop 2.5 and
above which causes Jackson class incompatibility for HBase as reported in
HBASE-13149. Refer to HADOOP-10104 and YARN-2092 for additional information.
Jackson1.9.13 is not completely backward compatible with the prior version
1.8.8 used in HBase. See the Compatibility reports attached in HBASE-13149 and
http://svn.codehaus.org/jackson/trunk/release-notes/VERSION for more
information.
This upgrade does not have direct impact on HBase users and HBase applications
in most cases. In the rare case where your HBase application uses Jackson
directly AND your application has compatibility issue with Jackson 1.9.13, you
can do the following to mitigate the problem.
1. If you are on Hadoop 2.5 or above, and your HBase application involves
running Yarn jobs, we recommend you update your application to use Jackson
1.9.13. You may be able to explore classpath isolation options (e.g.
HADOOP-10893) or have your own classpath isolation strategy that works for you,
but the general recommendation is that you upgrade to Jackson 1.9.13.
2. You may choose to continue using Jackson 1.8.8 and not to use Jackson 1.9.13
in your classpath. You can also choose to replace the Jackson 1.9.13 jars in
$HBASE_HOME/lib with 1.8.8 jars. It can work for you in the following cases:
a) You are on a Hadoop version earlier than Hadoop 2.5, or
b) You are on Hadoop 2.5 or above, but your HBase application does not involve
running Yarn jobs.
3. You may experiment with further isolation using the shaded jars introduced
with 1.1.0 via HBASE-13517.
Note that it may not be tested or guaranteed that using Jackson 1.8.8 in
$HBASE_HOME/lib will work in future HBase releases.
It is recommended that your HBase application matches the Jackson version
provided in HBase.
In HBase 0.98.x and HBase 1.0.x, we have NOT upgraded the version of Jackson
dependencies. If you are on Hadoop 2.5 or above, and your HBase application
involves running Yarn jobs, you may encounter Jackson class incomparability
issue, as reported in HBASE-13149.
You can do the following to mitigate the problem:
1. Use 'hadoop jar' command to run your HBase jobs.
2. Explore classpath isolation options (e.g. HADOOP-10893) or have your own
classpath isolation strategy that works for you.
3. You can also choose to replace the Jackson 1.8.8 jars in $HBASE_HOME/lib
with 1.9.13 jars from your Hadoop lib directory. We have tested HBase 0.98 with
Jackson 1.9.13.
was:
In HBase 1.1.0 and above we have upgraded the version of Jackson dependencies
(jackson-core-asl, jackson-mapper-asl, jackson-jaxrs and jackson-xc) from 1.8.8
to 1.9.13.
This is to follow the upgrade to Jackson 1.9.13 in Hadoop 2.5 and above which
causes Jackson class incompatibility for HBase as reported in HBASE-13149.
Refer to HADOOP-10104 and YARN-2092 for additional information.
Jackson1.9.13 is not completely backward compatible with the prior version
1.8.8 used in HBase. See the Compatibility reports attached in HBASE-13149 and
http://svn.codehaus.org/jackson/trunk/release-notes/VERSION for more
information.
This upgrade does not have direct impact on HBase users and HBase applications
in most cases.
In the rare case where your HBase application uses Jackson directly AND your
application has compatibility issue with Jackson 1.9.13,
you can do the following to mitigate the problem.
1. If you are on Hadoop 2.5 or above, and your HBase application involves
running Yarn jobs, we recommend you update your application to use Jackson
1.9.13. You may be able to explore classpath isolation options (e.g.
HADOOP-10893) or have your own classpath isolation strategy that works for you,
but the general recommendation is that you upgrade to Jackson 1.9.13.
2. You may choose to continue using Jackson 1.8.8 and not to use Jackson 1.9.13
in your classpath. You can also choose to replace the Jackson 1.9.13 jars in
$HBASE_HOME/lib with 1.8.8 jars. It can work for you in the following cases:
a) You are on a Hadoop version earlier than Hadoop 2.5, or
b) You are on Hadoop 2.5 or above, but your HBase application does not involve
running Yarn jobs.
Note that it may not be tested or guaranteed that using Jackson 1.8.8 in
$HBASE_HOME/lib will work in future HBase releases.
It is recommended that your HBase application matches the Jackson version
provided in HBase.
In HBase 0.98.x, we have NOT upgraded the version of Jackson dependencies.
If you are on Hadoop 2.5 or above, and your HBase application involves running
Yarn jobs, you may encounter Jackson class incomparability issue, as reported
in HBASE-13149.
You can do the following to mitigate the problem:
1. Use 'hadoop jar' command to run your HBase jobs.
2. Explore classpath isolation options (e.g. HADOOP-10893) or have your own
classpath isolation strategy that works for you.
3. You can also choose to replace the Jackson 1.8.8 jars in $HBASE_HOME/lib
with 1.9.13 jars from your Hadoop lib directory. We have tested HBase 0.98 with
Jackson 1.9.13.
Really great note [~jinghe]! Let me add a quick bit about 1.0 and also
HBASE-13517.
> HBase MR is broken on Hadoop 2.5+ Yarn
> --------------------------------------
>
> Key: HBASE-13149
> URL: https://issues.apache.org/jira/browse/HBASE-13149
> Project: HBase
> Issue Type: Bug
> Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
> Reporter: Jerry He
> Assignee: Jerry He
> Priority: Blocker
> Fix For: 2.0.0, 1.1.0, 1.2.0
>
> Attachments: HBASE-13149-0.98.patch, HBASE-13149-master.patch,
> jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
> jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html,
> jackson_1.8_to_1.9_compat_report.html
>
>
> Running the server MR tools is not working on Yarn version 2.5+.
> Running org.apache.hadoop.hbase.mapreduce.Export:
> {noformat}
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
> at
> org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
> at
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.<clinit>(TimelineUtils.java:47)
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.<init>(ResourceMgrDelegate.java:96)
> at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112)
> at
> org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
> at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
> at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
> at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
> at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
> {noformat}
> The problem seems to be the jackson jar version. HADOOP-10104 updated
> jackson version to 1.9.13. YARN-2092 reported a problem as well.
> HBase is using jackson 1.8.8. This version of the jar in the classpath seem
> to cause the problem.
> Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
