[
https://issues.apache.org/jira/browse/HADOOP-17668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454172#comment-17454172
]
Viraj Jasani commented on HADOOP-17668:
---------------------------------------
As of today, it's almost not possible to upgrade hbase2 version for timeline
service without building hbase with hadoop 3 profile because of the below
mentioned reasons:
HBASE-23833 allows hbase async wal to check for the existance of
hadoop-shaded-protobuf_3_7 jar and if found, it uses class
_org.apache.hadoop.thirdparty.protobuf.MessageLite_ and if not found, it uses
class {_}com.google.protobuf.MessageLite{_}.
Timeline service has one option to use hbase 2.4.8 (latest) with hbase
compatible hadoop version 2.10.0 (rather than hadoop 3 because hadoop 3
compatible hbase artifacts are not available as default hadoop profile, but
rather we have to build hbase with hadoop3 profile, which is not a good
recommendation for users. Phoenix uses [this building
guide|https://github.com/apache/phoenix/blob/5.1/BUILDING.md#hbase-2-and-hadoop-3],
but it's complex and our devs/users might not prefer this level of complexity.
Devs would prefer running tests directly rather than building upstream
application exclusively and then run tests).
However, if Timeline service uses hbase 2.4.x with hadoop 2.10.0, we run into
this problem: hadoop-shaded-protobuf_3_7 comes in the hbase classpath as a
transitive dependency from hadoop-common or hadoop-yarn-api, and if hbase uses
{_}org.apache.hadoop.thirdparty.protobuf.MessageLite{_}, asyncwal would break
because shaded protobuf doesn't work with hadoop 2.10. Hence we can exclude
hadoop-shaded-protobuf_3_7 from hbase classpath, but if we do this, then yarn's
RecordFactoryPBImpl fails to get instantiated because it does require
hadoop-shaded-protobuf_3_7 classes. Hence, we are in deadlock.
We have two options:
# We can bump hbase2 version to 2.2.6 (not latest on hbase2 release line, but
should be good enough for timeline service based on the code analysis). Since
2.2.6 is not considered stable, we should keep hbase1 as the default profile.
# hbase3 alpha release has been rolled out, hence we can wait till hbase3 beta
and first prod ready version roll-out and then create new hbase3 profile. There
is no guarantee of this happening anytime soon though.
[~weichiu] [~aajisaka]
> Use profile hbase-2.0 by default and update hbase version
> ---------------------------------------------------------
>
> Key: HADOOP-17668
> URL: https://issues.apache.org/jira/browse/HADOOP-17668
> Project: Hadoop Common
> Issue Type: Task
> Affects Versions: 3.4.0
> Reporter: Wei-Chiu Chuang
> Priority: Major
>
> We currently use hbase1 profile by default (for those who aren't aware, the
> YARN timeline service uses HBase as the underlying storage). There isn't much
> development activity in HBase 1.x and 2.x is production ready. I think it's
> time to switch to hbase 2 by default.
>
> The HBase 2 version being used is 2.0.2. We should use the more recent
> versions. (e.g. 2.2/2.3/2.4) (And update hbase 1 version as well)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
