[
https://issues.apache.org/jira/browse/HBASE-20333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16502152#comment-16502152
]
Josh Elser commented on HBASE-20333:
------------------------------------
{quote}By saying "just give me hadoop however you can do that" we have options.
shaded hadoop jars on hadoop 3 and regular hadoop 2 jars on hadoop 2.
{quote}
Got it, wasn't considering the case of folks actually using our new shaded jars
with Hadoop2 at all (pessimistically assuming that folks on h2 will just keep
doing what they've always been doing – but that's a bad reason).
{quote}The provided but not relocate hadoop is what we have now; what we've had
since we first added a shaded jar
{quote}
Ok, if that's what we have been doing, I think it's fine to leave that one
as-is. I was mostly curious as I've noticed fun issues where Hadoop 2.7.4
client jars can't talk to a Hadoop 2.8.3 installation. I'm just worried about
the long-term stability of trying to provide Hadoop client libraries (or tell
people "rebuild hbase against the hadoop you use). I think this is related to
your changes here, but not something we need to hash out right now.
{quote}It's not relocated because parts of Hadoop are in our API. I can go back
and find the jira with me and Elliott going around about it if you'd like
background.
{quote}
No, don't put yourself out :)
> break up shaded client into one with no Hadoop and one that's standalone
> ------------------------------------------------------------------------
>
> Key: HBASE-20333
> URL: https://issues.apache.org/jira/browse/HBASE-20333
> Project: HBase
> Issue Type: Sub-task
> Components: shading
> Affects Versions: 2.0.0
> Reporter: Sean Busbey
> Assignee: Sean Busbey
> Priority: Critical
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20333.WIP.0.patch
>
>
> there are contexts where we want to stay out of our downstream users way wrt
> dependencies, but they need more Hadoop classes than we provide. i.e. any
> downstream client that wants to use both HBase and HDFS in their application,
> or any non-MR YARN application.
> Now that Hadoop also has shaded client artifacts for Hadoop 3, we're also
> providing less incremental benefit by including our own rewritten Hadoop
> classes to avoid downstream needing to pull in all of Hadoop's transitive
> dependencies.
> right now those users need to ensure that any jars from the Hadoop project
> are loaded in the classpath prior to our shaded client jar. This is brittle
> and prone to weird debugging trouble.
> instead, we should have two artifacts: one that just lists Hadoop as a
> prerequisite and one that still includes the rewritten-but-not-relocated
> Hadoop classes.
> We can then use docs to emphasize when each of these is appropriate to use.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
