[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440540#comment-16440540
]
stack commented on HBASE-20332:
-------------------------------
How's this work relate to what we have in guide since time immemorial?
http://hbase.apache.org/book.html#hbase.mapreduce.classpath Especially your #1
and #2 above.
That maven-shade-plugin bug sounds like something that would cause the burning
of hours of WTF!
bq. i.e. remove the top level listing of it as a dependency
Agree. Was privvy to a half-hearted attempt in the past. Need to come back to
it.
The workarounds are for the doubled mention of the shaded hbase mr jar?
Skimmed the patch.
Is this right?
199 <artifactId>hbase-server</artifactId>
200 <scope>provided</scope>
How is hbase-server provided at runtime if not in the hbase-shaded-mapreduce
jar?
Nice additions in ensure-jars-have-correct-contents.sh
s/pacakge/package/
s/ and and / and /
Say more why the new module hbase-shaded-with-hadoop-check-invariants?
Following...
> shaded mapreduce module shouldn't include hadoop
> ------------------------------------------------
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, shading
> Affects Versions: 2.0.0
> Reporter: Sean Busbey
> Assignee: Sean Busbey
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20332.0.patch
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround
> (e.g. for some library your application needs that hadoop provides) then our
> inclusion of *some but not all* hadoop classes then causes everything to fall
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're
> just wasting space
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
