[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446206#comment-16446206
]
Mike Drob commented on HBASE-20332:
-----------------------------------
patch you uploaded appears to actually be 4 patches, the first three of which
have been committed.
hbase-anti needs to print a footer link, I can't even find the results by
poking in the build output patchprocess directory. (fine to do in a follow on
if you can manually link me to the results)
htrace is retiring. there is a chance that it will go _back_ to using
org.htrace package space if it lives on somewhere. we'll address that if it
happens, i suppose.
are we supposed to still exclude htrace from the ban transitive deps enforcer
rule? this is getting confusing.
bravo on commenting why we need some of the dependencies. valiant effort, i'm
sure it will be stale in two weeks, but at least it was up to date once.
do we end up with jackson 1/2 conflict between ourselves and hadoop? looks like
you massaged it all away, maybe? do we need to make BlockCacheUtil and
ObjectModel go away?
so now we build a shaded with hadoop and shaded without hadoop MR artifact?
did we not have a netty.hadoop.version defined before? could've sworn I've seen
it
> shaded mapreduce module shouldn't include hadoop
> ------------------------------------------------
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, shading
> Affects Versions: 2.0.0
> Reporter: Sean Busbey
> Assignee: Sean Busbey
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch,
> HBASE-20332.2.WIP.patch, HBASE-20332.3.patch
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround
> (e.g. for some library your application needs that hadoop provides) then our
> inclusion of *some but not all* hadoop classes then causes everything to fall
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're
> just wasting space
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
