[ https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Busbey updated HBASE-20332: -------------------------------- Status: Patch Available (was: In Progress) putting up my WIP branch so I can get qabot to run unit tests. -v1 - includes patches for HBASE-20439, HBASE-20440, and HBASE-20442 - remove "provided" scope from things we marked as such in v0 for hbase-mapreduce module (the hbase-mapreduce dependency itself should be at 'provided' scope when used downstream) - clean up hbase-server dependencies in pom (and some modules that rely on it for transitives) > shaded mapreduce module shouldn't include hadoop > ------------------------------------------------ > > Key: HBASE-20332 > URL: https://issues.apache.org/jira/browse/HBASE-20332 > Project: HBase > Issue Type: Sub-task > Components: mapreduce, shading > Affects Versions: 2.0.0 > Reporter: Sean Busbey > Assignee: Sean Busbey > Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch > > > AFAICT, we should just entirely skip including hadoop in our shaded mapreduce > module > 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}} > 2) those commands include all the needed Hadoop jars in your classpath by > default (both client side and in the containers) > 3) If you try to use "user classpath first" for your job as a workaround > (e.g. for some library your application needs that hadoop provides) then our > inclusion of *some but not all* hadoop classes then causes everything to fall > over because of mixing rewritten and non-rewritten hadoop classes > 4) if you don't use "user classpath first" then all of our > non-relocated-but-still-shaded hadoop classes are ignored anyways so we're > just wasting space -- This message was sent by Atlassian JIRA (v7.6.3#76005)