Sean Busbey commented on HBASE-20332:

The reason I'm concerned about the checkstyle issues is because I know we have 
some modules configured to failOnViolation=true and I don't want the build to 
start failing due to these changes.{quote}

Oh sweet. I wasn't aware we had reached that point anywhere. Lemme figure 
something out.

> shaded mapreduce module shouldn't include hadoop
> ------------------------------------------------
>                 Key: HBASE-20332
>                 URL: https://issues.apache.org/jira/browse/HBASE-20332
>             Project: HBase
>          Issue Type: Sub-task
>          Components: mapreduce, shading
>    Affects Versions: 2.0.0
>            Reporter: Sean Busbey
>            Assignee: Sean Busbey
>            Priority: Critical
>             Fix For: 3.0.0, 2.1.0
>         Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch, 
> HBASE-20332.2.WIP.patch, HBASE-20332.3.patch, HBASE-20332.4.patch, 
> HBASE-20332.5.patch
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce 
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by 
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround 
> (e.g. for some library your application needs that hadoop provides) then our 
> inclusion of *some but not all* hadoop classes then causes everything to fall 
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our 
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're 
> just wasting space

This message was sent by Atlassian JIRA

Reply via email to