[
https://issues.apache.org/jira/browse/HADOOP-11804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634487#comment-15634487
]
Andrew Wang commented on HADOOP-11804:
--------------------------------------
* It looks like we generate dummy JARs for some of the new modules, since we
inherit the jar-plugin executions from the parent pom in hadoop-project-dist.
This isn't a big deal, but I have an example for how to disable them at
HADOOP-13784.
* We need to update {{dev-support/bin/dist-layout-stitching}} so the new
artifacts are included in the release layout. You can run maven with "-Pdist"
to test this without running the full create-release script.
I'm kicking the tires by trying to move Avro to use the shaded api/runtime
artifacts. Compile worked, but the MR tests die with this when trying to make a
new JobConf:
{noformat}
testSort(org.apache.avro.mapred.TestWeather) Time elapsed: 0.011 sec <<<
ERROR!
java.lang.NoClassDefFoundError:
org/apache/hadoop/shaded/org/apache/commons/logging/LogFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:197)
at org.apache.avro.mapred.TestWeather.testSort(TestWeather.java:144)
{noformat}
I looked inside hadoop-client-runtime and don't see commons-logging shaded in
there. dependency-tree shows commons-logging being pulled in as a normal dep,
which I think is expected.
I can provide my WIP avro patch if you'd like to play with it yourself, but it
was pretty trivial.
> POC Hadoop Client w/o transitive dependencies
> ---------------------------------------------
>
> Key: HADOOP-11804
> URL: https://issues.apache.org/jira/browse/HADOOP-11804
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
> Reporter: Sean Busbey
> Assignee: Sean Busbey
> Attachments: HADOOP-11804.1.patch, HADOOP-11804.2.patch,
> HADOOP-11804.3.patch, HADOOP-11804.4.patch, HADOOP-11804.5.patch
>
>
> make a hadoop-client-api and hadoop-client-runtime that i.e. HBase can use to
> talk with a Hadoop cluster without seeing any of the implementation
> dependencies.
> see proposal on parent for details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
