[
https://issues.apache.org/jira/browse/OOZIE-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16013298#comment-16013298
]
Robert Kanter commented on OOZIE-2886:
--------------------------------------
Here's the full list of hadoop jars with your patch. That does fix the mixed
versions issue. I see now that it's because of transitive dependencies that we
weren't declaring versions for:
{noformat}
>> [136] 17:07 : oozie-git (master) :: find . -name hadoop*.jar
./client/target/oozie-client-4.4.0-SNAPSHOT-client/oozie-client-4.4.0-SNAPSHOT/lib/hadoop-auth-2.7.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-annotations-2.7.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-api-2.7.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-common-2.7.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-common-2.7.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-resourcemanager-2.7.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-web-proxy-2.7.0.jar
./server/target/dependency/hadoop-annotations-2.7.0.jar
./server/target/dependency/hadoop-auth-2.7.0.jar
./server/target/dependency/hadoop-common-2.7.0.jar
./server/target/dependency/hadoop-yarn-api-2.7.0.jar
./server/target/dependency/hadoop-yarn-common-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-common-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-resourcemanager-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-web-proxy-2.7.0.jar
./sharelib/distcp/target/partial-sharelib/share/lib/distcp/hadoop-distcp-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-annotations-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-api-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-common-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-server-common-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-server-resourcemanager-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-server-web-proxy-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-annotations-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-api-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-common-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-server-common-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-server-resourcemanager-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-server-web-proxy-2.7.0.jar
./sharelib/streaming/target/partial-sharelib/share/lib/mapreduce-streaming/hadoop-streaming-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/distcp/hadoop-distcp-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-annotations-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-api-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-common-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-server-common-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-server-resourcemanager-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-server-web-proxy-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-annotations-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-api-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-common-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-server-common-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-server-resourcemanager-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-server-web-proxy-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/mapreduce-streaming/hadoop-streaming-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-annotations-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-api-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-common-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-common-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-resourcemanager-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-web-proxy-2.7.0.jar
{noformat}
However, we shouldn't be including any Hadoop jars in any of the sharelibs
(other than streaming and distcp) because they'll come in from the cluster
automatically. And both Hive sharelibs are pulling in a bunch of extra Hadoop
jars; we should exclude all of those. Especially because it's stuff like
{{hadoop-yarn-server-resourcemanager}}, which really doesn't belong in Oozie
anyway.
> Ensure consistent versioning of hadoop jars in server and webapp
> ----------------------------------------------------------------
>
> Key: OOZIE-2886
> URL: https://issues.apache.org/jira/browse/OOZIE-2886
> Project: Oozie
> Issue Type: Bug
> Components: build
> Affects Versions: 5.0.0
> Reporter: Robert Kanter
> Assignee: Artem Ervits
> Attachments: OOZIE-2886-0.patch, OOZIE-2886-0.patch
>
>
> The hive and hive2 sharelibs are pulling in Hadoop jars - they shouldn't be.
> Also, we should ensure that if you specify {{-Dhadoop.version=foo}} (e.g.
> 2.7.0), that all Hadoop jars end up as that version.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
