[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[ https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13873281#comment-13873281 ] Hudson commented on YARN-1601: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #454 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/454/]) YARN-1601. 3rd party JARs are missing from hadoop-dist output. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1558465) * /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-yarn-dist.xml * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/pom.xml 3rd party JARs are missing from hadoop-dist output -- Key: YARN-1601 URL: https://issues.apache.org/jira/browse/YARN-1601 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.0.0, 2.4.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Fix For: 2.4.0 Attachments: YARN-1601.patch With the build changes of YARN-888 we are leaving out all 3rd party JArs used directly by YARN under /share/hadoop/yarn/lib/. We did not notice this when running minicluster because they all happen to be in the classpath from hadoop-common and hadoop-yarn. As 3d party JARs are not 'public' interfaces we cannot rely on them being provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd party dependency that yarn uses this would break yarn if yarn does not pull that dependency explicitly). Also, this will break bigtop hadoop build when they move to use branch-2 as they expect to find jars in /share/hadoop/yarn/lib/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[ https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13873371#comment-13873371 ] Hudson commented on YARN-1601: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1671 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1671/]) YARN-1601. 3rd party JARs are missing from hadoop-dist output. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1558465) * /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-yarn-dist.xml * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/pom.xml 3rd party JARs are missing from hadoop-dist output -- Key: YARN-1601 URL: https://issues.apache.org/jira/browse/YARN-1601 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.0.0, 2.4.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Fix For: 2.4.0 Attachments: YARN-1601.patch With the build changes of YARN-888 we are leaving out all 3rd party JArs used directly by YARN under /share/hadoop/yarn/lib/. We did not notice this when running minicluster because they all happen to be in the classpath from hadoop-common and hadoop-yarn. As 3d party JARs are not 'public' interfaces we cannot rely on them being provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd party dependency that yarn uses this would break yarn if yarn does not pull that dependency explicitly). Also, this will break bigtop hadoop build when they move to use branch-2 as they expect to find jars in /share/hadoop/yarn/lib/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[ https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13873393#comment-13873393 ] Hudson commented on YARN-1601: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1646 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1646/]) YARN-1601. 3rd party JARs are missing from hadoop-dist output. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1558465) * /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-yarn-dist.xml * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/pom.xml 3rd party JARs are missing from hadoop-dist output -- Key: YARN-1601 URL: https://issues.apache.org/jira/browse/YARN-1601 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.0.0, 2.4.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Fix For: 2.4.0 Attachments: YARN-1601.patch With the build changes of YARN-888 we are leaving out all 3rd party JArs used directly by YARN under /share/hadoop/yarn/lib/. We did not notice this when running minicluster because they all happen to be in the classpath from hadoop-common and hadoop-yarn. As 3d party JARs are not 'public' interfaces we cannot rely on them being provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd party dependency that yarn uses this would break yarn if yarn does not pull that dependency explicitly). Also, this will break bigtop hadoop build when they move to use branch-2 as they expect to find jars in /share/hadoop/yarn/lib/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[ https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13871837#comment-13871837 ] Steve Loughran commented on YARN-1601: -- Makes sense. Let me do a branch-2 build before and after this patch see what comes out 3rd party JARs are missing from hadoop-dist output -- Key: YARN-1601 URL: https://issues.apache.org/jira/browse/YARN-1601 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.0.0, 2.4.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-1601.patch With the build changes of YARN-888 we are leaving out all 3rd party JArs used directly by YARN under /share/hadoop/yarn/lib/. We did not notice this when running minicluster because they all happen to be in the classpath from hadoop-common and hadoop-yarn. As 3d party JARs are not 'public' interfaces we cannot rely on them being provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd party dependency that yarn uses this would break yarn if yarn does not pull that dependency explicitly). Also, this will break bigtop hadoop build when they move to use branch-2 as they expect to find jars in /share/hadoop/yarn/lib/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[
https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13871920#comment-13871920
]
Steve Loughran commented on YARN-1601:
--
before the patch
{code}
$ find hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn/
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn/
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-api-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-applications-distributedshell-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-applications-unmanaged-am-launcher-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-client-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-common-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-server-common-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-server-nodemanager-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-server-resourcemanager-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-server-tests-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-server-web-proxy-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-api-2.4.0-SNAPSHOT-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-applications-distributedshell-2.4.0-SNAPSHOT-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-applications-distributedshell-2.4.0-SNAPSHOT-test-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-applications-unmanaged-am-launcher-2.4.0-SNAPSHOT-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-applications-unmanaged-am-launcher-2.4.0-SNAPSHOT-test-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-client-2.4.0-SNAPSHOT-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-client-2.4.0-SNAPSHOT-test-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-common-2.4.0-SNAPSHOT-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-common-2.4.0-SNAPSHOT-test-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-server-common-2.4.0-SNAPSHOT-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-server-common-2.4.0-SNAPSHOT-test-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-server-nodemanager-2.4.0-SNAPSHOT-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-server-nodemanager-2.4.0-SNAPSHOT-test-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-server-resourcemanager-2.4.0-SNAPSHOT-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-server-resourcemanager-2.4.0-SNAPSHOT-test-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-server-tests-2.4.0-SNAPSHOT-test-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-server-web-proxy-2.4.0-SNAPSHOT-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-server-web-proxy-2.4.0-SNAPSHOT-test-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//test
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//test/hadoop-yarn-server-tests-2.4.0-SNAPSHOT-tests.jar
{code}
3rd party JARs are missing from hadoop-dist output
--
Key: YARN-1601
URL: https://issues.apache.org/jira/browse/YARN-1601
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 3.0.0, 2.4.0
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
Attachments: YARN-1601.patch
With the build changes of YARN-888 we are leaving out all 3rd party JArs used
directly by YARN under /share/hadoop/yarn/lib/.
We did not notice this when running minicluster because they all happen to be
in the classpath from hadoop-common and hadoop-yarn.
As 3d party JARs are not 'public' interfaces we cannot rely on them being
provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd
party dependency that yarn uses this would break yarn if yarn does not pull
that dependency explicitly).
Also, this will
[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[
https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13871921#comment-13871921
]
Steve Loughran commented on YARN-1601:
--
after the patch
{code}
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn/
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//.DS_Store
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-api-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-applications-distributedshell-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-applications-unmanaged-am-launcher-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-client-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-common-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-server-common-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-server-nodemanager-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-server-resourcemanager-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-server-tests-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//hadoop-yarn-server-web-proxy-2.4.0-SNAPSHOT.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/activation-1.1.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/aopalliance-1.0.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/asm-3.2.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/commons-cli-1.2.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/commons-codec-1.4.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/commons-compress-1.4.1.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/commons-httpclient-3.1.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/commons-io-2.4.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/commons-lang-2.6.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/commons-logging-1.1.3.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/guava-11.0.2.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/guice-3.0.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/guice-servlet-3.0.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jackson-core-asl-1.8.8.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jackson-jaxrs-1.8.8.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jackson-mapper-asl-1.8.8.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jackson-xc-1.8.8.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/javax.inject-1.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jaxb-api-2.2.2.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jaxb-impl-2.2.3-1.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jersey-client-1.9.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jersey-core-1.9.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jersey-guice-1.9.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jersey-json-1.9.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jersey-server-1.9.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jettison-1.1.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jetty-6.1.26.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jetty-util-6.1.26.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jline-0.9.94.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/jsr305-1.3.9.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/log4j-1.2.17.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/protobuf-java-2.5.0.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/servlet-api-2.5.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/stax-api-1.0-2.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/stax-api-1.0.1.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/xz-1.0.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//lib/zookeeper-3.4.5.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-api-2.4.0-SNAPSHOT-sources.jar
hadoop-dist/target/hadoop-2.4.0-SNAPSHOT/share/hadoop/yarn//sources/hadoop-yarn-applications-distributedshell-2.4.0-SNAPSHOT-sources.jar
[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[ https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13871923#comment-13871923 ] Steve Loughran commented on YARN-1601: -- confirmed that problem exists patch does as claimed, +1 3rd party JARs are missing from hadoop-dist output -- Key: YARN-1601 URL: https://issues.apache.org/jira/browse/YARN-1601 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.0.0, 2.4.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-1601.patch With the build changes of YARN-888 we are leaving out all 3rd party JArs used directly by YARN under /share/hadoop/yarn/lib/. We did not notice this when running minicluster because they all happen to be in the classpath from hadoop-common and hadoop-yarn. As 3d party JARs are not 'public' interfaces we cannot rely on them being provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd party dependency that yarn uses this would break yarn if yarn does not pull that dependency explicitly). Also, this will break bigtop hadoop build when they move to use branch-2 as they expect to find jars in /share/hadoop/yarn/lib/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[ https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13872235#comment-13872235 ] Sean Mackrory commented on YARN-1601: - +1 (non-committer) - I can confirm the problem from a Bigtop perspective. If you were to try and build the hadoop-client package it would break because there wouldn't be any other Hadoop package that would provide one of the dependencies Maven determined was needed for a YARN client (specifically, stax-api 1.0-2, which has a different group ID than stax-api 1.0.1). 3rd party JARs are missing from hadoop-dist output -- Key: YARN-1601 URL: https://issues.apache.org/jira/browse/YARN-1601 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.0.0, 2.4.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-1601.patch With the build changes of YARN-888 we are leaving out all 3rd party JArs used directly by YARN under /share/hadoop/yarn/lib/. We did not notice this when running minicluster because they all happen to be in the classpath from hadoop-common and hadoop-yarn. As 3d party JARs are not 'public' interfaces we cannot rely on them being provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd party dependency that yarn uses this would break yarn if yarn does not pull that dependency explicitly). Also, this will break bigtop hadoop build when they move to use branch-2 as they expect to find jars in /share/hadoop/yarn/lib/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[ https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13872270#comment-13872270 ] Alejandro Abdelnur commented on YARN-1601: -- Thanks Steve. Thanks Sean, regarding the stax-api JAR, I've opened another JIRA to fix that, HADOOP-10235. 3rd party JARs are missing from hadoop-dist output -- Key: YARN-1601 URL: https://issues.apache.org/jira/browse/YARN-1601 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.0.0, 2.4.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-1601.patch With the build changes of YARN-888 we are leaving out all 3rd party JArs used directly by YARN under /share/hadoop/yarn/lib/. We did not notice this when running minicluster because they all happen to be in the classpath from hadoop-common and hadoop-yarn. As 3d party JARs are not 'public' interfaces we cannot rely on them being provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd party dependency that yarn uses this would break yarn if yarn does not pull that dependency explicitly). Also, this will break bigtop hadoop build when they move to use branch-2 as they expect to find jars in /share/hadoop/yarn/lib/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[ https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13872287#comment-13872287 ] Hudson commented on YARN-1601: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5001 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5001/]) YARN-1601. 3rd party JARs are missing from hadoop-dist output. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1558465) * /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-yarn-dist.xml * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/pom.xml 3rd party JARs are missing from hadoop-dist output -- Key: YARN-1601 URL: https://issues.apache.org/jira/browse/YARN-1601 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.0.0, 2.4.0 Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Fix For: 2.4.0 Attachments: YARN-1601.patch With the build changes of YARN-888 we are leaving out all 3rd party JArs used directly by YARN under /share/hadoop/yarn/lib/. We did not notice this when running minicluster because they all happen to be in the classpath from hadoop-common and hadoop-yarn. As 3d party JARs are not 'public' interfaces we cannot rely on them being provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd party dependency that yarn uses this would break yarn if yarn does not pull that dependency explicitly). Also, this will break bigtop hadoop build when they move to use branch-2 as they expect to find jars in /share/hadoop/yarn/lib/ -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1601) 3rd party JARs are missing from hadoop-dist output
[
https://issues.apache.org/jira/browse/YARN-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13871556#comment-13871556
]
Hadoop QA commented on YARN-1601:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12623023/YARN-1601.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-assemblies.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2885//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2885//console
This message is automatically generated.
3rd party JARs are missing from hadoop-dist output
--
Key: YARN-1601
URL: https://issues.apache.org/jira/browse/YARN-1601
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 3.0.0, 2.4.0
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
Attachments: YARN-1601.patch
With the build changes of YARN-888 we are leaving out all 3rd party JArs used
directly by YARN under /share/hadoop/yarn/lib/.
We did not notice this when running minicluster because they all happen to be
in the classpath from hadoop-common and hadoop-yarn.
As 3d party JARs are not 'public' interfaces we cannot rely on them being
provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd
party dependency that yarn uses this would break yarn if yarn does not pull
that dependency explicitly).
Also, this will break bigtop hadoop build when they move to use branch-2 as
they expect to find jars in /share/hadoop/yarn/lib/
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
