[jira] [Commented] (YARN-10314) YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars
[
https://issues.apache.org/jira/browse/YARN-10314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17138090#comment-17138090
]
Hudson commented on YARN-10314:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18355 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/18355/])
YARN-10314. YarnClient throws NoClassDefFoundError for (github: rev
fc4ebb0499fe1095b87ff782c265e9afce154266)
* (edit) hadoop-client-modules/hadoop-client-minicluster/pom.xml
* (edit) hadoop-client-modules/hadoop-client-runtime/pom.xml
> YarnClient throws NoClassDefFoundError for WebSocketException with only
> shaded client jars
> --
>
> Key: YARN-10314
> URL: https://issues.apache.org/jira/browse/YARN-10314
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Blocker
> Fix For: 3.3.0
>
>
> After YARN-8778, with only shaded hadoop client jars in classpath Unable to
> submit job.
> CC: [~ayushtkn] confirmed the same. Hive 4.0 doesnot work due to this, shaded
> client is necessary there to avoid guava jar's conflicts.
> {noformat}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/shaded/org/eclipse/jetty/websocket/api/WebSocketException
> at
> org.apache.hadoop.yarn.client.api.YarnClient.createYarnClient(YarnClient.java:92)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.(ResourceMgrDelegate.java:109)
> at org.apache.hadoop.mapred.YARNRunner.(YARNRunner.java:153)
> at
> org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
> at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:130)
> at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:109)
> at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:102)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1545)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1541)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
> at org.apache.hadoop.mapreduce.Job.connect(Job.java:1541)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1570)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1594)
> at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.shaded.org.eclipse.jetty.websocket.api.WebSocketException
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 16 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10314) YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars
[
https://issues.apache.org/jira/browse/YARN-10314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136382#comment-17136382
]
Masatake Iwasaki commented on YARN-10314:
-
Oops.. I needed conf dir on the classpath.
{noformat}
$ java -classpath
etc/hadoop:share/hadoop/common/lib/*:share/hadoop/client/*:share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0-SNAPSHOT.jar
org.apache.hadoop.examples.QuasiMonteCarlo 1 100
Number of Maps = 1
Samples per Map = 100
2020-06-16 16:18:08,250 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
Wrote input for Map #0
Starting Job
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/shaded/org/eclipse/jetty/websocket/api/WebSocketException
at
org.apache.hadoop.yarn.client.api.YarnClient.createYarnClient(YarnClient.java:92)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.(ResourceMgrDelegate.java:109)
at org.apache.hadoop.mapred.YARNRunner.(YARNRunner.java:153)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:130)
at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:109)
at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:102)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1545)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1541)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1541)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1570)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1594)
at
org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:307)
at
org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:360)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at
org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:368)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.shaded.org.eclipse.jetty.websocket.api.WebSocketException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 19 more
{noformat}
> YarnClient throws NoClassDefFoundError for WebSocketException with only
> shaded client jars
> --
>
> Key: YARN-10314
> URL: https://issues.apache.org/jira/browse/YARN-10314
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Blocker
>
> After YARN-8778, with only shaded hadoop client jars in classpath Unable to
> submit job.
> CC: [~ayushtkn] confirmed the same. Hive 4.0 doesnot work due to this, shaded
> client is necessary there to avoid guava jar's conflicts.
> {noformat}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/shaded/org/eclipse/jetty/websocket/api/WebSocketException
> at
> org.apache.hadoop.yarn.client.api.YarnClient.createYarnClient(YarnClient.java:92)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.(ResourceMgrDelegate.java:109)
> at org.apache.hadoop.mapred.YARNRunner.(YARNRunner.java:153)
> at
> org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
> at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:130)
> at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:109)
> at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:102)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1545)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1541)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
> at org.apache.hadoop.mapreduce.Job.connect(Job.java:1541)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1570)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1594)
> at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)
> Caused by: java.lang.ClassNotFoundException:
>
[jira] [Commented] (YARN-10314) YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars
[
https://issues.apache.org/jira/browse/YARN-10314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136379#comment-17136379
]
Masatake Iwasaki commented on YARN-10314:
-
[~vinayakumarb] could you elaborate the condition to reproduce the issue?
I tried following on hadoop dist built by {{mvn package -DskipTests -Pdist
-Pnative}} on trunk.
{noformat}
$ java -classpath
share/hadoop/common/lib/*:share/hadoop/client/*:share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0-SNAPSHOT.jar
org.apache.hadoop.examples.QuasiMonteCarlo 1 100
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
Number of Maps = 1
Samples per Map = 100
Wrote input for Map #0
Starting Job
Job Finished in 1.458 seconds
Estimated value of Pi is 3.2000
{noformat}
> YarnClient throws NoClassDefFoundError for WebSocketException with only
> shaded client jars
> --
>
> Key: YARN-10314
> URL: https://issues.apache.org/jira/browse/YARN-10314
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Blocker
>
> After YARN-8778, with only shaded hadoop client jars in classpath Unable to
> submit job.
> CC: [~ayushtkn] confirmed the same. Hive 4.0 doesnot work due to this, shaded
> client is necessary there to avoid guava jar's conflicts.
> {noformat}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/shaded/org/eclipse/jetty/websocket/api/WebSocketException
> at
> org.apache.hadoop.yarn.client.api.YarnClient.createYarnClient(YarnClient.java:92)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.(ResourceMgrDelegate.java:109)
> at org.apache.hadoop.mapred.YARNRunner.(YARNRunner.java:153)
> at
> org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
> at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:130)
> at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:109)
> at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:102)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1545)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1541)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
> at org.apache.hadoop.mapreduce.Job.connect(Job.java:1541)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1570)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1594)
> at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.shaded.org.eclipse.jetty.websocket.api.WebSocketException
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 16 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10314) YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars
[
https://issues.apache.org/jira/browse/YARN-10314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135669#comment-17135669
]
Vinayakumar B commented on YARN-10314:
--
Simple wordcount example to confirm
{noformat}
export
CLASSPATH=$HADOOP_HOME/share/hadoop/common/lib/*.jar:$HADOOP_HOME/share/hadoop/client/*.jar:$HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples*.jar;
java org.apache.hadoop.examples.WordCount /in /out
{noformat}
> YarnClient throws NoClassDefFoundError for WebSocketException with only
> shaded client jars
> --
>
> Key: YARN-10314
> URL: https://issues.apache.org/jira/browse/YARN-10314
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Blocker
>
> After YARN-8778, with only shaded hadoop client jars in classpath Unable to
> submit job.
> CC: [~ayushtkn] confirmed the same. Hive 4.0 doesnot work due to this, shaded
> client is necessary there to avoid guava jar's conflicts.
> {noformat}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/shaded/org/eclipse/jetty/websocket/api/WebSocketException
> at
> org.apache.hadoop.yarn.client.api.YarnClient.createYarnClient(YarnClient.java:92)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.(ResourceMgrDelegate.java:109)
> at org.apache.hadoop.mapred.YARNRunner.(YARNRunner.java:153)
> at
> org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
> at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:130)
> at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:109)
> at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:102)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1545)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1541)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
> at org.apache.hadoop.mapreduce.Job.connect(Job.java:1541)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1570)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1594)
> at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.shaded.org.eclipse.jetty.websocket.api.WebSocketException
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 16 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
