[
https://issues.apache.org/jira/browse/HADOOP-19652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18014475#comment-18014475
]
ASF GitHub Bot commented on HADOOP-19652:
-----------------------------------------
pan3793 opened a new pull request, #7878:
URL: https://github.com/apache/hadoop/pull/7878
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
When trying to use the Hadoop trunk version client with Spark 4.0.0,
`NoClassDefFoundError` was raised.
```
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/shaded/javax/ws/rs/WebApplicationException
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:962)
at
java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:144)
at
java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:776)
at
java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:691)
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:620)
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:578)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:490)
at
org.apache.spark.deploy.yarn.YarnRMClient.getAmIpFilterParams(YarnRMClient.scala:109)
at
org.apache.spark.deploy.yarn.ApplicationMaster.addAmIpFilter(ApplicationMaster.scala:698)
at
org.apache.spark.deploy.yarn.ApplicationMaster.runExecutorLauncher(ApplicationMaster.scala:555)
at
org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:265)
at
org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:942)
at
org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:941)
at
java.base/jdk.internal.vm.ScopedValueContainer.callWithoutScope(ScopedValueContainer.java:162)
at
java.base/jdk.internal.vm.ScopedValueContainer.call(ScopedValueContainer.java:147)
at java.base/java.lang.ScopedValue$Carrier.call(ScopedValue.java:419)
at java.base/javax.security.auth.Subject.callAs(Subject.java:331)
at org.apache.hadoop.util.SubjectUtil.callAs(SubjectUtil.java:134)
at org.apache.hadoop.util.SubjectUtil.doAs(SubjectUtil.java:166)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:2039)
at
org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:941)
at
org.apache.spark.deploy.yarn.ExecutorLauncher$.main(ApplicationMaster.scala:973)
at
org.apache.spark.deploy.yarn.ExecutorLauncher.main(ApplicationMaster.scala)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.shaded.javax.ws.rs.WebApplicationException
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:580)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:490)
... 24 more
```
### How was this patch tested?
Hadoop Client Runtime 3.4.2 RC2
<img width="407" height="459" alt="image"
src="https://github.com/user-attachments/assets/9ba6eb0c-2d52-4034-8f97-01c73195d795";
/>
Hadoop Client Runtime 3.5.0-SNAPSHOT trunk
<img width="516" height="432" alt="image"
src="https://github.com/user-attachments/assets/81115ab8-3f86-4336-9d95-fe61093b09d1";
/>
Hadoop Client Runtime 3.5.0-SNAPSHOT HADOOP-19652
<img width="509" height="433" alt="image"
src="https://github.com/user-attachments/assets/eb08a2d8-6b52-45f4-b56e-4ea3a92e950a";
/>
Tested by submitting a Spark application to a YARN cluster.
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Fix dependency exlucsion list of hadoop-client-runtime
> ------------------------------------------------------
>
> Key: HADOOP-19652
> URL: https://issues.apache.org/jira/browse/HADOOP-19652
> Project: Hadoop Common
> Issue Type: Bug
> Components: client-mounts
> Affects Versions: 3.5.0
> Reporter: Cheng Pan
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
