[
https://issues.apache.org/jira/browse/HADOOP-17324?focusedWorklogId=504395&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504395
]
ASF GitHub Bot logged work on HADOOP-17324:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 24/Oct/20 01:54
Start Date: 24/Oct/20 01:54
Worklog Time Spent: 10m
Work Description: sunchao commented on pull request #2411:
URL: https://github.com/apache/hadoop/pull/2411#issuecomment-715655226
> I'm not sure I understand the use case here. Is the MiniYARNCluster simply
broken out of box? Or that Spark wants to test using MiniYARNCluster but
substitue a custom version of BouncyCastle?
Yes I'd say it's broken out of box since 3.3.0: there are two issues, this
one and [HADOOP-17327](https://issues.apache.org/jira/browse/HADOOP-17327).
Spark doesn't use a custom version but since bouncycastle is relocated _and_
not shaded, there is no way for Spark to introduce the dependency, so this
never works ...
> Our client jar contains only one IT ITUseMiniCluster and it checks only
hdfs and webhdfs clients. How about adding an IT for MiniYARNCluster?
This is a good suggestion, thanks. I'll perhaps add it in HADOOP-17327 since
without MiniYARNCluster won't work without both being resolved.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 504395)
Time Spent: 50m (was: 40m)
> Don't relocate org.bouncycastle in shaded client jars
> -----------------------------------------------------
>
> Key: HADOOP-17324
> URL: https://issues.apache.org/jira/browse/HADOOP-17324
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Chao Sun
> Assignee: Chao Sun
> Priority: Critical
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When downstream apps depend on {{hadoop-client-api}},
> {{hadoop-client-runtime}} and {{hadoop-client-minicluster}}, it seems the
> {{MiniYARNCluster}} could have issue because
> {{org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException}}
> is not in any of the above jars.
> {code}
> Error: Caused by: sbt.ForkMain$ForkError: java.lang.ClassNotFoundException:
> org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
> Error: at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> Error: at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
> Error: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> Error: at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
> Error: at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:862)
> Error: at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error: at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1296)
> Error: at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:339)
> Error: at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error: at
> org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:353)
> Error: at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:127)
> Error: at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:488)
> Error: at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error: at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
> Error: at
> org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:321)
> Error: at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error: at
> org.apache.spark.deploy.yarn.BaseYarnClusterSuite.beforeAll(BaseYarnClusterSuite.scala:94)
> Error: at
> org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
> Error: at
> org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
> Error: at
> org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
> Error: at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
> Error: at
> org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
> Error: at
> org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
> Error: at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
> Error: at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> Error: at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Error: at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> Error: at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
