Github user liyezhang556520 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3212#discussion_r20199515
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -451,24 +451,25 @@ class SparkSubmitSuite extends FunSuite with Matchers
{
}
}
-object JarCreationTest {
+object JarCreationTest extends Logging {
def main(args: Array[String]) {
Utils.configTestLog4j("INFO")
val conf = new SparkConf()
val sc = new SparkContext(conf)
val result = sc.makeRDD(1 to 100, 10).mapPartitions { x =>
- var foundClasses = false
+ var exception: String = null
try {
Class.forName("SparkSubmitClassA", true,
Thread.currentThread().getContextClassLoader)
Class.forName("SparkSubmitClassA", true,
Thread.currentThread().getContextClassLoader)
- foundClasses = true
} catch {
- case _: Throwable => // catch all
+ case t: Throwable =>
+ exception = t + "\n" + t.getStackTraceString
+ exception = exception.replaceAll("\n", "\n\t")
}
- Seq(foundClasses).iterator
+ Option(exception).toSeq.iterator
}.collect()
- if (result.contains(false)) {
- throw new Exception("Could not load user defined classes inside of
executors")
+ if (result.nonEmpty) {
+ throw new Exception("Could not load user class from jar:\n" +
result(0))
--- End diff --
Hi @andrewor14 , only show one class not found? Is it better to show all
classes not found when there are more than one?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
