[
https://issues.apache.org/jira/browse/SPARK-35532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-35532:
-----------------------------
Description:
Execute
{code:java}
mvn test -pl mllib -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
-Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
-Pscala-2.13
{code}
6 java tests error:
{code:java}
[ERROR] Errors:
[ERROR] JavaStreamingLogisticRegressionSuite.javaAPI:78 » TestFailed 20005
was not les...
[ERROR] JavaStreamingKMeansSuite.javaAPI:78 » TestFailed 20040 was not less
than 20000...
[ERROR] JavaPrefixSpanSuite.runPrefixSpan:45 » NoClassDefFound
scala/collection/parall...
[ERROR] JavaPrefixSpanSuite.runPrefixSpanSaveLoad:67 » NoClassDefFound
scala/collectio...
[ERROR] JavaStreamingLinearRegressionSuite.javaAPI:77 » TestFailed 20014 was
not less ...
[ERROR] JavaStatisticsSuite.streamingTest:112 » TestFailed 20043 was not less
than 200...
{code}
The error message is
{code:java}
Exception in thread "streaming-start" java.lang.NoClassDefFoundError:
scala/collection/parallel/TaskSupport
at
org.apache.spark.streaming.util.FileBasedWriteAheadLog.<init>(FileBasedWriteAheadLog.scala:62)
at
org.apache.spark.streaming.util.WriteAheadLogUtils$.$anonfun$createLog$2(WriteAheadLogUtils.scala:125)
at scala.Option.getOrElse(Option.scala:201)
at
org.apache.spark.streaming.util.WriteAheadLogUtils$.createLog(WriteAheadLogUtils.scala:123)
at
org.apache.spark.streaming.util.WriteAheadLogUtils$.createLogForDriver(WriteAheadLogUtils.scala:81)
at
org.apache.spark.streaming.scheduler.ReceivedBlockTracker.$anonfun$createWriteAheadLog$1(ReceivedBlockTracker.scala:265)
at scala.Option.map(Option.scala:242)
at
org.apache.spark.streaming.scheduler.ReceivedBlockTracker.createWriteAheadLog(ReceivedBlockTracker.scala:263)
at
org.apache.spark.streaming.scheduler.ReceivedBlockTracker.<init>(ReceivedBlockTracker.scala:76)
at
org.apache.spark.streaming.scheduler.ReceiverTracker.<init>(ReceiverTracker.scala:114)
at
org.apache.spark.streaming.scheduler.JobScheduler.start(JobScheduler.scala:86)
at
org.apache.spark.streaming.StreamingContext.$anonfun$start$1(StreamingContext.scala:590)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
at org.apache.spark.util.ThreadUtils$$anon$2.run(ThreadUtils.scala:218)
Caused by: java.lang.ClassNotFoundException:
scala.collection.parallel.TaskSupport
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 14 more
{code}
Execute
{code:java}
mvn test -pl external/kafka-0-10 -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos
-Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes
-Phive -Pscala-2.13
{code}
1 java test error:
{code:java}
[ERROR] Errors:
[ERROR]
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream
[ERROR] Run 1: JavaDirectKafkaStreamSuite.testKafkaStream:170
expected:<[topic1-1, topic1-2, topic2-1, topic1-3, topic2-2, topic2-3]> but
was:<[]>
[ERROR] Run 2: JavaDirectKafkaStreamSuite.tearDown:57 » NoClassDefFound
scala/collection/para...
{code}
The error message is:
{code:java}
[ERROR] Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 20.772
s <<< FAILURE! - in
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite
[ERROR]
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream
Time elapsed: 20.765 s <<< FAILURE!
java.lang.AssertionError: expected:<[topic1-1, topic1-2, topic2-1, topic1-3,
topic2-2, topic2-3]> but was:<[]>
at
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream(JavaDirectKafkaStreamSuite.java:170)
[ERROR]
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream
Time elapsed: 20.766 s <<< ERROR!
java.lang.NoClassDefFoundError: scala/collection/parallel/immutable/ParVector
at
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.tearDown(JavaDirectKafkaStreamSuite.java:57)
{code}
was:
Execute
{code:java}
mvn test -pl mllib -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
-Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
-Pscala-2.13
{code}
6 java tests error:
{code:java}
[ERROR] Errors:
[ERROR] JavaStreamingLogisticRegressionSuite.javaAPI:78 » TestFailed 20005
was not les...
[ERROR] JavaStreamingKMeansSuite.javaAPI:78 » TestFailed 20040 was not less
than 20000...
[ERROR] JavaPrefixSpanSuite.runPrefixSpan:45 » NoClassDefFound
scala/collection/parall...
[ERROR] JavaPrefixSpanSuite.runPrefixSpanSaveLoad:67 » NoClassDefFound
scala/collectio...
[ERROR] JavaStreamingLinearRegressionSuite.javaAPI:77 » TestFailed 20014 was
not less ...
[ERROR] JavaStatisticsSuite.streamingTest:112 » TestFailed 20043 was not less
than 200...
{code}
The error message is
{code:java}
Exception in thread "streaming-start" java.lang.NoClassDefFoundError:
scala/collection/parallel/TaskSupport
at
org.apache.spark.streaming.util.FileBasedWriteAheadLog.<init>(FileBasedWriteAheadLog.scala:62)
at
org.apache.spark.streaming.util.WriteAheadLogUtils$.$anonfun$createLog$2(WriteAheadLogUtils.scala:125)
at scala.Option.getOrElse(Option.scala:201)
at
org.apache.spark.streaming.util.WriteAheadLogUtils$.createLog(WriteAheadLogUtils.scala:123)
at
org.apache.spark.streaming.util.WriteAheadLogUtils$.createLogForDriver(WriteAheadLogUtils.scala:81)
at
org.apache.spark.streaming.scheduler.ReceivedBlockTracker.$anonfun$createWriteAheadLog$1(ReceivedBlockTracker.scala:265)
at scala.Option.map(Option.scala:242)
at
org.apache.spark.streaming.scheduler.ReceivedBlockTracker.createWriteAheadLog(ReceivedBlockTracker.scala:263)
at
org.apache.spark.streaming.scheduler.ReceivedBlockTracker.<init>(ReceivedBlockTracker.scala:76)
at
org.apache.spark.streaming.scheduler.ReceiverTracker.<init>(ReceiverTracker.scala:114)
at
org.apache.spark.streaming.scheduler.JobScheduler.start(JobScheduler.scala:86)
at
org.apache.spark.streaming.StreamingContext.$anonfun$start$1(StreamingContext.scala:590)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
at org.apache.spark.util.ThreadUtils$$anon$2.run(ThreadUtils.scala:218)
Caused by: java.lang.ClassNotFoundException:
scala.collection.parallel.TaskSupport
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 14 more
{code}
Execute
{code:java}
mvn test -pl mllib -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
-Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
-Pscala-2.13
{code}
1 java test error:
{code:java}
[ERROR] Errors:
[ERROR]
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream
[ERROR] Run 1: JavaDirectKafkaStreamSuite.testKafkaStream:170
expected:<[topic1-1, topic1-2, topic2-1, topic1-3, topic2-2, topic2-3]> but
was:<[]>
[ERROR] Run 2: JavaDirectKafkaStreamSuite.tearDown:57 » NoClassDefFound
scala/collection/para...
{code}
The error message is:
{code:java}
[ERROR] Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 20.772
s <<< FAILURE! - in
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite
[ERROR]
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream
Time elapsed: 20.765 s <<< FAILURE!
java.lang.AssertionError: expected:<[topic1-1, topic1-2, topic2-1, topic1-3,
topic2-2, topic2-3]> but was:<[]>
at
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream(JavaDirectKafkaStreamSuite.java:170)
[ERROR]
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream
Time elapsed: 20.766 s <<< ERROR!
java.lang.NoClassDefFoundError: scala/collection/parallel/immutable/ParVector
at
org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.tearDown(JavaDirectKafkaStreamSuite.java:57)
{code}
> Ensure mllib and kafka-0-10 module can maven test independently in Scala 2.13
> -----------------------------------------------------------------------------
>
> Key: SPARK-35532
> URL: https://issues.apache.org/jira/browse/SPARK-35532
> Project: Spark
> Issue Type: Sub-task
> Components: Tests
> Affects Versions: 3.2.0
> Reporter: Yang Jie
> Priority: Minor
>
> Execute
>
> {code:java}
> mvn test -pl mllib -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
> -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
> -Pscala-2.13
> {code}
>
> 6 java tests error:
>
> {code:java}
> [ERROR] Errors:
> [ERROR] JavaStreamingLogisticRegressionSuite.javaAPI:78 » TestFailed 20005
> was not les...
> [ERROR] JavaStreamingKMeansSuite.javaAPI:78 » TestFailed 20040 was not less
> than 20000...
> [ERROR] JavaPrefixSpanSuite.runPrefixSpan:45 » NoClassDefFound
> scala/collection/parall...
> [ERROR] JavaPrefixSpanSuite.runPrefixSpanSaveLoad:67 » NoClassDefFound
> scala/collectio...
> [ERROR] JavaStreamingLinearRegressionSuite.javaAPI:77 » TestFailed 20014
> was not less ...
> [ERROR] JavaStatisticsSuite.streamingTest:112 » TestFailed 20043 was not
> less than 200...
> {code}
>
> The error message is
>
> {code:java}
> Exception in thread "streaming-start" java.lang.NoClassDefFoundError:
> scala/collection/parallel/TaskSupport
> at
> org.apache.spark.streaming.util.FileBasedWriteAheadLog.<init>(FileBasedWriteAheadLog.scala:62)
> at
> org.apache.spark.streaming.util.WriteAheadLogUtils$.$anonfun$createLog$2(WriteAheadLogUtils.scala:125)
> at scala.Option.getOrElse(Option.scala:201)
> at
> org.apache.spark.streaming.util.WriteAheadLogUtils$.createLog(WriteAheadLogUtils.scala:123)
> at
> org.apache.spark.streaming.util.WriteAheadLogUtils$.createLogForDriver(WriteAheadLogUtils.scala:81)
> at
> org.apache.spark.streaming.scheduler.ReceivedBlockTracker.$anonfun$createWriteAheadLog$1(ReceivedBlockTracker.scala:265)
> at scala.Option.map(Option.scala:242)
> at
> org.apache.spark.streaming.scheduler.ReceivedBlockTracker.createWriteAheadLog(ReceivedBlockTracker.scala:263)
> at
> org.apache.spark.streaming.scheduler.ReceivedBlockTracker.<init>(ReceivedBlockTracker.scala:76)
> at
> org.apache.spark.streaming.scheduler.ReceiverTracker.<init>(ReceiverTracker.scala:114)
> at
> org.apache.spark.streaming.scheduler.JobScheduler.start(JobScheduler.scala:86)
> at
> org.apache.spark.streaming.StreamingContext.$anonfun$start$1(StreamingContext.scala:590)
> at
> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
> at org.apache.spark.util.ThreadUtils$$anon$2.run(ThreadUtils.scala:218)
> Caused by: java.lang.ClassNotFoundException:
> scala.collection.parallel.TaskSupport
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 14 more
> {code}
>
> Execute
>
> {code:java}
> mvn test -pl external/kafka-0-10 -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud
> -Pmesos -Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl
> -Pkubernetes -Phive -Pscala-2.13
> {code}
>
> 1 java test error:
>
> {code:java}
> [ERROR] Errors:
> [ERROR]
> org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream
> [ERROR] Run 1: JavaDirectKafkaStreamSuite.testKafkaStream:170
> expected:<[topic1-1, topic1-2, topic2-1, topic1-3, topic2-2, topic2-3]> but
> was:<[]>
> [ERROR] Run 2: JavaDirectKafkaStreamSuite.tearDown:57 » NoClassDefFound
> scala/collection/para...
> {code}
>
> The error message is:
> {code:java}
> [ERROR] Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed:
> 20.772 s <<< FAILURE! - in
> org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite
> [ERROR]
> org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream
> Time elapsed: 20.765 s <<< FAILURE!
> java.lang.AssertionError: expected:<[topic1-1, topic1-2, topic2-1, topic1-3,
> topic2-2, topic2-3]> but was:<[]>
> at
> org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream(JavaDirectKafkaStreamSuite.java:170)
> [ERROR]
> org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream
> Time elapsed: 20.766 s <<< ERROR!
> java.lang.NoClassDefFoundError: scala/collection/parallel/immutable/ParVector
> at
> org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.tearDown(JavaDirectKafkaStreamSuite.java:57)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
