Pawel Pasterz created BEAM-10088:
------------------------------------
Summary: Spark Runner tests failing [Java 11]
Key: BEAM-10088
URL: https://issues.apache.org/jira/browse/BEAM-10088
Project: Beam
Issue Type: Improvement
Components: runner-spark
Reporter: Pawel Pasterz
Gradle *_:examples:java:sparkRunnerPreCommit _*fails during Java 11 Precommit
job
Example stack trace:
{code:java}
> Task :examples:java:sparkRunnerPreCommit
20/05/26 07:11:13 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Entering directly-translatable composite transform:
'WordCount.CountWords/Count.PerElement/Combine.perKey(Count)'
20/05/26 07:11:13 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Entering directly-translatable composite transform:
'WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle'
20/05/26 07:11:14 INFO
org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics
accumulator: {
"metrics": {
}
}
20/05/26 07:11:14 INFO
org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated
aggregators accumulator:
20/05/26 07:11:14 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating Read(CompressedSource)
20/05/26 07:11:14 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating org.apache.beam.examples.WordCount$ExtractWordsFn@174ba402
20/05/26 07:11:15 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating org.apache.beam.sdk.transforms.MapElements$1@4c4d8e23
20/05/26 07:11:15 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Entering directly-translatable composite transform:
'WordCount.CountWords/Count.PerElement/Combine.perKey(Count)'
20/05/26 07:11:15 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating Combine.perKey(Count)
20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating org.apache.beam.sdk.transforms.MapElements$1@6bfdc26a
20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating Window.Assign
20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating org.apache.beam.sdk.io.WriteFiles$WriteUnshardedTempFilesFn@57c4b142
20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating GroupByKey
20/05/26 07:11:19 WARN
org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions:
Either coder ShardedKeyCoder(VarIntCoder) or GlobalWindow$Coder is not
consistent with equals. That might cause issues on some runners.
20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn@3122d00d
20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating
org.apache.beam.sdk.io.WriteFiles$WriteUnshardedBundlesToTempFiles$1@7639f2af
20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating Flatten.PCollections
20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating
org.apache.beam.sdk.transforms.View$VoidKeyToMultimapMaterialization$VoidKeyToMultimapMaterializationDoFn@bcb712b
20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator:
Evaluating View.CreatePCollectionView
org.apache.beam.examples.WordCountIT > testE2EWordCount FAILED
java.lang.IllegalArgumentException: Unsupported class file major version 55
1 test completed, 1 failed
Finished generating test XML results (0.002 secs) into:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java11_Phrase/src/examples/java/build/test-results/sparkRunnerPreCommit
Generating HTML test report...
Finished generating test html results (0.003 secs) into:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java11_Phrase/src/examples/java/build/reports/tests/sparkRunnerPreCommit
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
