Lukasz Gajowy created BEAM-5164:
-----------------------------------
Summary: ParquetIOIT fails on Spark and Flink
Key: BEAM-5164
URL: https://issues.apache.org/jira/browse/BEAM-5164
Project: Beam
Issue Type: Bug
Components: testing
Reporter: Lukasz Gajowy
When run on Spark or Flink remote cluster, ParquetIOIT fails with the following
stacktrace:
{code:java}
org.apache.beam.sdk.io.parquet.ParquetIOIT > writeThenReadAll FAILED
org.apache.beam.sdk.Pipeline$PipelineExecutionException:
java.lang.NoSuchMethodError:
org.apache.parquet.hadoop.ParquetWriter$Builder.<init>(Lorg/apache/parquet/io/OutputFile;)V
at
org.apache.beam.runners.spark.SparkPipelineResult.beamExceptionFrom(SparkPipelineResult.java:66)
at
org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:99)
at
org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:87)
at org.apache.beam.runners.spark.TestSparkRunner.run(TestSparkRunner.java:116)
at org.apache.beam.runners.spark.TestSparkRunner.run(TestSparkRunner.java:61)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:350)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:331)
at
org.apache.beam.sdk.io.parquet.ParquetIOIT.writeThenReadAll(ParquetIOIT.java:133)
Caused by:
java.lang.NoSuchMethodError:
org.apache.parquet.hadoop.ParquetWriter$Builder.<init>(Lorg/apache/parquet/io/OutputFile;)V{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
