[
https://issues.apache.org/jira/browse/BEAM-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Cwik updated BEAM-8989:
----------------------------
Description:
[PR/9275|https://github.com/apache/beam/pull/9275] changed
*ParDo.getSideInputs* from *List<PCollectionView>* to *Map<String,
PCollectionView>* which is backwards incompatible change and was released as
part of Beam 2.16.0 erroneously.
Running the Apache Nemo Quickstart fails with:
{code:java}
Exception in thread "main" java.lang.RuntimeException: Translator private
static void
org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput)
have failed to translate
org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27Exception in thread
"main" java.lang.RuntimeException: Translator private static void
org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput)
have failed to translate
org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27 at
org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:113)
at
org.apache.nemo.compiler.frontend.beam.PipelineVisitor.visitPrimitiveTransform(PipelineVisitor.java:46)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:665)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:317)
at
org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:251)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:460) at
org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:80) at
org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:31) at
org.apache.beam.sdk.Pipeline.run(Pipeline.java:315) at
org.apache.beam.sdk.Pipeline.run(Pipeline.java:301) at
org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185) at
org.apache.beam.examples.WordCount.main(WordCount.java:192)Caused by:
java.lang.reflect.InvocationTargetException at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:109)
... 14 moreCaused by: java.lang.NoSuchMethodError:
org.apache.beam.sdk.transforms.ParDo$MultiOutput.getSideInputs()Ljava/util/List;
at
org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(PipelineTranslator.java:236)
... 19 more{code}
was:
This changed `ParDo.getSideInputs` from `List<PCollectionView>` to `Map<String,
PCollectionView>` which is backwards incompatible change and was released as
part of Beam 2.16.0 erroneously.
```
Exception in thread "main" java.lang.RuntimeException: Translator private
static void
org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput)
have failed to translate
org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27Exception in thread
"main" java.lang.RuntimeException: Translator private static void
org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput)
have failed to translate
org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27 at
org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:113)
at
org.apache.nemo.compiler.frontend.beam.PipelineVisitor.visitPrimitiveTransform(PipelineVisitor.java:46)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:665)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:317)
at
org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:251)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:460) at
org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:80) at
org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:31) at
org.apache.beam.sdk.Pipeline.run(Pipeline.java:315) at
org.apache.beam.sdk.Pipeline.run(Pipeline.java:301) at
org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185) at
org.apache.beam.examples.WordCount.main(WordCount.java:192)Caused by:
java.lang.reflect.InvocationTargetException at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:109)
... 14 moreCaused by: java.lang.NoSuchMethodError:
org.apache.beam.sdk.transforms.ParDo$MultiOutput.getSideInputs()Ljava/util/List;
at
org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(PipelineTranslator.java:236)
... 19 more
```
> Apache Nemo quickstart broken for Apache Beam 2.16 and 2.17 due to backwards
> incompatible change in ParDo.getSideInputs
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-8989
> URL: https://issues.apache.org/jira/browse/BEAM-8989
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: 2.16.0, 2.17.0, 2.18.0
> Reporter: Luke Cwik
> Assignee: Mikhail Gryzykhin
> Priority: Blocker
>
> [PR/9275|https://github.com/apache/beam/pull/9275] changed
> *ParDo.getSideInputs* from *List<PCollectionView>* to *Map<String,
> PCollectionView>* which is backwards incompatible change and was released as
> part of Beam 2.16.0 erroneously.
> Running the Apache Nemo Quickstart fails with:
>
> {code:java}
> Exception in thread "main" java.lang.RuntimeException: Translator private
> static void
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput)
> have failed to translate
> org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27Exception in thread
> "main" java.lang.RuntimeException: Translator private static void
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput)
> have failed to translate
> org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27 at
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:113)
> at
> org.apache.nemo.compiler.frontend.beam.PipelineVisitor.visitPrimitiveTransform(PipelineVisitor.java:46)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:665)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:317)
> at
> org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:251)
> at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:460) at
> org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:80) at
> org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:31) at
> org.apache.beam.sdk.Pipeline.run(Pipeline.java:315) at
> org.apache.beam.sdk.Pipeline.run(Pipeline.java:301) at
> org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185) at
> org.apache.beam.examples.WordCount.main(WordCount.java:192)Caused by:
> java.lang.reflect.InvocationTargetException at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:109)
> ... 14 moreCaused by: java.lang.NoSuchMethodError:
> org.apache.beam.sdk.transforms.ParDo$MultiOutput.getSideInputs()Ljava/util/List;
> at
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(PipelineTranslator.java:236)
> ... 19 more{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
