[
https://issues.apache.org/jira/browse/BEAM-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045312#comment-17045312
]
Ismaël Mejía edited comment on BEAM-8989 at 2/26/20 9:52 AM:
-------------------------------------------------------------
And back to this issue I think we should fix this as part of 2.20.0, this is a
backwards incompatibility issue not addressed (we are missing this situation in
the Jira priorities document but we should probably make backwards incompatible
issues in core to be P1 and release blocking). The fact that nobody is
complaining and we have missed it in previous releases maybe shows that this
method was rarely used, so one alternative is to fix it, or otherwise we could
mark this backwards incompatible change in the release notes. We have no reason
to delay this anymore.
was (Author: iemejia):
And back to this issue I think we should fix this as part of 2.20.0, this is a
backwards incompatibility issue not addressed (we are missing this situation in
the Jira priorities document but we should probably make backwards incompatible
issues in core to be P1 and release blocking). The fact that nobody is
complaining and we have missed it in previous releases maybe is an indicative
that this was not much used, so one alternative could be to mark this backwards
incompatible change in the release notes otherwise we should fix it. We have no
reason to delay this anymore.
> Backwards incompatible change in ParDo.getSideInputs (caught by failure when
> running Apache Nemo quickstart)
> ------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-8989
> URL: https://issues.apache.org/jira/browse/BEAM-8989
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
> Reporter: Luke Cwik
> Assignee: Reuven Lax
> Priority: Critical
> Fix For: 2.20.0
>
>
> [PR/9275|https://github.com/apache/beam/pull/9275] changed
> *ParDo.getSideInputs* from *List<PCollectionView>* to *Map<String,
> PCollectionView>* which is backwards incompatible change and was released as
> part of Beam 2.16.0 erroneously.
> Running the Apache Nemo Quickstart fails with:
>
> {code:java}
> Exception in thread "main" java.lang.RuntimeException: Translator private
> static void
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput)
> have failed to translate
> org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27Exception in thread
> "main" java.lang.RuntimeException: Translator private static void
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput)
> have failed to translate
> org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27 at
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:113)
> at
> org.apache.nemo.compiler.frontend.beam.PipelineVisitor.visitPrimitiveTransform(PipelineVisitor.java:46)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:665)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:317)
> at
> org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:251)
> at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:460) at
> org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:80) at
> org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:31) at
> org.apache.beam.sdk.Pipeline.run(Pipeline.java:315) at
> org.apache.beam.sdk.Pipeline.run(Pipeline.java:301) at
> org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185) at
> org.apache.beam.examples.WordCount.main(WordCount.java:192)Caused by:
> java.lang.reflect.InvocationTargetException at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:109)
> ... 14 moreCaused by: java.lang.NoSuchMethodError:
> org.apache.beam.sdk.transforms.ParDo$MultiOutput.getSideInputs()Ljava/util/List;
> at
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(PipelineTranslator.java:236)
> ... 19 more{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
