[
https://issues.apache.org/jira/browse/BEAM-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046625#comment-17046625
]
Won Wook Song edited comment on BEAM-8989 at 2/27/20 1:35 PM:
--------------------------------------------------------------
Luckily, I have happened to notice the change a few weeks ago, and have
reflected the necessary changes on the Nemo master branch a few weeks ago
already :) ([link|https://github.com/apache/incubator-nemo/pull/281]). It will
be reflected on the upcoming Nemo release, and I'll file a PR on the Beam
repository to update the dependency and the Quickstart appropriately to use the
newer release, and the issue would be resolved. I'll assign myself for the
ticket, and get back to you once the new release process has been handled!
was (Author: wonook):
Luckily, I have happened to notice the change a few weeks ago, and have
reflected the necessary changes on the Nemo master branch a few weeks ago
already :) ([link](https://github.com/apache/incubator-nemo/pull/281)). It will
be reflected on the upcoming Nemo release, and I'll file a PR on the Beam
repository to update the dependency and the Quickstart appropriately to use the
newer release, and the issue would be resolved. I'll assign myself for the
ticket, and get back to you once the new release process has been handled!
> Backwards incompatible change in ParDo.getSideInputs (caught by failure when
> running Apache Nemo quickstart)
> ------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-8989
> URL: https://issues.apache.org/jira/browse/BEAM-8989
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
> Reporter: Luke Cwik
> Assignee: Reuven Lax
> Priority: Critical
> Fix For: 2.20.0
>
>
> [PR/9275|https://github.com/apache/beam/pull/9275] changed
> *ParDo.getSideInputs* from *List<PCollectionView>* to *Map<String,
> PCollectionView>* which is backwards incompatible change and was released as
> part of Beam 2.16.0 erroneously.
> Running the Apache Nemo Quickstart fails with:
>
> {code:java}
> Exception in thread "main" java.lang.RuntimeException: Translator private
> static void
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput)
> have failed to translate
> org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27Exception in thread
> "main" java.lang.RuntimeException: Translator private static void
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(org.apache.nemo.compiler.frontend.beam.PipelineTranslationContext,org.apache.beam.sdk.runners.TransformHierarchy$Node,org.apache.beam.sdk.transforms.ParDo$MultiOutput)
> have failed to translate
> org.apache.beam.examples.WordCount$ExtractWordsFn@600b9d27 at
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:113)
> at
> org.apache.nemo.compiler.frontend.beam.PipelineVisitor.visitPrimitiveTransform(PipelineVisitor.java:46)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:665)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:317)
> at
> org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:251)
> at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:460) at
> org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:80) at
> org.apache.nemo.compiler.frontend.beam.NemoRunner.run(NemoRunner.java:31) at
> org.apache.beam.sdk.Pipeline.run(Pipeline.java:315) at
> org.apache.beam.sdk.Pipeline.run(Pipeline.java:301) at
> org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185) at
> org.apache.beam.examples.WordCount.main(WordCount.java:192)Caused by:
> java.lang.reflect.InvocationTargetException at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.translatePrimitive(PipelineTranslator.java:109)
> ... 14 moreCaused by: java.lang.NoSuchMethodError:
> org.apache.beam.sdk.transforms.ParDo$MultiOutput.getSideInputs()Ljava/util/List;
> at
> org.apache.nemo.compiler.frontend.beam.PipelineTranslator.parDoMultiOutputTranslator(PipelineTranslator.java:236)
> ... 19 more{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
