[
https://issues.apache.org/jira/browse/BEAM-5528?focusedWorklogId=151438&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151438
]
ASF GitHub Bot logged work on BEAM-5528:
----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Oct/18 00:03
Start Date: 05/Oct/18 00:03
Worklog Time Spent: 10m
Work Description: tweise closed pull request #6523: [BEAM-5528] Re-add
CREATE_VIEW in portable Flink translators
URL: https://github.com/apache/beam/pull/6523
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkBatchPortablePipelineTranslator.java
b/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkBatchPortablePipelineTranslator.java
index bb3a8903100..e67020d42f0 100644
---
a/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkBatchPortablePipelineTranslator.java
+++
b/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkBatchPortablePipelineTranslator.java
@@ -140,6 +140,13 @@ public static FlinkBatchPortablePipelineTranslator
createTranslator() {
translatorMap.put(
PTransformTranslation.RESHUFFLE_URN,
FlinkBatchPortablePipelineTranslator::translateReshuffle);
+ translatorMap.put(
+ PTransformTranslation.CREATE_VIEW_TRANSFORM_URN,
+ // https://issues.apache.org/jira/browse/BEAM-5649
+ // Need to support this via a NOOP until the primitive is removed
+ (PTransformNode transform,
+ RunnerApi.Pipeline pipeline,
+ BatchTranslationContext context) -> {});
return new FlinkBatchPortablePipelineTranslator(translatorMap.build());
}
diff --git
a/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkStreamingPortablePipelineTranslator.java
b/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkStreamingPortablePipelineTranslator.java
index 3f4f7c8e34c..9fe2b2b2de2 100644
---
a/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkStreamingPortablePipelineTranslator.java
+++
b/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkStreamingPortablePipelineTranslator.java
@@ -169,6 +169,12 @@ public StreamExecutionEnvironment
getExecutionEnvironment() {
translatorMap.put(ExecutableStage.URN, this::translateExecutableStage);
translatorMap.put(PTransformTranslation.RESHUFFLE_URN,
this::translateReshuffle);
+ translatorMap.put(
+ // https://issues.apache.org/jira/browse/BEAM-5649
+ // Need to support this via a NOOP until the primitive is removed
+ PTransformTranslation.CREATE_VIEW_TRANSFORM_URN,
+ (String id, RunnerApi.Pipeline pipeline, StreamingTranslationContext
context) -> {});
+
this.urnToTransformTranslator = translatorMap.build();
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 151438)
Time Spent: 2h 10m (was: 2h)
> Java PortableRunner pipeline fails on FlinkRunner due to CREATE_VIEW
> --------------------------------------------------------------------
>
> Key: BEAM-5528
> URL: https://issues.apache.org/jira/browse/BEAM-5528
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Critical
> Labels: portability-flink
> Fix For: 2.8.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> We recently removed the CREATE_VIEW translation which was a NOOP. Turns out
> we still need to support it until it is removed.
> {noformat}
> java.lang.IllegalArgumentException: Unknown type of URN
> beam:transform:create_view:v1 for PTransform with id
> TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/View.CreatePCollectionView.
> at
> org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator.urnNotFound(FlinkBatchPortablePipelineTranslator.java:578)
> at
> org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator.translate(FlinkBatchPortablePipelineTranslator.java:233)
> at
> org.apache.beam.runners.flink.FlinkJobInvocation.runPipeline(FlinkJobInvocation.java:112)
> at
> org.apache.beam.repackaged.beam_runners_flink_2.11.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)
> at
> org.apache.beam.repackaged.beam_runners_flink_2.11.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58)
> at
> org.apache.beam.repackaged.beam_runners_flink_2.11.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
