[
https://issues.apache.org/jira/browse/BEAM-5528?focusedWorklogId=151118&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151118
]
ASF GitHub Bot logged work on BEAM-5528:
----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Oct/18 10:26
Start Date: 04/Oct/18 10:26
Worklog Time Spent: 10m
Work Description: mxm commented on issue #6523: [BEAM-5528] Re-add
CREATE_VIEW in portable Flink translators
URL: https://github.com/apache/beam/pull/6523#issuecomment-426966532
Removing this requires other changes to the pipeline creation/fusioning,
DirectRunner, non-portable FlinkRunner, ApexRunner, SamzaRunner. Definitely
doable but we should fix the pending issue first, and then move towards
removing CREATE_VIEW.
> I would prefer we fix this in the Java SDK as otherwise it will leak into
every runner.
It's already leaked into every Runner since it was originally part of the
pipeline translation.
I'm for supporting this with a NOOP. It's very little maintenance cost for
the portable FlinkRunner and fixes a critical issue which prevents pipeline
execution.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 151118)
Time Spent: 1h (was: 50m)
> Java PortableRunner pipeline fails on FlinkRunner due to CREATE_VIEW
> --------------------------------------------------------------------
>
> Key: BEAM-5528
> URL: https://issues.apache.org/jira/browse/BEAM-5528
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Major
> Labels: portability-flink
> Fix For: 2.8.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> We recently removed the CREATE_VIEW translation which was a NOOP. Turns out
> we still need to support it until it is removed.
> {noformat}
> java.lang.IllegalArgumentException: Unknown type of URN
> beam:transform:create_view:v1 for PTransform with id
> TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/View.CreatePCollectionView.
> at
> org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator.urnNotFound(FlinkBatchPortablePipelineTranslator.java:578)
> at
> org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator.translate(FlinkBatchPortablePipelineTranslator.java:233)
> at
> org.apache.beam.runners.flink.FlinkJobInvocation.runPipeline(FlinkJobInvocation.java:112)
> at
> org.apache.beam.repackaged.beam_runners_flink_2.11.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)
> at
> org.apache.beam.repackaged.beam_runners_flink_2.11.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58)
> at
> org.apache.beam.repackaged.beam_runners_flink_2.11.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
