AFAIK, the side input is not yet supported in Flink runner in streaming
mode. The Spark runner doesn't support it neither ;)
I think Max is working on it for Flink runner. I will work with Amit on
Spark runner.
Regards
JB
On 05/20/2016 09:22 AM, Ismaël Mejía wrote:
Hello,
Thanks for your answer, I am really happy to know you are working on
removing
this flag for batch/streaming since this is not really Beam-like.
I tried to execute the pipeline again in streaming mode, and I got this
exception now:
java.lang.UnsupportedOperationException: The transform
View.CreatePCollectionView is currently not supported.
Any info about this one ?
Thanks again,
Ismael
On Thu, May 19, 2016 at 7:31 PM, Aljoscha Krettek <[email protected]
<mailto:[email protected]>> wrote:
Hi,
as an addition, there are plans to get rid of the "streaming" flag
for all runners and determine the mode from the job.
-Aljoscha
On Thu, 19 May 2016 at 18:52 Maximilian Michels <[email protected]
<mailto:[email protected]>> wrote:
Hi Ismaël,
There is a pull request which enables Windows on batch. It will be
merged soon. In the meantime, please use the streaming execution
mode
of the Flink Runner.
Use setStreaming(true) to enable it on the FlinkPipelineOptions.
Cheers,
Max
On Thu, May 19, 2016 at 6:32 PM, Ismaël Mejía <[email protected]
<mailto:[email protected]>> wrote:
> I was trying to run a local Pipeline we have (basically a
batch one we
> rewrote from TPC-H) in all the possible runners, for the
moment I had
> success with the DirectPipelineRunner,
InProcessPipelineRunner and
> SparkPipelineRunner. However when I executed it in the
FlinkPipelineRunner I
> got this exception:
>
> java.lang.UnsupportedOperationException: The transform
Window.Into() is
> currently not supported.
>
> I thought it was probably a configuration error since I
remember that in the
> flink runner we had to explicitly define if it was a batch or
streaming job,
> but when I checked the options:
>
> Current Settings:
> appName: TestQ08Task
> filesToStage: ...
> flinkMaster: [auto]
> parallelism: 1
> runner: class org.apache.beam.runners.flink.FlinkPipelineRunner
> stableUniqueNames: WARNING
> streaming: false
>
> I noticed I was in batch mode, is there something missing in
the options
> configuration, or is it that the UnsoppertedOperation is a
WIP or not
> supported at all by Flink ?
>
> And extra question, this option to define if it is in batch
or stream mode
> is going to stay for long, can't be this inferred some how ?
>
> Thanks,
> -Ismaël
>
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
