Re: Flink 1.7.2 + Beam 2.11 error: The transform beam:transform:create_view:v1 is currently not supported.

Great. Here is the JIRA issue including a PR which fixes the problem and adds additional tests to prevent this in the future: https://issues.apache.org/jira/browse/BEAM-6937 Thanks, Max On 29.03.19 15:43, Kaymak, Tobias wrote: Can confirm that this is the issue, starting with streaming=True f

Re: Imposing parallelism

Hi Max, Again, thanks for your answer. Is there anyone that can point me to some example or documentation on how to develop your own reader? Is this (https://beam.apache.org/documentation/io/developing-io-java/) the best reference to look at? Best regards, Augusto On 2019/03/29 14:23:42, Ma

Re: Flink 1.7.2 + Beam 2.11 error: The transform beam:transform:create_view:v1 is currently not supported.

Can confirm that this is the issue, starting with streaming=True fixes it. On Fri, Mar 29, 2019 at 11:53 AM Maximilian Michels wrote: > Hi Tobias, > > Thank for reporting. Can confirm, this is a regression with the > detection of the execution mode. Everything should work fine if you set > the "

Re: Imposing parallelism

Hi Augusto, In Beam there is no way to specify how parallel a specific transform should be. There is only a general indicator for how parallel a pipeline should be, i.e. Dataflow has "numWorkers", Spark/Flink have "parallelism". You should see 16 parallel operations for your Read if you confi

Re: Flink 1.7.2 + Beam 2.11 error: The transform beam:transform:create_view:v1 is currently not supported.

Hi Tobias, Thank for reporting. Can confirm, this is a regression with the detection of the execution mode. Everything should work fine if you set the "streaming" flag to true. Will be fixed for the 2.12.0 release. Thanks, Max On 28.03.19 17:28, Lukasz Cwik wrote: +dev