PR for the update: https://github.com/apache/beam/pull/11751
Regards,
Sruthi
On Tue, May 19, 2020 at 3:51 PM Maximilian Michels wrote:
> Looks like an accidental change to me. Running with either version, 1.9
> or 1.10 works, but this should be changed back to using the latest version.
>
> Do
On the documentation, the version of Flink runner is changed to 1.9 which
was 1.10(latest) before
https://github.com/apache/beam/commit/1d2700818474c008eaa324ac1b5c49c9d2857298#diff-0e75160f4b09a1a300671557930589d9
.
Is this an accidental change or is there any particular reason for this
Looks like an accidental change to me. Running with either version, 1.9
or 1.10 works, but this should be changed back to using the latest version.
Do you mind creating a PR?
Thanks,
Max
On 19.05.20 13:02, Sruthi Sree Kumar wrote:
> On the documentation, the version of Flink runner is changed
A heads-up if anybody else sees this, we have removed the flag:
https://jira.apache.org/jira/browse/BEAM-9900
Further contributions are very welcome :)
-Max
On 11.05.20 17:05, Sruthi Sree Kumar wrote:
> I have opened a PR with the documentation change.
>
The flag is needed when checkpointing is enabled because Flink is unable
to create a new checkpoint when not all operators are running.
By default, operators shut down when all input has been read. That will
trigger sending out the maximum (final) watermark at the sources. The
flag name is a bit
We should always want to shut down sources on final watermark. All incoming
data should be dropped anyhow.
Kenn
On Tue, Apr 21, 2020 at 1:34 PM Luke Cwik wrote:
> +dev
>
> When would we not want --shutdownSourcesOnFinalWatermark=true ?
>
> On Tue, Apr 21, 2020 at 1:22 PM Ismaël Mejía wrote:
>
+dev
When would we not want --shutdownSourcesOnFinalWatermark=true ?
On Tue, Apr 21, 2020 at 1:22 PM Ismaël Mejía wrote:
> You need to instruct the Flink runner to shutdown the the source
> otherwise it will stay waiting.
> You can this by adding the extra
>
