The flag is needed when checkpointing is enabled because Flink is unable to create a new checkpoint when not all operators are running.
By default, operators shut down when all input has been read. That will trigger sending out the maximum (final) watermark at the sources. The flag name is a bit confusing in this regard because shutting down the sources triggers sending out the watermark, not the other way around. -Max On 22.04.20 06:26, Kenneth Knowles wrote: > We should always want to shut down sources on final watermark. All > incoming data should be dropped anyhow. > > Kenn > > On Tue, Apr 21, 2020 at 1:34 PM Luke Cwik <lc...@google.com > <mailto:lc...@google.com>> wrote: > > +dev > > When would we not want --shutdownSourcesOnFinalWatermark=true ? > > On Tue, Apr 21, 2020 at 1:22 PM Ismaël Mejía <ieme...@gmail.com > <mailto:ieme...@gmail.com>> wrote: > > You need to instruct the Flink runner to shutdown the the source > otherwise it will stay waiting. > You can this by adding the extra > argument`--shutdownSourcesOnFinalWatermark=true` > And if that works and you want to open a PR to update our > documentation that would be greatly appreciated. > > Regards, > Ismaël > > > On Tue, Apr 21, 2020 at 10:04 PM Sruthi Sree Kumar > <sruthisreekumar2...@gmail.com > <mailto:sruthisreekumar2...@gmail.com>> wrote: > > > > Hello, > > > > I am trying to run nexmark queries using flink runner > streaming. Followed the documentation and used the command > > ./gradlew :sdks:java:testing:nexmark:run \ > > > > -Pnexmark.runner=":runners:flink:1.10" \ > > -Pnexmark.args=" > > --runner=FlinkRunner > > --suite=SMOKE > > --streamTimeout=60 > > --streaming=true > > --manageResources=false > > --monitorJobs=true > > --flinkMaster=[local]" > > > > > > But after the events are read from the source, there is no > further progress and the job is always stuck at 99%. Is there > any configuration that I am missing? > > > > Regards, > > Sruthi >