[
https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136892#comment-15136892
]
ASF GitHub Bot commented on FLINK-2111:
---------------------------------------
Github user tillrohrmann commented on the pull request:
https://github.com/apache/flink/pull/750#issuecomment-181350773
I strongly agree with @StephanEwen not to make the distinction between
streaming and batch jobs in the runtime explicit. Even though this might only
be a cosmetic change for this PR but who knows what other people will build on
top of it once it is there. I fear that we won't be able to remove such a flag
once introduced.
Concerning whether all `SourceFunction` should have a `stop` method: I
think it is not a priori clear that all streaming sources are gracefully
stoppable. Take for example the `FlinkKafkaConsumer08`. Currently, the stop
method is implemented by calling `cancel`. However, `cancel` does not guarantee
that the sources stop gracefully. Exceptions might be thrown and the state
might be inconsistent. In fact, with the current implementation, `stop` won't
stop the `FlinkKafkaConsumer08` at all. The `cancel` call only works in
combination with the `TaskCanceler` which interrupts the `Task` thread (in this
case `LegacyFetcher.run`). If the interrupt does not happen, the
`SimpleConsumerThreads` might be stuck for all eternity in the fetch loop.
Furthermore, adding the `stop` method to the `SourceFunction` interface
will be an API breaking change and all users would have to implement `stop`,
henceforth. I think a `StoppableOperator` interface would be more modular and
less restrictive because you have the freedom not to implement a graceful stop
operation.
What do you think?
> Add "stop" signal to cleanly shutdown streaming jobs
> ----------------------------------------------------
>
> Key: FLINK-2111
> URL: https://issues.apache.org/jira/browse/FLINK-2111
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Runtime, JobManager, Local Runtime,
> Streaming, TaskManager, Webfrontend
> Reporter: Matthias J. Sax
> Assignee: Matthias J. Sax
> Priority: Minor
>
> Currently, streaming jobs can only be stopped using "cancel" command, what is
> a "hard" stop with no clean shutdown.
> The new introduced "stop" signal, will only affect streaming source tasks
> such that the sources can stop emitting data and shutdown cleanly, resulting
> in a clean shutdown of the whole streaming job.
> This feature is a pre-requirment for
> https://issues.apache.org/jira/browse/FLINK-1929
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
