Github user tillrohrmann commented on the pull request:
https://github.com/apache/flink/pull/750#issuecomment-180617867
Ok let us include as a first version a stop implementation where you don't
have the strong consistency guarantees. But before I think we should still
change three things.
The first thing I would like to change is the `JobType`. I feel a bit
uneasy adding explicit information whether a job is a streaming or a batch job
to the `JobGraph`. So far we've succeeded in keeping the runtime agnostic to
the type of job. I would propose to change it to a `stoppable` flag. Thus,
indicating whether the job is stoppable or not. Semantically, it should not
make a big difference.
The second thing I would propose is to remove `stop` from the
`StreamSource` class in order to make it not mandatory for all streaming
sources to be stoppable. Instead I would also introduce a `StoppableOperator`
interface which can be implemented by streaming sources which are in fact
stoppable. I'm not so sure, for example, whether the Kafka sources are properly
stoppable without throwing exceptions. By introducing this interface we can
selectively implement stop methods for sources. A nice side effect is that
existing code won't break if you have a custom source implementation.
And as a last point I would propose to automatically infer whether a job is
stoppable or not. What we have to do is to check that all source functions
implement the `StoppableOperator` interface. This should be doable in the
`StreamGraphGenerator`. Based on this information we can then set the stoppable
flag in the `JobGraph` or not.
What do you think? Would that be feasible?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
