you're right that Flink currently does not support a graceful stop
mechanism for the Kafka source. The community has already a good idea how
to solve it in the general case and will hopefully soon add it to Flink.
Concerning the StoppableFunction: This interface was introduced quite some
time ago and currently only works for some batch sources. In order to make
it work with streaming, we need to add some more functionality to the
engine in order to properly stop and take a savepoint.
On Mon, Feb 19, 2018 at 3:36 PM, Bart Kastermans <fl...@kasterma.net> wrote:
> In https://ci.apache.org/projects/flink/flink-docs-
> release-1.4/ops/cli.html it is shown that
> for gracefully stopping a job you need to implement the StoppableFunction
> interface. This
> appears not (yet) implemented for Kafka consumers. Am I missing
> something, or is there a
> different way to gracefully stop a job using a kafka source so we can
> restart it later without
> losing any (in flight) events?
> - bart