[
https://issues.apache.org/jira/browse/FLINK-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16216917#comment-16216917
]
Antoine Philippot commented on FLINK-7883:
------------------------------------------
Glad to see there are ideas and interest about this behaviour.
I personnaly think that there are too many "close/shutdown/cancel/stop"
commands / methods.
There are {{cancel}} and {{stop}} commands for a job (it should only exist one
clean shutdown command)
And there are {{close}} on {{RichFunction}}, {{cancel}} on {{SourceFunction}}
and also {{stop}} on {{StoppableFunction}}.
2 steps clean function shutdown should be available on both consumer and
producer.
Why not reuse {{RichFunction}} and {{AbstractRichFunction}} which already have
a {{close}} function
and add a new method {{prepareShutdown}} (I don't know how to name it) which
return a Future
which replace {{SourceFunction.cancel}} and {{StoppableFunction.stop}}
When a job shutdown is triggered, we could first call all {{prepareShutdown}}
methods and wait completion
with configurable timeout. Only after, we could trigger a savepoint if
requested and finally call all {{close}} methods.
> Stop fetching source before a cancel with savepoint
> ---------------------------------------------------
>
> Key: FLINK-7883
> URL: https://issues.apache.org/jira/browse/FLINK-7883
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API, Kafka Connector, State Backends,
> Checkpointing
> Affects Versions: 1.4.0, 1.3.2
> Reporter: Antoine Philippot
>
> For a cancel with savepoint command, the JobManager trigger the cancel call
> once the savepoint is finished, but during the savepoint execution, kafka
> source continue to poll new messages which will not be part of the savepoint
> and will be replayed on the next application start.
> A solution could be to stop fetching the source stream task before triggering
> the savepoint.
> I suggest to add an interface {{StoppableFetchingSourceFunction}} with a
> method {{stopFetching}} that existant SourceFunction implementations could
> implement.
> We can add a {{stopFetchingSource}} property in
> {{CheckpointOptions}} class to pass the desired behaviour from
> {{JobManager.handleMessage(CancelJobWithSavepoint)}} to
> {{SourceStreamTask.triggerCheckpoint}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
