[
https://issues.apache.org/jira/browse/FLINK-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471807#comment-16471807
]
Sihua Zhou edited comment on FLINK-9043 at 5/11/18 12:07 PM:
-------------------------------------------------------------
Hi, [~StephanEwen] Sorry that I skipped the discussion period and jump into the
implementation...In fact, the expected behavior from me is quite similar as you
mentioned above. When the user try to recover the job from the given
`directory`, we do the follow thing.
- If more than one jobs checkpointed there before, this operation would fail.
- If there is only one job checkpointed there, we enter the directory and find
the last successful checkpoint.
- if the directory is already the job checkpoint directory then we find the
last successful checkpoint directly.
But I didn't introduce a new option, I reuse the {{-s}} option, because I feel
that introduce a new option would make the user to learn more information. What
do you think? Please let me know your expected behavior, thanks!
was (Author: sihuazhou):
Hi, [~StephanEwen] Sorry that I skipped the discussion period and jump into the
implementation...In fact, the expected behavior from me is quite similar as you
metioned. When the user try to recover the job from the given `directory`, we
do the follow thing.
- If more than one jobs checkpointed there before, this operation would fail.
- If there is only one job checkpointed there, we enter the directory and find
the last successful checkpoint.
- if the directory is already the job checkpoint directory then we find the
last successful checkpoint directly.
But I didn't introduce a new option, I reuse the {{-s}} option, because I feel
that introduce a new option would make the user to learn more information. What
do you think? Please let me know your expected behavior, thanks!
> Introduce a friendly way to resume the job from externalized checkpoints
> automatically
> --------------------------------------------------------------------------------------
>
> Key: FLINK-9043
> URL: https://issues.apache.org/jira/browse/FLINK-9043
> Project: Flink
> Issue Type: New Feature
> Reporter: godfrey johnson
> Assignee: Sihua Zhou
> Priority: Major
>
> I know a flink job can reovery from checkpoint with restart strategy, but can
> not recovery as spark streaming jobs when job is starting.
> Every time, the submitted flink job is regarded as a new job, while , in the
> spark streaming job, which can detect the checkpoint directory first, and
> then recovery from the latest succeed one. However, Flink only can recovery
> until the job failed first, then retry with strategy.
>
> So, would flink support to recover from the checkpoint directly in a new job?
> h2. New description by [~sihuazhou]
> Currently, it's quite a bit not friendly for users to recover job from the
> externalized checkpoint, user need to find the dedicate dir for the job which
> is not a easy thing when there are too many jobs. This ticket attend to
> introduce a more friendly way to allow the user to use the externalized
> checkpoint to do recovery.
> The implementation steps are copied from the comments of [~StephanEwen]:
> - We could make this an option where you pass a flag (-r) to automatically
> look for the latest checkpoint in a given directory.
> - If more than one jobs checkpointed there before, this operation would fail.
> - We might also need a way to have jobs not create the UUID subdirectory,
> otherwise the scanning for the latest checkpoint would not easily work.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
