Hi Milind,
A job can be stopped with a savepoint in the following way [1]: ./bin/flink
stop --savepointPath [:targetDirectory] :jobId
Best,
Matthias
[1]
https://ci.apache.org/projects/flink/flink-docs-master/docs/ops/state/savepoints/#stopping-a-job-with-savepoint
On Sun, May 16, 2021 at 1:12
Sounds good.
How do I achieve stop with savepoint ?
- Milind
On Mon, Apr 26, 2021 at 12:55 AM Matthias Pohl
wrote:
> I'm not sure what you're trying to achieve. Are you trying to simulate a
> task failure? Or are you trying to pick up the state from a stopped job?
> You could achieve the
I'm not sure what you're trying to achieve. Are you trying to simulate a
task failure? Or are you trying to pick up the state from a stopped job?
You could achieve the former one by killing the TaskManager instance or by
throwing a custom failure as part of your job pipeline. The latter one can
be
Hi Matthias,
Yeah you are right. I am canceling the job and hence it is creating new job
with new job id and hence it is no respecting previous checkpoint. I
observed same behaviour even for local FS backend.
Is there any way to simulated failing of job locally ?
As far as config is concerned,
One additional question: How did you stop and restart the job? The behavior
you're expecting should work with stop-with-savepoint. Cancelling the job
and then just restarting it wouldn't work. The latter approach would lead
to a new job being created.
Best,
Matthias
On Thu, Apr 22, 2021 at 3:12
Hi Milind,
I bet someone else might have a faster answer. But could you provide the
logs and config to get a better understanding of what your issue is?
In general, the state is maintained even in cases where a TaskManager fails.
Best,
Matthias
On Thu, Apr 22, 2021 at 5:11 AM Milind Vaidya
Hi
I see MemoryStateBackend being used in TM Log
org.apache.flink.streaming.runtime.tasks.StreamTask - No state backend has
been configured, using default (Memory / JobManager) MemoryStateBackend
(data in heap memory / checkpoints to JobManager) (checkpoints: 'null',
savepoints: 'null',
