RE: Basic questions about resuming stateful Flink jobs

2022-02-17 Thread Schwalbe Matthias
oj...@apache.org>> Sent: 16 February 2022 16:36 To: James Sandys-Lumsdaine mailto:jas...@hotmail.com>> Cc: user@flink.apache.org<mailto:user@flink.apache.org> Subject: Re: Basic questions about resuming stateful Flink jobs CAUTION: External email. The email originated outside o

Re: Basic questions about resuming stateful Flink jobs

2022-02-17 Thread Piotr Nowojski
gt;> >> >> From what I’ve read there is state, checkpoints and save points – all of >> them hold state - and currently I can’t get any of these to restore when >> developing in an IDE and the program builds up all state from scratch. So >> what else do I need to do in

Re: Basic questions about resuming stateful Flink jobs

2022-02-16 Thread Cristian Constantinescu
currently I can’t get any of these to restore when > developing in an IDE and the program builds up all state from scratch. So > what else do I need to do in my Java code to tell Flink to load a savepoint? > > > > Thanks, > > > > James. > > > > > > *From:

RE: Basic questions about resuming stateful Flink jobs

2022-02-16 Thread Sandys-Lumsdaine, James
s up all state from scratch. So what else do I need to do in my Java code to tell Flink to load a savepoint? Thanks, James. From: Piotr Nowojski Sent: 16 February 2022 16:36 To: James Sandys-Lumsdaine Cc: user@flink.apache.org Subject: Re: Basic questions about resuming stateful Flink jobs CAU

Re: Basic questions about resuming stateful Flink jobs

2022-02-16 Thread Piotr Nowojski
Hi James, Sure! The basic idea of checkpoints is that they are fully owned by the running job and used for failure recovery. Thus by default if you stopped the job, checkpoints are being removed. If you want to stop a job and then later resume working from the same point that it has previously

Basic questions about resuming stateful Flink jobs

2022-02-16 Thread James Sandys-Lumsdaine
Hi all, I have a 1.14 Flink streaming workflow with many stateful functions that has a FsStateBackend and checkpointed enabled, although I haven't set a location for the checkpointed state. I've really struggled to understand how I can stop my Flink job and restart it and ensure it carries