Re: [DISCUSS] FLIP-201: Persist local state in working directory

2022-01-10 Thread Till Rohrmann
gt; environment? > > > Best, > Yun Tang > > From: David Morávek > Sent: Wednesday, January 5, 2022 19:51 > To: dev > Subject: Re: [DISCUSS] FLIP-201: Persist local state in working directory > > +1 the general direction here see

Re: [DISCUSS] FLIP-201: Persist local state in working directory

2022-01-10 Thread Yun Tang
: [DISCUSS] FLIP-201: Persist local state in working directory +1 the general direction here seems pretty solid D. On Wed, Jan 5, 2022 at 11:57 AM Till Rohrmann wrote: > If there is no other larger feedback, I would start the vote soonish. > > Cheers, > Till > > On Thu, Dec 3

Re: [DISCUSS] FLIP-201: Persist local state in working directory

2022-01-05 Thread David Morávek
+1 the general direction here seems pretty solid D. On Wed, Jan 5, 2022 at 11:57 AM Till Rohrmann wrote: > If there is no other larger feedback, I would start the vote soonish. > > Cheers, > Till > > On Thu, Dec 30, 2021 at 4:28 PM Till Rohrmann > wrote: > > > Hi David, > > > > Thanks for

Re: [DISCUSS] FLIP-201: Persist local state in working directory

2022-01-05 Thread Till Rohrmann
If there is no other larger feedback, I would start the vote soonish. Cheers, Till On Thu, Dec 30, 2021 at 4:28 PM Till Rohrmann wrote: > Hi David, > > Thanks for your feedback. > > With the graceful shutdown I mean a way to stop the TaskManager and to > clean up the working directory. At the

Re: [DISCUSS] FLIP-201: Persist local state in working directory

2021-12-30 Thread Till Rohrmann
Hi David, Thanks for your feedback. With the graceful shutdown I mean a way to stop the TaskManager and to clean up the working directory. At the moment, I think we always kill the process via SIGTERM or SIGKILL. This won't clean up the working directory because it could also originate from a

Re: [DISCUSS] FLIP-201: Persist local state in working directory

2021-12-30 Thread David Morávek
Hi Till, thanks for drafting the FLIP, it looks really good. I did a quick pass over the PR and it seems to be heading in a right direction. It might be required to introduce a graceful shutdown of the TaskExecutor > in order to support proper cleanup of resources. > This is actively being

Re: [DISCUSS] FLIP-201: Persist local state in working directory

2021-12-29 Thread Till Rohrmann
I've created draft PR for the desired changes [1]. It might be easier to take a look at than the branch. [1] https://github.com/apache/flink/pull/18237 Cheers, Till On Tue, Dec 28, 2021 at 3:22 PM Till Rohrmann wrote: > Hi everyone, > > I would like to start a discussion about using the

[DISCUSS] FLIP-201: Persist local state in working directory

2021-12-28 Thread Till Rohrmann
Hi everyone, I would like to start a discussion about using the working directory to persist local state for faster recovery (FLIP-201) [1]. Persisting the local state will be beneficial if a crashed process is restarted with the same working directory. In this case, Flink does not have to