Re: Task Local Recovery with mountable disks in the cloud

2021-05-10 Thread Yang Wang
onday, May 10, 2021 2:26 AM > *To:* Sonam Mandal > *Cc:* dev ; user@flink.apache.org < > user@flink.apache.org> > *Subject:* Re: Task Local Recovery with mountable disks in the cloud > > Hi Sonam, > > I think it would be great to create a FLIP for this feature. FLIPs

Re: Task Local Recovery with mountable disks in the cloud

2021-05-10 Thread Sonam Mandal
org>>; Sonam Mandal mailto:soman...@linkedin.com>> Subject: Re: Task Local Recovery with mountable disks in the cloud Hi Sonam, sorry for the late reply. We were a bit caught in the midst of the feature freeze for the next major Flink release. In general, I think it is a very good idea

Re: Task Local Recovery with mountable disks in the cloud

2021-05-10 Thread Till Rohrmann
t; ticket do? > > Thanks, > Sonam > -- > *From:* Till Rohrmann > *Sent:* Monday, April 26, 2021 10:24 AM > *To:* dev > *Cc:* user@flink.apache.org ; Sonam Mandal < > soman...@linkedin.com> > *Subject:* Re: Task Local Recovery with mou

Re: Task Local Recovery with mountable disks in the cloud

2021-05-07 Thread Sonam Mandal
0:24 AM To: dev Cc: user@flink.apache.org ; Sonam Mandal Subject: Re: Task Local Recovery with mountable disks in the cloud Hi Sonam, sorry for the late reply. We were a bit caught in the midst of the feature freeze for the next major Flink release. In general, I think it is a very good idea

Re: Task Local Recovery with mountable disks in the cloud

2021-04-26 Thread Till Rohrmann
Hi Sonam, sorry for the late reply. We were a bit caught in the midst of the feature freeze for the next major Flink release. In general, I think it is a very good idea to disaggregate the local state storage to make it reusable across TaskManager failures. However, it is also not trivial to do.

Re: Task Local Recovery with mountable disks in the cloud

2021-04-21 Thread Stephan Ewen
/cc dev@flink On Tue, Apr 20, 2021 at 1:29 AM Sonam Mandal wrote: > Hello, > > We've been experimenting with Task-local recovery using Kubernetes. We > have a way to specify mounting the same disk across Task Manager > restarts/deletions for when the pods get recreated. In this scenario, we > n

Task Local Recovery with mountable disks in the cloud

2021-04-19 Thread Sonam Mandal
Hello, We've been experimenting with Task-local recovery using Kubernetes. We have a way to specify mounting the same disk across Task Manager restarts/deletions for when the pods get recreated. In this scenario, we noticed that task local recovery does not kick in (as expected based on the doc