+1 (non-binding) Regarding Yu's suggestion about *Roadmap* or *Future Work* section, I think it's a good idea. Currently, some MVP limitations are mentioned at the end of the document, so we can extract and expand it. As for the recovery speed it's not a priority currently, but we could also mention it in this section.
On Wed, Mar 11, 2020 at 4:11 PM Zhijiang <wangzhijiang...@aliyun.com.invalid> wrote: > +1 (binding). > > As for David's concern of smaller buffers after recovery, I ever had a > draft design [1] to solve this issue. > You can take a look and leave comments if still have concerns. :) > > [1] > https://docs.google.com/document/d/16_MOQymzxrKvUHXh6QFr2AAXIKt_2vPUf8vzKy4H_tU/edit > > Best, > Zhijiang > > > ------------------------------------------------------------------ > From:Piotr Nowojski <pi...@ververica.com> > Send Time:2020 Mar. 11 (Wed.) 21:19 > To:dev <dev@flink.apache.org> > Subject:Re: [VOTE] [FLIP-76] Unaligned checkpoints > > +1 (binding). > > Piotrek > > > On 11 Mar 2020, at 09:19, David Anderson <da...@ververica.com> wrote: > > > > +1 I like where this is headed. > > > > One question: during restore, it could happen that a new task manager is > > configured with fewer or smaller buffers than was previously the case. > How > > will this be handled? > > > > David > > > > > > On Wed, Mar 11, 2020 at 8:31 AM Arvid Heise <ar...@ververica.com> wrote: > > > >> Hi Thomas, > >> > >> it's like you said. The first version will not support rescaling and > mostly > >> addresses the concerns about making little to no progress because of > >> frequent crashes. > >> > >> The main reason is that we cannot guarantee the ordering of non-keyed > data > >> (and even keyed data in some weird cases) when rescaling currently. We > have > >> a general concept to address that, which would also enable dynamic > >> rescaling in the future, but that would make the changes even bigger > and we > >> would not have any version ready for 1.11. > >> > >> The current plan, of course, is to continue improving unaligned > checkpoints > >> immediately after release, such that we have the full feature set for > 1.12. > >> Potentially, unaligned checkpoints (with timeouts) would even become the > >> default option. > >> > >> On Tue, Mar 10, 2020 at 11:14 PM Thomas Weise <t...@apache.org> wrote: > >> > >>> +1 > >>> > >>> Thanks for putting this together, looking forward to the experimental > >>> support in the next release. > >>> > >>> One clarification: since the MVP won't support rescaling, does it imply > >>> that savepoints will always use aligned checkpointing? If so, this > would > >>> still block the user from taking a savepoint and resume with increased > >>> parallelism to resolve a prolonged/permanent backpressure condition? > >>> > >>> Thanks, > >>> Thomas > >>> > >>> > >>> On Tue, Mar 10, 2020 at 6:33 AM Arvid Heise <ar...@ververica.com> > wrote: > >>> > >>>> Hi all, > >>>> > >>>> I would like to start the vote for FLIP-76 [1], which is discussed and > >>>> reached a consensus in the discussion thread [2]. > >>>> > >>>> The vote will be open until March. 13th (72h), unless there is an > >>> objection > >>>> or not enough votes. > >>>> > >>>> Thanks, > >>>> Arvid > >>>> > >>>> [1] > >>>> > >>>> > >>> > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-76%3A+Unaligned+Checkpoints > >>>> [2] > >>>> > >>>> > >>> > >> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-76-Unaligned-checkpoints-td33651.html > >>>> > >>> > >> > > -- Regards, Roman
