I've seen this in BEAM as well and I really like it. - I'm finishing up the initial work for FLIP-3 (Docs restructuring) and hope to post something to the mailing list later today about it.
- I will start working on top of Aljoscha's and Stefan's pull request #2376 with an initial version for non-partitioned rescalable state. I think that this will continue into next week. - Other than that, I want to write down a shirt FLIP for exposing the savepoint/checkpoint unification to the user (internal refactoring is an open PR #2366). On Wed, Aug 17, 2016 at 10:39 AM, Till Rohrmann <trohrm...@apache.org> wrote: > Good initiative @Aljoscha. > > - Just a convenience link for FLIP-6: The feature branch can be found here > https://github.com/apache/flink/tree/flip-6. > > - My main task is the review of the Mesos PR of Eron ( > https://github.com/apache/flink/pull/2315) so that we can continue making > progress there. > > On Wed, Aug 17, 2016 at 10:14 AM, Stephan Ewen <se...@apache.org> wrote: > >> Nice idea, I like that. I'll join with some HEADS-UP: >> >> - Till and me started working on FLIP-6 (rework the Yarn/Mesos/standalone >> mode) together with some contributors from Alibaba. The work is currently >> in a feature branch. >> >> - As part of that rework, we experimented with a new RPC abstraction that >> builds on top of Akka. Think of it as "Flink-flavored typed actors". It has >> actually turned out to be very helpful, we plan to write about it in a bit >> (once it has converged) >> >> - Other than that, Ufuk and me did some work on unifying checkpoints and >> savepoints. That will make it easier to recover on another cluster from a >> regular checkpoint (in case the system gets stuck) and that a regular >> savepoint will be used for recovery in the program that created it. >> >> >> >> >> >> On Tue, Aug 16, 2016 at 10:34 PM, Aljoscha Krettek <aljos...@apache.org> >> wrote: >> >> > Hi Folks, >> > I just had an idea that I want to try now. The idea is to give little >> > HEADS-UP emails roughly at the beginning of each week. I would give a >> very >> > short summary of the big points that I want to work on for that week and >> if >> > others want to jump in they can also quickly outline what they are doing. >> > >> > The reason I'm proposing this is that it can be hard to follow what is >> > going on in Flink. One could try and follow all the Jira issues and see >> who >> > is assigned to what but that would only give a rough indication. >> Currently, >> > people are working away on a new feature, sometimes for several weeks, >> > until they make a big code drop in a PR. I though it would be nice to >> know >> > roughly who's doing what. This would also give people the opportunity to >> > get involved in something if they find it interesting. If people find >> this >> > interesting I'll try and do this every week from now on. >> > >> > Here goes my first heads-up: This (and the last weeks) week Stefan and I >> > have been working on the issues around >> > https://issues.apache.org/jira/browse/FLINK-3755. We were mostly >> > resurrecting an old PR from Till. Now we got it into good shape, we just >> > opened a big PR (https://github.com/apache/flink/pull/2376) that >> > introduces >> > API for key-groups and support in some layers. For the rest of the week, >> > we're planning to refactor the state backends to make them work with >> > key-group aware state. >> > >> > Happy coding :-) >> > Aljoscha >> > >>
