I understand your concerns Robert but I don't fully agree. The Checkpointed interface works indeed but there are so many use cases that it is not suitable for in the long run, and also the whole interface is slightly awkward in my opinion when returning simple fields which are already serializable.
This motivated the introduction of the OperatorStateInterface which you can call the first rework of the checkpointed interface, but I see that as the first version which is actually capable of handling many issues that were obvious with the Checkpointed interfaces. This is actually not only a rework of the interface but the rework of the state concept and runtime handling. This needs to be clean if we are moving streaming out of beta, and should provide the needed funcionality. I think we can afford to experiment around a little bit with these interfaces and see the implications for the applications that we can develop with them as we think of statefulness as a major advantage of Flink streaming. So actually I think this is the only time when we can afford rework these interfaces without big costs to make it work for the future. Robert Metzger <rmetz...@apache.org> ezt írta (időpont: 2015. júl. 1., Sze, 11:25): > Whats causing me the biggest headache here is that I don't see an end on > all these "state interface" reworks. > I think this is now the third big change to the interface. > It is a horrible user experience to rework your old code with each new > Flink release. > > I understand that there are always ways to improve interfaces, and I'm sure > Flink has many that we can improve. > But there are (in my opinion) more important things than reworking the > interfaces every second week ... for example that the functionality they > are providing is actually working and well tested. > > > > On Wed, Jul 1, 2015 at 11:15 AM, Ufuk Celebi <u...@apache.org> wrote: > > > > > On 01 Jul 2015, at 10:57, Gyula Fóra <gyula.f...@gmail.com> wrote: > > > > > Hey, > > > > > > Thanks for the feedback guys: > > > > > > @Max: You are right, this is not top priority to changes, I was just > > > mocking up some alternatives to try to make the state usage even > simpler > > so > > > that the user can keep his current implementations and just add 1-2 > > > annotations. > > > > I agree. It's good to cover the "basic" case with a simple solution. :-) > > > > > @Stephan, Robert: You are right that the checkpointed interface has > some > > > advantages from that point of view. Maybe a way to go would be to > > separate > > > this signaling functionality (when the checkpoint is taken and maybe > also > > > the commits) from the snapshotting itself. One advantage I see there is > > > that we would not need to have 3 different interfaces doing pretty much > > the > > > same thing (OperatorState - needed for partitioned state and different > > > backends/out-of-core, Checkpointed - needed for special actions after > > > checkpoints, Annotations - checkpointing simple fields natively). > > > > I also agree with Stephan and Robert that there are other use cases, > which > > require the interfaces. I cannot judge your proposal at this point > though. > > I'm eager to hear what the others say who worked on this. > > > > – Ufuk >
