Semver was broken with Async checkpointing. The behavior was changed as pointed out before in the discussion. Also making it difficult for operator developer doesn't give us anything.
+1 for fixing it in the way Thomas suggested. On Wed, Nov 25, 2015 at 4:07 PM, Chetan Narsude (cnarsude) < [email protected]> wrote: > Yes - a few but cannot share the details - protected under NDA - ping me > in private and I can probably be able to give you more generic details on > similar cooked up examples. > > The part that follows “e.g.” below is an example that probably is > sufficient to infer the use case logically, I think. I shared that to > exemplify how changing the semantics will break semver. > > — > Chetan > > > > > > > > > > > > On 11/25/15, 3:51 PM, "Thomas Weise" <[email protected]> wrote: > > >Do you have a specific example? > > > >I see this happening in committed(), but not in checkpointed() where the > >checkpoint remains intermediate, whether it was copied to HDFS or not. > > > > > >On Wed, Nov 25, 2015 at 3:42 PM, Chetan Narsude (cnarsude) < > >[email protected]> wrote: > > > >> > > >> >Until we have this, how about we restore the previous behavior > >> >temporarily? > >> >Calling checkpointed() immediately does not seem to pose any practical > >> >issue but ensures that the code that was written under this assumption > >>is > >> >not broken. > >> > >> We can¹t do it. It would be incorrect. It breaks all the other code that > >> (unassumingly) correctly complied to the semantics. e.g. an operator > >>which > >> informs interesting parties that the checkpointed data is available for > >> immediate consumption from storage. > >> > >> ‹ > >> Chetan > >> > >> > >
