I think we should focus on getting APEXCORE-78 done instead. It will address a number of existing use cases, including those where we tried to utilize checkpointed(...).
On Fri, Dec 11, 2015 at 9:36 AM, Chandni Singh <[email protected]> wrote: > Hi, > > There hasn't been a close bracket here (borrowing Ram's expression :-) ). > From what I see the majority agrees with making the fix. > > Do we need to start a vote for this? > > On Fri, Nov 27, 2015 at 3:00 PM, Thomas Weise <[email protected]> > wrote: > > > Chetan, > > > > Would like to understand how the checkpointed callback helps you with > what > > you indicated. This may require a specific example. Let's take it > offline. > > > > Thomas > > > > On Wed, Nov 25, 2015 at 4:07 PM, Chetan Narsude (cnarsude) < > > [email protected]> wrote: > > > > > Yes - a few but cannot share the details - protected under NDA - ping > me > > > in private and I can probably be able to give you more generic details > on > > > similar cooked up examples. > > > > > > The part that follows “e.g.” below is an example that probably is > > > sufficient to infer the use case logically, I think. I shared that to > > > exemplify how changing the semantics will break semver. > > > > > > — > > > Chetan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 11/25/15, 3:51 PM, "Thomas Weise" <[email protected]> wrote: > > > > > > >Do you have a specific example? > > > > > > > >I see this happening in committed(), but not in checkpointed() where > the > > > >checkpoint remains intermediate, whether it was copied to HDFS or not. > > > > > > > > > > > >On Wed, Nov 25, 2015 at 3:42 PM, Chetan Narsude (cnarsude) < > > > >[email protected]> wrote: > > > > > > > >> > > > > >> >Until we have this, how about we restore the previous behavior > > > >> >temporarily? > > > >> >Calling checkpointed() immediately does not seem to pose any > > practical > > > >> >issue but ensures that the code that was written under this > > assumption > > > >>is > > > >> >not broken. > > > >> > > > >> We can¹t do it. It would be incorrect. It breaks all the other code > > that > > > >> (unassumingly) correctly complied to the semantics. e.g. an operator > > > >>which > > > >> informs interesting parties that the checkpointed data is available > > for > > > >> immediate consumption from storage. > > > >> > > > >> ‹ > > > >> Chetan > > > >> > > > >> > > > > > > > > >
