Do you have a specific example? I see this happening in committed(), but not in checkpointed() where the checkpoint remains intermediate, whether it was copied to HDFS or not.
On Wed, Nov 25, 2015 at 3:42 PM, Chetan Narsude (cnarsude) < [email protected]> wrote: > > > >Until we have this, how about we restore the previous behavior > >temporarily? > >Calling checkpointed() immediately does not seem to pose any practical > >issue but ensures that the code that was written under this assumption is > >not broken. > > We can¹t do it. It would be incorrect. It breaks all the other code that > (unassumingly) correctly complied to the semantics. e.g. an operator which > informs interesting parties that the checkpointed data is available for > immediate consumption from storage. > > ‹ > Chetan > >
