There seems to be confusion between "checkpointed" and "committed". I put my comment on that JIRA.
Serializing the state of the operator is always blocking. Write to HDFS can be asynchronous. Commit notification is always asynchronous. Thomas On Sat, Sep 19, 2015 at 10:19 AM, Amol Kekre <[email protected]> wrote: > Bhupesh, > If you want to guaranteed that checkpoint is fully done before next tuple > is processed, you will need to use blocking checkpoint attribute. > > Thks > Amol > > On Sat, Sep 19, 2015 at 8:59 AM, Sandesh Hegde <[email protected]> > wrote: > > > Blocking checkpoint is supported, please refer to the following jira, > > https://malhar.atlassian.net/browse/APEX-19 > > > > On Fri, Sep 18, 2015 at 9:08 PM Bhupesh Chawda <[email protected]> > > wrote: > > > > > Thanks for the replies. > > > My intention was to ask if the call to next begin window will start > only > > > after the checkpointed call has done it's job. > > > So if just the call is sequenced, but it's a different thread, then it > > does > > > not really help in my case. I need to finish the job which I need to do > > in > > > checkpointed call before any new tuples are processed. > > > On 17-Sep-2015 5:56 am, "Chetan Narsude" <[email protected]> > wrote: > > > > > > > Short answer is yes. > > > > > > > > All the control tuples are scheduled to be delivered outside of the > > > window. > > > > As checkpointed callback is triggered because of CHECKPOINT control > > > tuple, > > > > it will happen after endWindow and before the next beginWindow. > > > > > > > > The windowId supplied to CheckpointListener and the one provided to > > > > Operator need not match even though the sequence is defined. So I am > > > > curious how you intend to use this knowledge. > > > > > > > > -- > > > > Chetan > > > > > > > > > > > > On Tue, Sep 15, 2015 at 8:31 AM, Thomas Weise < > [email protected]> > > > > wrote: > > > > > > > > > It has not changed the operator execution model. State > serialization > > is > > > > > still synchronous, write to HDFS is taken out of the operator > thread. > > > > > > > > > > On Tue, Sep 15, 2015 at 8:18 AM, Amol Kekre <[email protected]> > > > > wrote: > > > > > > > > > > > > > > > > > Sent too soon. Has asynchronous checkpointing changed this? > > > > > > > > > > > > Amol > > > > > > > > > > > > Sent from my iPhone > > > > > > > > > > > > > On Sep 15, 2015, at 12:38 AM, Bhupesh Chawda < > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > Is it safe to assume that the checkpointed() and the > > beginWindow() > > > > > calls > > > > > > > are sequenced? > > > > > > > In other words, are these calls part of the same thread and may > > not > > > > run > > > > > > in > > > > > > > parallel? > > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > -- > > > > > > > -Bhupesh > > > > > > > > > > > > > > > > > > > > >
