Bhupesh, Platform supports sliding window feature natively. You can refer to SLIDE_BY_WINDOW_COUNT attribute (https://github.com/apache/incubator-apex-core/blob/master/api/src/main/java/com/datatorrent/api/Context.java <https://github.com/apache/incubator-apex-core/blob/master/api/src/main/java/com/datatorrent/api/Context.java>) and you can also look at SliderTest https://github.com/apache/incubator-apex-core/blob/master/engine/src/test/java/com/datatorrent/stram/engine/SliderTest.java
Thanks - Gaurav > On Nov 9, 2015, at 12:00 AM, Bhupesh Chawda <bhup...@datatorrent.com> wrote: > > Yes, it will be reset in the endWindow call. So nothing should be needed to > be done explicitly. > Also, after the crash, the operator is restored to the checkpoint which had > happened after beginWindow but before endWindow. In this case the > beginWindow() call will be skipped when the operator restarts after > recovery. > It is easy to wrongly assume (which I did) that checkpoints always happen > "after endWindow" (in which case, the variable can be made transient). > > I also had a concern regarding *sliding application windows*. It is not > clear as to how the beginWindow and endWindow calls would be structured. In > addition to the application window size, we would also need some *delta* > which is the amount by which the window would "slide" by. How is this > specified? Is this per tuple? Also how would checkpointing and recovery act > in such a case? > > Thanks. > Bhupesh > > On Mon, Nov 9, 2015 at 12:39 PM, Milind Barve <mili...@gmail.com> wrote: > >> You might want to initialize it in beginwidow() as well. Should keep it non >> transient so for recovery. >> On Nov 9, 2015 12:34 PM, "Amol Kekre" <a...@datatorrent.com> wrote: >> >>> Bhupesh. >>> I would recommend reseting the variable in endWindow call and leaving it >>> non-transient. Usually an empty variable should not add too much to >>> serialization. >>> >>> Thks, >>> Amol >>> >>> >>> On Sun, Nov 8, 2015 at 10:57 PM, Bhupesh Chawda <bhup...@datatorrent.com >>> >>> wrote: >>> >>>> Hi All, >>>> >>>> I have a question regarding maintaining state of a variable within a >>> single >>>> application window. >>>> If I have a variable which is just needed for a single application >> window >>>> (just between beginWindow and endWindow calls), do I need to serialize >>> and >>>> checkpoint it? or can I make it transient? >>>> >>>> I understand that in case of checkpoint windows being aligned with the >>>> application windows, we can do so. Since in this case, the variable >> will >>>> start afresh immediately after the checkpoint. However in case of >>>> checkpoints happening in between application windows, this might not be >>> the >>>> case. >>>> >>>> In the second case, is there any mechanism using which I can >>> conditionally >>>> make it transient and non-transient based on when the checkpoints >> happen >>>> with respect to the application windows? >>>> >>>> Thanks. >>>> Bhupesh >>>> >>> >>
