Thanks Gaurav, that helps. -Bhupesh
On Mon, Nov 9, 2015 at 1:57 PM, Gaurav Gupta <gau...@datatorrent.com> wrote: > Bhupesh, > > Platform supports sliding window feature natively. You can refer to > SLIDE_BY_WINDOW_COUNT attribute ( > https://github.com/apache/incubator-apex-core/blob/master/api/src/main/java/com/datatorrent/api/Context.java > < > https://github.com/apache/incubator-apex-core/blob/master/api/src/main/java/com/datatorrent/api/Context.java>) > and you can also look at SliderTest > https://github.com/apache/incubator-apex-core/blob/master/engine/src/test/java/com/datatorrent/stram/engine/SliderTest.java > > Thanks > - Gaurav > > > On Nov 9, 2015, at 12:00 AM, Bhupesh Chawda <bhup...@datatorrent.com> > wrote: > > > > Yes, it will be reset in the endWindow call. So nothing should be needed > to > > be done explicitly. > > Also, after the crash, the operator is restored to the checkpoint which > had > > happened after beginWindow but before endWindow. In this case the > > beginWindow() call will be skipped when the operator restarts after > > recovery. > > It is easy to wrongly assume (which I did) that checkpoints always happen > > "after endWindow" (in which case, the variable can be made transient). > > > > I also had a concern regarding *sliding application windows*. It is not > > clear as to how the beginWindow and endWindow calls would be structured. > In > > addition to the application window size, we would also need some *delta* > > which is the amount by which the window would "slide" by. How is this > > specified? Is this per tuple? Also how would checkpointing and recovery > act > > in such a case? > > > > Thanks. > > Bhupesh > > > > On Mon, Nov 9, 2015 at 12:39 PM, Milind Barve <mili...@gmail.com> wrote: > > > >> You might want to initialize it in beginwidow() as well. Should keep it > non > >> transient so for recovery. > >> On Nov 9, 2015 12:34 PM, "Amol Kekre" <a...@datatorrent.com> wrote: > >> > >>> Bhupesh. > >>> I would recommend reseting the variable in endWindow call and leaving > it > >>> non-transient. Usually an empty variable should not add too much to > >>> serialization. > >>> > >>> Thks, > >>> Amol > >>> > >>> > >>> On Sun, Nov 8, 2015 at 10:57 PM, Bhupesh Chawda < > bhup...@datatorrent.com > >>> > >>> wrote: > >>> > >>>> Hi All, > >>>> > >>>> I have a question regarding maintaining state of a variable within a > >>> single > >>>> application window. > >>>> If I have a variable which is just needed for a single application > >> window > >>>> (just between beginWindow and endWindow calls), do I need to serialize > >>> and > >>>> checkpoint it? or can I make it transient? > >>>> > >>>> I understand that in case of checkpoint windows being aligned with the > >>>> application windows, we can do so. Since in this case, the variable > >> will > >>>> start afresh immediately after the checkpoint. However in case of > >>>> checkpoints happening in between application windows, this might not > be > >>> the > >>>> case. > >>>> > >>>> In the second case, is there any mechanism using which I can > >>> conditionally > >>>> make it transient and non-transient based on when the checkpoints > >> happen > >>>> with respect to the application windows? > >>>> > >>>> Thanks. > >>>> Bhupesh > >>>> > >>> > >> > >
