Hey Guys, Our thought was that sliding window could be implemented with a buffer inside of a process() call. For example, you might have a list of 10 elements, and every time process() is invoked, you could add the message to the ahead of the list, and dequeue the last element from the buffer (if it already has 10 elements in it).
So, Samza currently doesn't support any explicit sliding window, but it seems to me that you could implement it in StreamTask.process() if you need to. Cheers, Chris On 9/9/14 10:31 PM, "Mayur Rustagi" <[email protected]> wrote: >Slide window is another dimension to processing > >say my batch is 3 sec & window is 9 sec then this is what I get > >[ x1 x2 x3][x4 x5 x6][x7 x8 x9] > >This is using slide but slide is equal to window size, but i may want to >get last 3 elements at any point of time then that would be > > >[ x1 x2 x3] after3sec [ x2 x3 x4] after3sec [ x3 x4 x5] after3sec [ >x4 x5 x6] > >to implement this you use batch of 3 sec, window of 9 sec & slide >duration of 3 sec. So we are sliding every 3 sec & also getting a batch >every 3 sec. > > > > > >-- >Regards, >Mayur Rustagi >Ph: +1 (760) 203 3257 >http://www.sigmoidanalytics.com >@mayur_rustagi > >On Wed, Sep 10, 2014 at 5:25 AM, Yan Fang <[email protected]> wrote: > >> Hi guys, >> realize that both Storm and Spark Streaming have sliding window >> implementation while Samza only has the fixed window (not sure if it's a >> correct name). I think you guys must consider this idea at the >>beginning of >> designing the Samza. What was the thought? Thank you. >> Cheers, >> Fang, Yan >> [email protected] >> +1 (206) 849-4108
