Hi, Shekar, First, I would like to clarify what you meant by sliding window: is it defined as windows with size N and advance step size of 1 (which means that windows overlap and each input message would contribute to multiple counts in different windows)? Or windows with size N and advance step size of N (i.e. each incoming message only contribute to one counter in a single window)?
If your use case falls into the first category, you will need something more sophisticated as discussed in SAMZA-552. If your use case is the second one, there could be a simpler version of SAMZA-552 that you can go with: 1) Initiate a KV-store that uses the application name as the key 2) For each incoming message, look for the windows that the message by the application name 3) Update the counter and update the value in the KV-store based on the application name 4) Every 5 min when window() method is triggered, set all counters to zero (this can be done in a lazy way as well, by keeping the last reset timestamp in the record in the KV-store, keyed by application name. Then, resetting counter to zero can be done when next time the application counter is updated again) Hope that makes sense. -Yi On Mon, Jun 29, 2015 at 10:06 AM, Shekar Tippur <ctip...@gmail.com> wrote: > Benjamin, > > Thanks for the explanation. We dont have any specific partition scheme as > yet. We just have 2 topics - raw and processed and we use default > partitioning scheme. > Can you share any code snippet so I can understand it better? > > - Shekar >
