Re: Window processing in Stateful Functions

2020-05-12 Thread Igal Shilman
Hi, One way to keep the state size under control would be: 1) attach for every incoming edge it's "insertion time" into the vertex function's state. 2) in addition, the vertex function would send a delayed message, with a delay of insertion time + expiration duration 3) once a delayed message arriv

Re: Window processing in Stateful Functions

2020-05-08 Thread m@xi
Dear Igal,Very insightful answer. Thanks. Igal Shilman wrote > An alternative approach would be to implement a * > thumbling window * > per vertex(a stateful function instance)by sending to itself a delayed > message [2]. When that specific delayedmessage arrives you wouldhave to > purge the olde

Re: Window processing in Stateful Functions

2020-05-07 Thread Igal Shilman
Hi all, Data stream windows are not yet supported in statefun, but it seems like the main motivation here is to purge old edges? If this is the case perhaps we need to integrate state TTL [1] into persisted values/persistedtables. An alternative approach would be to implement a thumbling window p

Re: Window processing in Stateful Functions

2020-05-06 Thread Oytun Tez
Oops – will follow the thread 🙊 -- [image: MotaWord] Oytun Tez M O T A W O R D | CTO & Co-Founder oy...@motaword.com On Wed, May 6, 2020 at 5:37 PM m@xi wrote: > Hello Tez, > > With all the respect, I doubt your answer is related the question. > > *Jus

Re: Window processing in Stateful Functions

2020-05-06 Thread m@xi
Hello Tez, With all the respect, I doubt your answer is related the question. *Just to re-phase a bit*: Assuming we use SF for our application, how can we apply window logic when a function does some processing? *Is there a proper way?* @*Igal*: we would very much appreciate your answer. :) Bes

Re: Window processing in Stateful Functions

2020-05-06 Thread Oytun Tez
I think this is also related to my question about CEP in Statefun. @Annemarie Burger , I was looking into using Siddhi library within the Function's context. -- [image: MotaWord] Oytun Tez M O T A W O R D | CTO & Co-Founder oy...@motaword.com On Wed,