Re: Contiguity and state storage in CEP library

2021-04-30 Thread tbud
Thanks Dawid. As I see in the code the buffered storage in between watermarks is stored in /MapState> elementQueueState /variable in /class CepOperator/. My question is, if we use rocksDb or some other state backend then would this state be stored on that and checkpointed ? or is it always in the

Re: Contiguity and state storage in CEP library

2021-04-26 Thread Dawid Wysakowicz
Hi, Yes you are correct that if an event can not match any pattern it won't be stored in state. If you process your records in event time it might be stored for a little while before processing in order to sort the incoming records based on time. Once a Watermark with a higher timestamp comes it

Contiguity and state storage in CEP library

2021-04-20 Thread tbud
We are evaluating a use-case where there will be 100s of events stream coming in per second and we want to run some fixed set of pattern matching rules on them And I use relaxed contiguity rules as described in the documentation. for example : /a pattern sequence "a b+ c" on the stream of "a",