Honestly, I don't think you need CEP or MATCH_RECOGNIZE for that use case. It can be solved with a simple process function that tracks the state for each id. Output a 1 when a job completes and a -1 if canceled. Output the sum. You can use a simple timer to clear the state for a job after 6 months have passed.
Seth On Wed, Sep 15, 2021 at 12:34 PM Pedro Silva <pedro.cl...@gmail.com> wrote: > Hello, > > As anyone used streaming sql pattern matching as shown in this email > thread to count certain transitions on a stream? > Is it feasible? > > Thank you, > Pedro Silva > > On 13 Sep 2021, at 11:16, Pedro Silva <pedro.cl...@gmail.com> wrote: > > > Hello Seth, > > Thank you very much for your reply. I've taken a look at MATCH_RECOGNIZE > but I have the following doubt. Can I implement a state machine that detect > patterns with multiple end states? > To give you a concrete example: > > I'm trying to count the number of *Jobs* that have been *cancelled* and > *completed*. The state machine associated with this Job concept is as > follows: > Started -> On-Going (Multiple Progress messages) -> Closed -> Completed > ----\ > > \--------------\--------------------------------------------------------\----------------\------------- > > Cancelled > > At any point the Job can be cancelled from the previous state. > This cancel message can take anywhere from 1-2 weeks to be received. > The duration of this state machine (Job lifecycle) is roughly 6 months. > > How can I keep a count of the number of Jobs that have been completed but > not cancelled such that when a cancel appears on a previously (completed | > closed) I decrease my counter but not when a cancel appears after a > started or progress state (no counter increment or decrement) ? > > I hope this example was clear. > > Thank you for your time! > Pedro Silva > > > Em sex., 10 de set. de 2021 às 20:18, Seth Wiesman <sjwies...@gmail.com> > escreveu: > >> Hi Pedro, >> >> The DataStream CEP library is not available in Python but you can use >> `MATCH_RECOGNIZE` in the table API which is implemented on-top of the CEP >> library from Python. >> >> >> https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/dev/table/sql/queries/match_recognize/ >> >> >> Seth >> >> On Fri, Sep 10, 2021 at 11:34 AM Pedro Silva <pedro.cl...@gmail.com> >> wrote: >> >>> Hello, >>> >>> Is Flink's CEP library >>> <https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/libs/cep/> >>> available >>> in python? From the documentation I see no references so I'm guessing the >>> answer is no but wanted some confirmation from the community or developers. >>> >>> Are there plans to support this library in python or alternatively, >>> another library altogether that can be used in python? >>> >>> Thank you and have a nice weekend, >>> Pedro Silva >>> >>
