clear State using business logic

2022-11-23 Thread Sigalit Eliazov
Hi all, the flow in our pipeline is: 1. read event X from kafka. open fixed window of 30 sec. 2. read event subscription from kafka. open GlobalWindow and store a state of all subscriptions. 3. match X and Y using key and if there is a match send an event to another kafka topic. (we use the

Re: Beam, Flink state and Avro Schema Evolution is problematic

2022-11-23 Thread Alexey Romanenko
+ dev Many thanks for sharing your observations and findings on this topic, Cristian! I copy it to dev@ as well to attract more attention to this problem. — Alexey > On 18 Nov 2022, at 18:21, Cristian Constantinescu wrote: > > Hi everyone, > > I'm using Beam on Flink with Avro generated