Re: Preparing keyed state before snapshot

2024-02-21 Thread Lorenzo Nicora
you to store operator state as BLOB directly if that would be a >doable option for you. > > > > Sincere greetings > > > > Thias > > > > > > > > > > *From:* Zakelly Lan > *Sent:* Wednesday, February 21, 2024 8:04 AM > *To:* Lorenzo Nicora > *Cc:* F

RE: Preparing keyed state before snapshot

2024-02-21 Thread Schwalbe Matthias
directly if that would be a doable option for you. Sincere greetings Thias From: Zakelly Lan Sent: Wednesday, February 21, 2024 8:04 AM To: Lorenzo Nicora Cc: Flink User Group Subject: Re: Preparing keyed state before snapshot ⚠EXTERNAL MESSAGE – CAUTION: Think Before You Click ⚠ Hi Lorenzo

Re: Preparing keyed state before snapshot

2024-02-20 Thread Zakelly Lan
tion interface in >>>> your KeyedProcessFunction. >>>> >>>> Btw. By the time initializeState(…) is called, the state backend is >>>> fully initialized and can be read and written to (which is not the case for >>>> when the open(…) func

Re: Preparing keyed state before snapshot

2024-02-20 Thread Lorenzo Nicora
time initializeState(…) is called, the state backend is >>> fully initialized and can be read and written to (which is not the case for >>> when the open(…) function is called. >>> >>> In initializeState(…) you also get access to state of different operator >>> key.

Re: Preparing keyed state before snapshot

2024-02-17 Thread Zakelly Lan
ction is called. >> >> In initializeState(…) you also get access to state of different operator >> key. >> >> SnapshotState(…) is called as part of the (each) checkpoint in order to >> store data. >> >> >> >> Sincere greetings >> &

Re: Preparing keyed state before snapshot

2024-02-16 Thread Lorenzo Nicora
tate(…) you also get access to state of different operator > key. > > SnapshotState(…) is called as part of the (each) checkpoint in order to > store data. > > > > Sincere greetings > > > > Thias > > > > *From:* Lorenzo Nicora > *Sent:* Thursday, Fe

RE: Preparing keyed state before snapshot

2024-02-15 Thread Schwalbe Matthias
To: Flink User Group Subject: Preparing keyed state before snapshot Hello everyone, I have a convoluted problem. I am implementing a KeyedProcessFunction that keeps some non-serializable "state" in memory, in a transient Map (key = stream key, value = the non-serializable "state&qu

Preparing keyed state before snapshot

2024-02-15 Thread Lorenzo Nicora
Hello everyone, I have a convoluted problem. I am implementing a KeyedProcessFunction that keeps some non-serializable "state" in memory, in a transient Map (key = stream key, value = the non-serializable "state"). I can extract a serializable representation to put in Flink state, and I can