Re: On efficient checkpoints with dynamic (self-evolving) keyed state

2020-04-06 Thread Salva Alcántara
Yet another option would be to use operator state instead, but this looks trickier to me. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: On efficient checkpoints with dynamic (self-evolving) keyed state

2020-04-06 Thread Salva Alcántara
I guess another option not mentioned in my question could be to use a custom serializer for the models. This way, I would not need to consider serialization issues myself within the process function and the snapshots for my models would be taken only once per checkpoint as desired -- Sent from:

On efficient checkpoints with dynamic (self-evolving) keyed state

2020-04-06 Thread Salva Alcántara
In a KeyedCoProcessFunction, I am managing a keyed state which consists of third-party library models. These models are created on reception of new data on the control stream within `processElement1`. Because the models are self-evolving, in the sense that have their own internal state, I need to