Hi! No, the operator does not need to pause processing input records while the checkpointing of its state is in progress. The checkpointing of operator state is asynchronous. The operator state does not need to be immutable, since its a copy of the snapshot state that’s checkpointed.
Regards, Gordon On October 17, 2016 at 10:28:34 AM, Li Wang (wangli1...@gmail.com) wrote: Hi All, Any feedback is highly appreciated. Thanks. Li > On Oct 15, 2016, at 11:17 AM, Li Wang <wangli1...@gmail.com> wrote: > > Hi all, > > As far as I know, a stateful operator will checkpoint its current state to a > persistent storage when it receives all the barrier from all of its upstream > operators. My question is that does the operator doing the checkpoint need to > pause processing the input tuples for the next batch until the checkpoint is > done? If yes, will it introduce significant processing latency when the state > is large. If no, does this need the operator state to be immutable? > > Thanks, > Li
