Re: Checkpoint size increasing even i enable increasemental checkpoint

2021-10-12 Thread Vijay Bhaskar
-- > *From:* Lei Wang > *Sent:* Monday, October 11, 2021 16:16 > *To:* user > *Subject:* Checkpoint size increasing even i enable increasemental > checkpoint > > > [image: image.png] > > The checkpointed data size became bigger and bigger and the n

Re: Checkpoint size increasing even i enable increasemental checkpoint

2021-10-12 Thread Yun Tang
From: Lei Wang Sent: Monday, October 11, 2021 16:16 To: user Subject: Checkpoint size increasing even i enable increasemental checkpoint [image.png] The checkpointed data size became bigger and bigger and the node cpu is very high when the job is doing checkpointing. But I have

Re: Checkpoint size increasing even i enable increasemental checkpoint

2021-10-11 Thread Caizhi Weng
Hi! Checkpoint sizes are highly related to your job. Incremental checkpointing will help only when the values in the state are converging (for example a distinct count aggregation). If possible, could you provide your user code or explain what jobs are you running? Lei Wang 于2021年10月11日周一

Checkpoint size increasing even i enable increasemental checkpoint

2021-10-11 Thread Lei Wang
[image: image.png] The checkpointed data size became bigger and bigger and the node cpu is very high when the job is doing checkpointing. But I have enabled incremental checkpointing: env.setStateBackend(new RocksDBStateBackend(checkpointDir, true)); I am using flink-1.11.2 and aliyun oss as