RE: RE: checkpointing seems to be throttled.

2020-12-24 Thread Colletta, Edward
, Edward Sent: Monday, December 21, 2020 12:32 PM To: Yun Gao ; user@flink.apache.org Subject: RE: RE: checkpointing seems to be throttled. Doh! Yeah, we set the state backend in code and I read the flink-conf.yaml file and use the high-availability storage dir. From: Yun Gao mailto:yungao

RE: RE: checkpointing seems to be throttled.

2020-12-21 Thread Colletta, Edward
Doh! Yeah, we set the state backend in code and I read the flink-conf.yaml file and use the high-availability storage dir. From: Yun Gao Sent: Monday, December 21, 2020 11:28 AM To: Colletta, Edward ; user@flink.apache.org Subject: Re: RE: checkpointing seems to be throttled. This email

Re: RE: checkpointing seems to be throttled.

2020-12-21 Thread Yun Gao
. We do monitor backpressure through rest api periodically and we do not see any. From: Yun Gao Sent: Monday, December 21, 2020 10:40 AM To: Colletta, Edward ; user@flink.apache.org Subject: Re: checkpointing seems to be throttled. This email is from an external source -exercise caution regarding

RE: checkpointing seems to be throttled.

2020-12-21 Thread Colletta, Edward
; user@flink.apache.org Subject: Re: checkpointing seems to be throttled. This email is from an external source - exercise caution regarding links and attachments. Hi Edward, For the second issue, have you also set the statebackend type? I'm asking so because except for the default heap

Re: checkpointing seems to be throttled.

2020-12-21 Thread Yun Gao
Hi Edward, For the second issue, have you also set the statebackend type? I'm asking so because except for the default heap statebackend, other statebackends should throws exception if the state.checkpoint.dir is not set. Since heap statebackend stores all the snapshots in the JM's memory,