Re: Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread Yun Tang
-alignment-timeout Best Yun Tang From: Senhong Liu Sent: Monday, May 31, 2021 10:33 To: JING ZHANG Cc: Kai Fu ; user Subject: Re: Dynamic configuration of Flink checkpoint interval Hi all, In fact, a pretty similar JIRA has been created, which is https

Re: Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread Kai Fu
Hi JING, Here is the issue link: https://issues.apache.org/jira/browse/FLINK-22805 On Mon, May 31, 2021 at 10:21 AM JING ZHANG wrote: > Hi Kai, > > Happy to hear that. > Would you please paste the JIRA link in the email after you create it. > Maybe it could help other users who encounter the

Re: Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread Senhong Liu
Hi all, In fact, a pretty similar JIRA has been created, which is https://issues.apache.org/jira/browse/FLINK-18578 and I am working on it. In the near future, I will publish a FLIP and start a discussion about that. We look forward to your participation. Best, Senhong Liu JING ZHANG

Re: Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread JING ZHANG
Hi Kai, Happy to hear that. Would you please paste the JIRA link in the email after you create it. Maybe it could help other users who encounter the same problem. Thanks very much. Best regards, JING ZHANG Kai Fu 于2021年5月30日周日 下午11:19写道: > Hi Jing, > > Yup, what you're describing is what I

Re: Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread Kai Fu
Hi Jing, Yup, what you're describing is what I want. I also tried the approach you suggested and it works. I'm going to take that approach for the moment and create a Jira issue for this feature. On Sun, May 30, 2021 at 8:57 PM JING ZHANG wrote: > Hi Kai, > > Do you try to find a way to hot

Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread Kai Fu
Hi team, We want to know if Flink has some dynamic configuration of the checkpoint interval. Our use case has a cold start phase where the entire dataset is replayed from the beginning until the most recent ones. In the cold start phase, the resources are fully utilized and the backpressure is