Re: [ANNOUNCE] Apache Flink 1.13.1 released

2021-05-30 Thread 김영우
Great work! Thank you Dawid and all of the contributors. I'm eager to adopt the new release, however can't find docker images for that from https://hub.docker.com/_/flink Hope it'll be available soon. Thanks, Youngwoo On Sat, May 29, 2021 at 1:49 AM Dawid Wysakowicz wrote: > The Apache Flink

Got exception when running the localhost cluster

2021-05-30 Thread Lingfeng Pu
Hi, I'm new to Flink. I got a problem when running the local cluster on my computer. Some key software information as follows: 1. Flink version: 1.13.0 for Scala 2.11; 2. OS: Fedora 34; 3. Java version: 16; 4. Scala version: 2.11.12. When I started up the local cluster by command line, everythin

(无主题)

2021-05-30 Thread 陈海剑
退订 | | 陈海剑 | | 邮箱:13469958...@163.com best regards | 签名由 网易邮箱大师 定制

Re: Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread Yun Tang
Hi Kai, I think unaligned checkpoint + alignment timeout [1] might also help you in this case. You could leverage unaligned checkpoint to help reduce the checkpoint duration. [1] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/config/#execution-checkpointing-align

Re: Error restarting job from Savepoint

2021-05-30 Thread Yun Tang
Hi Ganti, If you could ensure that newer class could keep backwards compatibility as previous class, you can try to set serialVesionUID explicitly of current class to -7317586767482317266. If you want to avoid such issue later, you must set the serialVesionUID explicitly first if not using cus

Re: Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread Kai Fu
Hi JING, Here is the issue link: https://issues.apache.org/jira/browse/FLINK-22805 On Mon, May 31, 2021 at 10:21 AM JING ZHANG wrote: > Hi Kai, > > Happy to hear that. > Would you please paste the JIRA link in the email after you create it. > Maybe it could help other users who encounter the sa

Re: Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread Senhong Liu
Hi all, In fact, a pretty similar JIRA has been created, which is https://issues.apache.org/jira/browse/FLINK-18578 and I am working on it. In the near future, I will publish a FLIP and start a discussion about that. We look forward to your participation. Best, Senhong Liu JING ZHANG 于2021年5月31

Re: Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread JING ZHANG
Hi Kai, Happy to hear that. Would you please paste the JIRA link in the email after you create it. Maybe it could help other users who encounter the same problem. Thanks very much. Best regards, JING ZHANG Kai Fu 于2021年5月30日周日 下午11:19写道: > Hi Jing, > > Yup, what you're describing is what I wan

Re: Apache Flink - A question about Tables API and SQL interfaces

2021-05-30 Thread Theo Diefenthal
Hi Mans, Regarding to your first question: I bookmarked the following mailing list discussion a while ago [1]. Fabian Hueske as one of the major contributors to Flink answered that there aren't yet any trigger semantics in Flink SQL, but linked a great idea with a SQL extension of "EMIT". I r

Re: Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread Kai Fu
Hi Jing, Yup, what you're describing is what I want. I also tried the approach you suggested and it works. I'm going to take that approach for the moment and create a Jira issue for this feature. On Sun, May 30, 2021 at 8:57 PM JING ZHANG wrote: > Hi Kai, > > Do you try to find a way to hot upd

Re: Alternate way to pass plugin Jars

2021-05-30 Thread Vijayendra Yadav
Ok thanks Pohl Sent from my iPhone > On May 28, 2021, at 5:49 AM, Matthias Pohl wrote: > >  > Hi Vijayendra, > Thanks for reaching out to the Flink community. There is no other way I know > of to achieve what you're looking for. The plugins support is provided > through the respective ./plug

Flink kafka

2021-05-30 Thread 周瑞
程序用于测试 flink kafka exactly once, 普通提交可以正常运行, 但如果从 savepoint 中恢复就会报下面的错误 kafka server 端, 配置了  transaction.timeout.ms = 90 public static final String KAFKA_TABLE_FORMAT = "CREATE TABLE "+TABLE_NAME+" (\n" + " "+COLUMN_NAME+" STRING\n" + ") WITH (\n" +

Dynamic configuration of Flink checkpoint interval

2021-05-30 Thread Kai Fu
Hi team, We want to know if Flink has some dynamic configuration of the checkpoint interval. Our use case has a cold start phase where the entire dataset is replayed from the beginning until the most recent ones. In the cold start phase, the resources are fully utilized and the backpressure is hi

classloader.resolve-order is not honored when submitting job to a remote cluster

2021-05-30 Thread tao xiao
Hi team, I discovered that child first class loader is always used to initialize the main program when submitting the job to a yarn cluster using application mode regardless of what value classloader.resolve-order is set in flink-conf.yaml. But this is not the case if I submit the same job with th