回复: Scaling Flink Jobs without Restarting Job

2023-07-23 Thread Chen Zhanghao
Hi Talat, In reactive mode, rescaling is performed by a whole-graph failover, which is already less costly compared to a full job restart where all containers need to be requested again. For simple stateless jobs, this usually won't take long (a few seconds), you can measure how long it takes

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-23 Thread Benchao Li
Congratulations, Yong! Well deserved! Yangze Guo 于2023年7月24日周一 12:16写道: > Congrats, Yong! > > Best, > Yangze Guo > > On Mon, Jul 24, 2023 at 12:02 PM xiangyu feng > wrote: > > > > Congratulations, Yong! > > > > Best, > > Xiangyu > > > > liu ron 于2023年7月24日周一 11:48写道: > > > > >

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-23 Thread Yangze Guo
Congrats, Yong! Best, Yangze Guo On Mon, Jul 24, 2023 at 12:02 PM xiangyu feng wrote: > > Congratulations, Yong! > > Best, > Xiangyu > > liu ron 于2023年7月24日周一 11:48写道: > > > Congratulations, > > > > Best, > > Ron > > > > Qingsheng Ren 于2023年7月24日周一 11:18写道: > > > > > Congratulations and

Re:Re: Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-23 Thread Wencong Liu
Hi Jing and Matthias, I believe it is reasonable to examine all classes that implement the IOReadableWritable interface and summarize their actual usage. However, due to time constraints, I suggest we minimize the scope of this FLIP to focus on the Path class. As for other components

Re: [DISCUSS] Update Flink Roadmap

2023-07-23 Thread Yuan Mei
Sorry have missed this email and respond a bit late. I will put a draft for the long-term vision for the state as well as large-scale state support into the roadmap. Best Yuan On Mon, Jul 17, 2023 at 10:34 AM Jark Wu wrote: > Hi Jiabao, > > Thank you for your suggestions. I have added them to

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-23 Thread xiangyu feng
Congratulations, Yong! Best, Xiangyu liu ron 于2023年7月24日周一 11:48写道: > Congratulations, > > Best, > Ron > > Qingsheng Ren 于2023年7月24日周一 11:18写道: > > > Congratulations and welcome aboard, Yong! > > > > Best, > > Qingsheng > > > > On Mon, Jul 24, 2023 at 11:14 AM Chen Zhanghao < >

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-23 Thread liu ron
Congratulations, Best, Ron Qingsheng Ren 于2023年7月24日周一 11:18写道: > Congratulations and welcome aboard, Yong! > > Best, > Qingsheng > > On Mon, Jul 24, 2023 at 11:14 AM Chen Zhanghao > wrote: > > > Congrats, Shammon! > > > > Best, > > Zhanghao Chen > > > > 发件人:

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-23 Thread Qingsheng Ren
Congratulations and welcome aboard, Yong! Best, Qingsheng On Mon, Jul 24, 2023 at 11:14 AM Chen Zhanghao wrote: > Congrats, Shammon! > > Best, > Zhanghao Chen > > 发件人: Weihua Hu > 发送时间: 2023年7月24日 11:11 > 收件人: dev@flink.apache.org > 抄送: Shammon FY > 主题: Re:

回复: [ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-23 Thread Chen Zhanghao
Congrats, Shammon! Best, Zhanghao Chen 发件人: Weihua Hu 发送时间: 2023年7月24日 11:11 收件人: dev@flink.apache.org 抄送: Shammon FY 主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang Congratulations! Best, Weihua On Mon, Jul 24, 2023 at 11:04 AM Paul Lam wrote:

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-23 Thread Weihua Hu
Congratulations! Best, Weihua On Mon, Jul 24, 2023 at 11:04 AM Paul Lam wrote: > Congrats, Shammon! > > Best, > Paul Lam > > > 2023年7月24日 10:56,Jingsong Li 写道: > > > > Shammon > >

[jira] [Created] (FLINK-32653) Add doc for catalog store

2023-07-23 Thread Feng Jin (Jira)
Feng Jin created FLINK-32653: Summary: Add doc for catalog store Key: FLINK-32653 URL: https://issues.apache.org/jira/browse/FLINK-32653 Project: Flink Issue Type: Sub-task Components:

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-23 Thread Paul Lam
Congrats, Shammon! Best, Paul Lam > 2023年7月24日 10:56,Jingsong Li 写道: > > Shammon

[ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-23 Thread Jingsong Li
Hi, everyone On behalf of the PMC, I'm very happy to announce Yong Fang (Shammon) (zjur...@gmail.com) as a new Flink Committer. Yong is an old flinker, he has been contributing to Flink since 2017. He actively participated in dev discussions and answered many questions on the user mailing list.

Re: Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-23 Thread weijie guo
+1 Best regards, Weijie Jing Ge 于2023年7月22日周六 00:47写道: > Hi Wencong, > > Thanks for the clarification. I got your point. It makes sense. > > Wrt IOReadableWritable, the suggestion was to check all classes that > implemented it, e.g. BlockInfo, Value, Configuration, etc. Not limited to > the

Re: [DISCUSS][2.0] FLIP-343: Remove parameter in WindowAssigner#getDefaultTrigger()

2023-07-23 Thread weijie guo
+1 for this. Best regards, Weijie liu ron 于2023年7月24日周一 09:58写道: > +1 > > Best, > Ron > > Yuxin Tan 于2023年7月21日周五 16:21写道: > > > +1 > > > > Best, > > Yuxin > > > > > > Jing Ge 于2023年7月21日周五 15:41写道: > > > > > +1 > > > > > > NIT: the release in the FLIP is still empty, it should be 2.0 > >

Re: [DISCUSS] FLIP 333 - Redesign Apache Flink website

2023-07-23 Thread liu ron
+1, The dark mode looks very cool. Best, Ron Matthias Pohl 于2023年7月20日周四 15:45写道: > I think Martijn and Markos brought up a few good points: > > - We shouldn't degrade the accessibility but ideally improve it as part of > the redesign. The current proposal doesn't look like we're doing

Re: [VOTE] FLIP-346: Deprecate ManagedTable related APIs

2023-07-23 Thread liu ron
+1 Best, Ron Lincoln Lee 于2023年7月21日周五 16:09写道: > +1 > > Best, > Lincoln Lee > > > Leonard Xu 于2023年7月21日周五 16:07写道: > > > +1 > > > > Best, > > Leonard > > > > > On Jul 21, 2023, at 4:02 PM, yuxia > wrote: > > > > > > +1(binging) > > > > > > Best regards, > > > Yuxia > > > > > > - 原始邮件

Re: [DISCUSS][2.0] FLIP-343: Remove parameter in WindowAssigner#getDefaultTrigger()

2023-07-23 Thread liu ron
+1 Best, Ron Yuxin Tan 于2023年7月21日周五 16:21写道: > +1 > > Best, > Yuxin > > > Jing Ge 于2023年7月21日周五 15:41写道: > > > +1 > > > > NIT: the release in the FLIP is still empty, it should be 2.0 > > > > Best regards, > > Jing > > > > On Fri, Jul 21, 2023 at 6:03 AM Xintong Song > > wrote: > > > > > +1

[jira] [Created] (FLINK-32652) Operator cannot scale standalone deployments in reactive mode

2023-07-23 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-32652: Summary: Operator cannot scale standalone deployments in reactive mode Key: FLINK-32652 URL: https://issues.apache.org/jira/browse/FLINK-32652 Project: Flink

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

2023-07-23 Thread Dong Lin
Hi Piotr, Thanks a lot for the explanation. Please see my reply inline. On Fri, Jul 21, 2023 at 10:49 PM Piotr Nowojski wrote: > Hi Dong, > > Thanks a lot for the answers. I can now only briefly answer your last > email. > > > It is possible that spilling to disks might cause larger overhead.

Re: [VOTE] Release 2.0 must-have work items - Round 2

2023-07-23 Thread Yuan Mei
+1 (binding) Thanks for driving the discussion through and for all the efforts in resolving the complexities :-) Best Yuan On Thu, Jul 20, 2023 at 5:23 PM Xintong Song wrote: > Hi all, > > I'd like to start another round of VOTE for the must-have work items for > release 2.0 [1]. The

[jira] [Created] (FLINK-32651) Benchmark Support for Changelog Statebackend

2023-07-23 Thread Yuan Mei (Jira)
Yuan Mei created FLINK-32651: Summary: Benchmark Support for Changelog Statebackend Key: FLINK-32651 URL: https://issues.apache.org/jira/browse/FLINK-32651 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-32650) Added the ability to split flink-protobuf codegen code

2023-07-23 Thread Jira
李精卫 created FLINK-32650: --- Summary: Added the ability to split flink-protobuf codegen code Key: FLINK-32650 URL: https://issues.apache.org/jira/browse/FLINK-32650 Project: Flink Issue Type: Improvement

Scaling Flink Jobs without Restarting Job

2023-07-23 Thread Talat Uyarer via dev
HI, We are using Flink with Adaptive Scheduler(Reactive Mode) on Kubernetes with Standalone deployment Application mode for our streaming infrastructure. Our autoscaler is scaling up or down our jobs. However, each scale action causes a job restart. Our customers complain about fluctuating