Re: [DISCUSS] Merge "flink run" and "flink run-application" in Flink 2.0

2024-05-16 Thread Xintong Song
AFAIK, the main purpose of having `run-application` was to make sure the user is aware that application mode is used, which executes the main method of the user program in JM rather than in client. This was important at the time application mode was first introduced, but maybe not that important

Re: [VOTE] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-15 Thread Xintong Song
+1 (binding) Best, Xintong On Wed, May 15, 2024 at 6:53 PM weijie guo wrote: > +1(binding) > > Best regards, > > Weijie > > > Rui Fan <1996fan...@gmail.com> 于2024年5月15日周三 17:50写道: > > > +1(binding) > > > > Best, > > Rui > > > > On Wed, May 15, 2024 at 5:01 PM Xuannan Su > wrote: > > > > >

[jira] [Created] (FLINK-35331) Download links for binary releases are displayed as source releases on website

2024-05-10 Thread Xintong Song (Jira)
Xintong Song created FLINK-35331: Summary: Download links for binary releases are displayed as source releases on website Key: FLINK-35331 URL: https://issues.apache.org/jira/browse/FLINK-35331

Re: [DISCUSSION] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-04-29 Thread Xintong Song
Thanks for driving this effort, Xuannan. +1 for the proposed changes. Just one suggestion: Some of the proposed changes involve not solely changing the configuration options, but are bound to changing / removal of certain features. E.g., the removal of hash-blocking shuffle and legacy hybrid

Re: [VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-17 Thread Xintong Song
+1 (binding) Best, Xintong On Wed, Apr 17, 2024 at 4:47 PM Zakelly Lan wrote: > +1 binding > > > Best, > Zakelly > > On Wed, Apr 17, 2024 at 2:05 PM Rui Fan <1996fan...@gmail.com> wrote: > > > +1(binding) > > > > Best, > > Rui > > > > On Wed, Apr 17, 2024 at 1:02 PM Xuannan Su > wrote: > >

Re: Re: [VOTE] FLIP-423: Disaggregated State Storage and Management (Umbrella FLIP)

2024-03-29 Thread Xintong Song
+1 (binding) Best, Xintong On Fri, Mar 29, 2024 at 10:28 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > On Thu, Mar 28, 2024 at 10:13 PM Jing Ge > wrote: > > > +1(binding) > > > > Best Regards, > > Jing > > > > On Thu, Mar 28, 2024 at 1:28 PM Feifan Wang wrote:

Re: [VOTE] FLIP-425: Asynchronous Execution Model

2024-03-29 Thread Xintong Song
+1 (binding) Best, Xintong On Fri, Mar 29, 2024 at 11:04 AM Yunfeng Zhou wrote: > +1 (non-bindling) > > Best, > Yunfeng > > On Wed, Mar 27, 2024 at 6:28 PM Yanfei Lei wrote: > > > > Hi everyone, > > > > Thanks for all the feedback about the FLIP-425: Asynchronous Execution > > Model [1].

Re: [VOTE] FLIP-424: Asynchronous State APIs

2024-03-29 Thread Xintong Song
+1 (binding) Best, Xintong On Fri, Mar 29, 2024 at 12:51 PM Yuepeng Pan wrote: > +1(non-binding) > > Best, > Yuepeng Pan > > > On 2024/03/29 03:03:53 Yunfeng Zhou wrote: > > +1 (non-binding) > > > > Best, > > Yunfeng > > > > On Wed, Mar 27, 2024 at 6:23 PM Zakelly Lan > wrote: > > > > > >

Re: [DISCUSS] Planning Flink 1.20

2024-03-25 Thread Xintong Song
ll. I'm looking > >> forward to getting more actively involved again. > >> > >> Cheers, > >> > >> Ufuk > >> > >> On Sun, Mar 24, 2024, at 11:27 AM, Ahmed Hamdy wrote: > >>> +1 for the proposed timeline and release managers. &g

Re: [DISCUSS] Planning Flink 1.20

2024-03-22 Thread Xintong Song
+1 for the proposed timeline and Weijie & Rui as the release managers. I think it would be welcomed if another 1-2 volunteers join as the release managers, but that's not a must. We used to have only 1-2 release managers for each release, Best, Xintong On Fri, Mar 22, 2024 at 2:55 PM Jark Wu

Re: [VOTE] FLIP-433: State Access on DataStream API V2

2024-03-20 Thread Xintong Song
+1 (binding) Best, Xintong On Wed, Mar 20, 2024 at 8:30 PM weijie guo wrote: > Hi everyone, > > > Thanks for all the feedback about the FLIP-433: State Access on > DataStream API V2 [1]. The discussion thread is here [2]. > > > The vote will be open for at least 72 hours unless there is an

Re: [DISCUSS] FLIP-423 ~FLIP-428: Introduce Disaggregated State Storage and Management in Flink 2.0

2024-03-18 Thread Xintong Song
keep minimal configurable items especially for the MVP. Given > that we have the opportunity to refine the functionality before the > framework transitions from @Experimental to @PublicEvolving, it makes sense > to refrain from presenting user-facing options until we have ensured > their ne

Re: [DISCUSS] FLIP-423 ~FLIP-428: Introduce Disaggregated State Storage and Management in Flink 2.0

2024-03-18 Thread Xintong Song
Sorry for joining the discussion late. I have two questions about FLIP-425. 1. Regarding Record-ordered and State-ordered of processElement. I understand that while State-ordered likely provides better performance, Record-ordered is sometimes required for correctness. The question is how should

Re: [ANNOUNCE] Apache Flink 1.19.0 released

2024-03-18 Thread Xintong Song
Congratulations~! Best, Xintong On Mon, Mar 18, 2024 at 7:02 PM Feng Jin wrote: > Congratulations! > > Best, > Feng > > On Mon, Mar 18, 2024 at 6:18 PM Yuepeng Pan wrote: > > > Congratulations! > > > > > > Thanks to release managers and everyone involved. > > > > > > > > > > Best, > >

Re: [VOTE] Release 1.19.0, release candidate #2

2024-03-11 Thread Xintong Song
+1 (binding) - verified signature and checksum - verified that source distribution does not contain binaries - built from source and played with example jobs, everything looks fine - reviewed the release announcement PR Best, Xintong On Thu, Mar 7, 2024 at 6:01 PM Lincoln Lee wrote: > Hi

Re: [DISCUSS] Move CheckpointingMode to flink-core

2024-02-28 Thread Xintong Song
Personally, I'd be in favor of option 2. And based on the fact that migrating from the deprecated CheckpointingMode to the new one takes barely any effort (simply re-import the class), I'd be fine with removing the deprecated class in 2.0. But I'd also be fine with the other options. Either way,

Re: [ANNOUNCE] Flink 1.19 Cross-team testing completed & sync summary on 02/27/2024

2024-02-27 Thread Xintong Song
Matthias has already said it on the release sync, but I still want to say it again. It's amazing how smooth the release testing goes for this release. Great thanks to the release managers and all contributors who make this happen. Best, Xintong On Wed, Feb 28, 2024 at 10:10 AM Lincoln Lee

Re: [VOTE] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-02-26 Thread Xintong Song
+1 (binding) Best, Xintong On Mon, Feb 26, 2024 at 6:10 PM weijie guo wrote: > Hi everyone, > > > Thanks for all the feedback about the FLIP-410: Config, Context and > Processing Timer Service of DataStream API V2 [1]. The discussion > thread is here [2]. > > > The vote will be open for at

Re: [VOTE] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-02-26 Thread Xintong Song
+1 (binding) Best, Xintong On Mon, Feb 26, 2024 at 6:09 PM weijie guo wrote: > Hi everyone, > > > Thanks for all the feedback about the FLIP-409: DataStream V2 Building > Blocks: DataStream, Partitioning and ProcessFunction [1]. The > discussion thread is here [2]. > > > The vote will be

Re: [VOTE] FLIP-408: [Umbrella] Introduce DataStream API V2

2024-02-26 Thread Xintong Song
+1 (binding) Best, Xintong On Mon, Feb 26, 2024 at 6:08 PM weijie guo wrote: > Hi everyone, > > > Thanks for all the feedback about the FLIP-408: [Umbrella] Introduce > DataStream API V2 [1]. The discussion thread is here [2]. > > > The vote will be open for at least 72 hours unless there

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-02-19 Thread Xintong Song
t; > > > This is to facilitate possible future extensions. But I thought it > > through, MetricGroup itself also plays the role of a manager. > > So I think you are right, I will add a `getMetricGroup` method directly > in > > `RuntimeContext`. > > > > Best regard

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-02-19 Thread Xintong Song
also updated accordingly. It would be nice to also mention those changes in > the FLIP-410 discussion thread. > > Yes, I've now mentioned those updates in the FLIP-410 discussion thread. > > > Best regards, > > Weijie > > > Xintong Song 于2024年2月5日周一 10:5

Re: [ANNOUNCE] Flink 1.19 feature freeze & sync summary on 01/30/2024

2024-02-04 Thread Xintong Song
users, the benefit of removing them > is to simplify Flink's code and reduce maintenance costs. > > If we just merge some user-related PRs recently, I could merge > it after 1.19. Thank you again~ > > Best, > Rui > > On Mon, Feb 5, 2024 at 12:21 PM Xintong Song > wr

Re: [ANNOUNCE] Flink 1.19 feature freeze & sync summary on 01/30/2024

2024-02-04 Thread Xintong Song
Hi Rui, Quick question, would there be any downside if this PR doesn't go into 1.19? Or any user benefit from getting it into this release? Best, Xintong On Sun, Feb 4, 2024 at 10:16 AM Rui Fan <1996fan...@gmail.com> wrote: > Hi release managers, > > > The feature freeze of 1.19 has started

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-02-04 Thread Xintong Song
FLIP has been updated. > > Best regards, > > Weijie > > > Xintong Song 于2024年2月1日周四 11:31写道: > > > OK, I see your point. > > > > I think the demand for updating states and emitting outputs upon > receiving > > a broadcast record makes sense.

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-01-31 Thread Xintong Song
need to call this method, unless they explicitly want to > > overwrite the timestamp. > > > > Make sense, I have updated this FLIP toward this new method name. > > > > > 2. While this method provides a way to set timestamps, how would users > &g

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-30 Thread Xintong Song
> > > How can users define custom metrics within the `ProcessFunction`? > Will there be a method like `getMetricGroup` available in the > `RuntimeContext`? > > I think this is a reasonable request. For extensibility, I have added the > getMetricManager instead of getMetricGroup to RuntimeContext,

Re: [VOTE] FLIP-331: Support EndOfStreamTrigger and isOutputOnlyAfterEndOfStream operator attribute to optimize task deployment

2024-01-30 Thread Xintong Song
+1 Best, Xintong On Wed, Jan 31, 2024 at 11:41 AM Xuannan Su wrote: > Hi everyone, > > Thanks for all the feedback about the FLIP-331: Support > EndOfStreamTrigger and isOutputOnlyAfterEndOfStream operator attribute > to optimize task deployment [1] [2]. > > I'd like to start a vote for it.

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-01-29 Thread Xintong Song
Just trying to understand. > Is there a particular reason we do not support a > `TwoInputProcessFunction` to combine a KeyedStream with a > BroadcastStream to result in a KeyedStream? There seems to be a valid > use case where a KeyedStream is enriched with a BroadcastStream and > returns a

Re: [DISCUSS] FLIP-408: [Umbrella] Introduce DataStream API V2

2024-01-29 Thread Xintong Song
Thanks for working on this, Weijie. The design flaws of the current DataStream API (i.e., V1) have been a pain for a long time. It's great to see efforts going on trying to resolve them. Significant changes to such an important and comprehensive set of public APIs deserves caution. From that

Re: Re: [DISCUSS] FLIP-331: Support EndOfStreamWindows and isOutputOnEOF operator attribute to optimize task deployment

2024-01-23 Thread Xintong Song
Sorry, I just sent an incomplete email draft by mistake. Thanks for reviving this, Xuannan. The updated FLIP LGTM. +1 for it. Best, Xintong On Tue, Jan 23, 2024 at 5:51 PM Xintong Song wrote: > Thanks > > Best, > > Xintong > > > > On Wed, Jan 10, 2024 at 5:56 PM

Re: Re: [DISCUSS] FLIP-331: Support EndOfStreamWindows and isOutputOnEOF operator attribute to optimize task deployment

2024-01-23 Thread Xintong Song
w assigner > in > > a DataStream program, without bothering users to decide whether it is > safe > > to treat data in a bounded source as backlog data. > > > > > > Regards, > > Dong > > > > > > > > > > > > > > On M

Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-08 Thread Xintong Song
+1 (binding) Best, Xintong On Tue, Jan 9, 2024 at 3:31 PM Benchao Li wrote: > +1 (non-binding) > > Feng Wang 于2024年1月9日周二 15:29写道: > > > > +1 non-binding > > Regards, > > Feng > > > > On Tue, Jan 9, 2024 at 3:05 PM Leonard Xu wrote: > > > > > Hello all, > > > > > > This is the official

Re: [VOTE] FLIP-405: Migrate string configuration key to ConfigOption

2024-01-08 Thread Xintong Song
+1 (binding) Best, Xintong On Mon, Jan 8, 2024 at 1:48 PM Hang Ruan wrote: > +1(non-binding) > > Best, > Hang > > Rui Fan <1996fan...@gmail.com> 于2024年1月8日周一 13:04写道: > > > +1(binding) > > > > Best, > > Rui > > > > On Mon, Jan 8, 2024 at 1:00 PM Xuannan Su wrote: > > > > > Hi everyone, > >

Re: [2.0] Help needed for release 2.0 work items

2024-01-02 Thread Xintong Song
1405707256). It > would be beneficial to offer a more intuitive and straightforward CLI > command to enhance usability. > > > Best, > Zakelly > > On Wed, Jan 3, 2024 at 10:45 AM Xintong Song > wrote: > > > Thanks a lot for offering the help, Rui. The plan s

Re: [2.0] Help needed for release 2.0 work items

2024-01-02 Thread Xintong Song
@Deprecated in 1.19 (Must do in 1.19) > - Refactor all usages of them to Java's Duration(Nice do in 1.19, must do > in 1.20) > - Remove them in 2.0 > > Is this plan reasonable? > > [1] https://issues.apache.org/jira/browse/FLINK-14068 > > Best, > Rui > > On Wed, Jan

Re: [ANNOUNCE] New Apache Flink Committer - Alexander Fedulov

2024-01-02 Thread Xintong Song
Congratulations~! Best, Xintong On Wed, Jan 3, 2024 at 9:36 AM Leonard Xu wrote: > Congrats, Alex! > > Best, > Leonard > > > 2024年1月3日 上午4:11,Tang, Zhiyan (udx2na) 写道: > > > > big congrats to Alex and Happy New Year everyone! > > > > Best > > Tony > > > > Get Outlook for

[2.0] Help needed for release 2.0 work items

2024-01-02 Thread Xintong Song
Hi devs, The release managers have been tracking the progress of release 2.0 work items. Unfortunately, some of the items are not in good progress, and either don't have a contributor or the original contributor no longer has capacity to work on them. We have already tried reaching out to some

Re: [VOTE] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-26 Thread Xintong Song
+1 (binding) Best, Xintong On Wed, Dec 27, 2023 at 2:54 PM Yong Fang wrote: > Hi devs, > > Thanks for all feedback about the FLIP-398: Improve Serialization > Configuration And Usage In Flink [1] which has been discussed in [2]. > > I'd like to start a vote for it. The vote will be open

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

2023-12-26 Thread Xintong Song
n, > T value)` as well. +1 Best, Xintong On Tue, Dec 26, 2023 at 8:44 PM Xintong Song wrote: > These features don't have a public option, but they work. I'm not sure >> whether these features are used by some advanced users. >> Actually, I think some of

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

2023-12-26 Thread Xintong Song
gt; > > question why would we keep the discouraged interfaces at all. I would > > > suggest the following: > > > ``` > > > We encourage users and developers to always use ConfigOption for > getting > > / > > > setting the configurations if possible, for it

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

2023-12-26 Thread Xintong Song
Thanks for the efforts, Rui and Xuannan. I think it's a good idea to migrate string-key configuration accesses to ConfigOption-s in general. I have a few suggestions / questions regarding the FLIP. 1. I think the default value for `TASK_MANAGER_LOG_PATH_KEY` should be "no default". We can

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-20 Thread Xintong Song
lic class(or Api), > is voting necessary? > > Best, > Rui > > On Wed, 20 Dec 2023 at 16:55, Xintong Song wrote: > > > Sounds good to me. > > > > Best, > > > > Xintong > > > > > > > > On Wed, Dec 20, 2023 at 9:40 AM Xuannan Su

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-20 Thread Xintong Song
keep "String getString(String key, String defaultValue)" or not. > > (It's not a right or wrong thing.) > > > > Judging from the discussion, most discussants can accept that keeping > > `String getString(String key, String defaultValue)` and depreate the > > rest of `ge

Re: [VOTE] FLIP-382: Unify the Provision of Diverse Metadata for Context-like APIs

2023-12-17 Thread Xintong Song
+1 (binding) Best, Xintong On Mon, Dec 18, 2023 at 3:05 PM Lijie Wang wrote: > +1 (binding) > > Best, > Lijie > > Yuxin Tan 于2023年12月15日周五 17:14写道: > > > +1 (non-binding) > > > > Best, > > Yuxin > > > > > > weijie guo 于2023年12月15日周五 10:05写道: > > > > > +1(binding) > > > > > > Best regards,

Re: [VOTE] FLIP-380: Support Full Partition Processing On Non-keyed DataStream

2023-12-17 Thread Xintong Song
+1 (binding) Best, Xintong On Fri, Dec 15, 2023 at 5:15 PM weijie guo wrote: > +1 (binding) > > Best regards, > > Weijie > > > Wencong Liu 于2023年12月15日周五 09:15写道: > > > Hi dev, > > > > I'd like to start a vote on FLIP-380. > > > > Discussion thread: > >

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-17 Thread Xintong Song
Hi Ken, I think the main purpose of this FLIP is to change how users interact with the knobs for customizing the serialization behaviors, from requiring code changes to working with pure configurations. Redesigning the knobs (i.e., names, semantics, etc.), on the other hand, is not the purpose of

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-14 Thread Xintong Song
configOption), and completely > > > delete all getXxx(String key, String defaultValue) in 2.0. > > > And I'm willing to pick it up~ > > > > > > To Xintong: > > > > > > > I think a `toMap` as suggested by Zhu and Xuannan should solve the > >

Re: [VOTE] FLIP-383: Support Job Recovery for Batch Jobs

2023-12-13 Thread Xintong Song
+1 (binding) Best, Xintong On Thu, Dec 14, 2023 at 3:15 PM Lijie Wang wrote: > Hi devs, Thanks for all feedback about the FLIP-383: Support Job Recovery > for Batch Jobs[1]. This FLIP was discussed in [2]. > > I'd like to start a vote for it. The vote will be open for at least 72 > hours

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Xintong Song
the > > Configuration. > > Yeah, `toMap` can solve the problem, and I also mentioned it in the > initial mail. > > Also Providing an iterator is fine, but we don't have a strong > requirement for now. Simple is better, how about considering it > if we have other strong

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Xintong Song
I'm leaning towards not allowing string-key based configuration access in the long term. I see the Configuration being used in two different ways: 1. Writing / reading a specific option. In such cases, ConfigOption has many advantages compared to string-key, such as explicit value type,

Re: [VOTE] FLIP-396: Trial to test GitHub Actions as an alternative for Flink's current Azure CI infrastructure

2023-12-13 Thread Xintong Song
+1 (binding) Best, Xintong On Thu, Dec 14, 2023 at 10:05 AM Jiabao Sun wrote: > Thanks Matthias for this hard work! > > +1(non-binding) > > Best, > Jiabao > > > > 2023年12月14日 09:57,Leonard Xu 写道: > > > > +1(binding) > > > > > > Best, > > Leonard > > > >> 2023年12月13日 下午10:59,Benchao Li 写道:

Re: [VOTE] FLIP-335: Deprecation and removal of Flink's Time classes in favor of Java's Duration

2023-12-12 Thread Xintong Song
+1 (binding) I guess I missed the FLIP discussion. But the FLIP LGTM. Thanks for driving this, Matthias. Best, Xintong On Tue, Dec 12, 2023 at 8:33 PM Jing Ge wrote: > +1(binding) > > Thanks. > > Best regards, > Jing > > On Tue, Dec 12, 2023 at 1:12 PM Zhu Zhu wrote: > > > +1 (binding) >

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-11 Thread Xintong Song
Thanks for working on this, Yong. The usability of the serialization mechanism has indeed been a pain for a long time. The proposed changes look good to me. +1 from my side. Best, Xintong On Thu, Dec 7, 2023 at 10:36 AM Yong Fang wrote: > Hi devs, > > I'd like to start a discussion about

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-06 Thread Xintong Song
A big +1 for this proposal. Thanks Leonard and the Flink CDC community. While there are many further details to be discussed, I believe this proposal is aligned with the long-term interests for both communities. Best, Xintong On Thu, Dec 7, 2023 at 12:06 PM Samrat Deb wrote: > That's

Re: [DISCUSS] FLIP-383: Support Job Recovery for Batch Jobs

2023-12-05 Thread Xintong Song
://cwiki.apache.org/confluence/display/FLINK/FLIP-194%3A+Introduce+the+JobResultStore On Tue, Dec 5, 2023 at 6:01 PM Xintong Song wrote: > Thanks for addressing my comments, Lijie. LGTM > > Best, > > Xintong > > > > On Tue, Dec 5, 2023 at 2:56 PM Paul Lam wrote: > >> Hi

Re: [DISCUSS] FLIP-383: Support Job Recovery for Batch Jobs

2023-12-05 Thread Xintong Song
Thanks for addressing my comments, Lijie. LGTM Best, Xintong On Tue, Dec 5, 2023 at 2:56 PM Paul Lam wrote: > Hi Lijie, > > Recovery for batch jobs is no doubt a long-awaited feature. Thanks for > the proposal! > > I’m concerned about the multi-job scenario. In session mode, users could >

Re: [DISCUSS] FLIP-383: Support Job Recovery for Batch Jobs

2023-12-03 Thread Xintong Song
Thanks for the proposal, Lijie and Zhu. I have been having offline discussions with the Apache Celeborn folks regarding integrating Apache Celeborn into Flink's Hybrid Shuffle mode. One thing coming from those discussions that might relate to this FLIP is that Celeborn maintains some internal

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

2023-11-29 Thread Xintong Song
Thanks for the efforts, Matthias. I think it would be helpful if we can at the end migrate the CI to an ASF-managed Github Action, as long as it provides us a similar computation capacity and stability. Given that the proposal is only to start a trial and investigate whether the migration is

Re: [VOTE] Release flink-shaded 16.2, release candidate #1

2023-11-14 Thread Xintong Song
+1 (binding) - verified checksum and signature - verified license and notice file - verified source archive does not contain any binary - reviewed the release notes and web pr Best, Xintong On Mon, Nov 13, 2023 at 4:21 PM Yuxin Tan wrote: > Thanks weijie for driving the new release! > > +1

Re: [DISCUSS] Planning Flink 1.19

2023-10-29 Thread Xintong Song
Thanks for kicking this off. +1 for the proposed release managers (Lincoln, Yun, Jing and Martijn) and targeting date (feature freeze: Jan 26). I'd like to bring up that it is likely many efforts in the 1.19 release cycle are also related to the 2.0 release. I think it would be better if the

Re: [VOTE] FLIP-366: Support standard YAML for FLINK configuration

2023-10-16 Thread Xintong Song
+1 (binding) Best, Xintong On Mon, Oct 16, 2023 at 7:23 PM Chesnay Schepler wrote: > +1 (binding) > > On 13/10/2023 04:12, Junrui Lee wrote: > > Hi all, > > > > Thank you to everyone for the feedback on FLIP-366[1]: Support standard > > YAML for FLINK configuration in the discussion thread

Re: [VOTE] FLIP-327: Support switching from batch to stream mode to improve throughput when processing backlog data

2023-09-24 Thread Xintong Song
+1 (binding) Best, Xintong On Sat, Sep 23, 2023 at 10:16 PM Yuepeng Pan wrote: > +1(non-binding), thank you for driving this proposal. > > Best, > Yuepeng Pan. > At 2023-09-22 14:07:45, "Dong Lin" wrote: > >Hi all, > > > >We would like to start the vote for FLIP-327: Support switching from

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

2023-09-18 Thread Xintong Song
Thanks for addressing my comments. LGTM Best, Xintong On Mon, Sep 18, 2023 at 3:10 PM Dong Lin wrote: > Hi Xintong, > > Thank you for all the comments. Please see my reply inline. > > > On Mon, Sep 18, 2023 at 11:31 AM Xintong Song > wrote: > > > Thanks fo

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

2023-09-17 Thread Xintong Song
more effort when reworking this with the new API. The rest of your replies make sense to me. Best, Xintong On Fri, Sep 15, 2023 at 10:05 PM Dong Lin wrote: > Hi Xintong, > > Thanks for your comments! Please see my reply inline. > > On Thu, Sep 14, 2023 at 4:58 PM Xintong Son

Re: Re: [DISCUSS] FLIP-331: Support EndOfStreamWindows and isOutputOnEOF operator attribute to optimize task deployment

2023-09-17 Thread Xintong Song
Thanks for addressing my comments, Dong. LGTM. Best, Xintong On Sat, Sep 16, 2023 at 3:34 PM Wencong Liu wrote: > Hi Dong & Jinhao, > > Thanks for your clarification! +1 > > Best regards, > Wencong > > > > > > > > > > > > > > > > > > At 2023-09-15 11:26:16, "Dong Lin" wrote: > >Hi

Re: [VOTE] FLIP-357: Deprecate Iteration API of DataStream

2023-09-14 Thread Xintong Song
+1 (binding) Best, Xintong On Thu, Sep 14, 2023 at 3:48 PM Jing Ge wrote: > +1(binding) > > Best regards, > Jing > > On Thu, Sep 14, 2023 at 7:31 AM Dong Lin wrote: > > > Thanks Wencong for the FLIP. > > > > +1 (binding) > > > > On Thu, Sep 14, 2023 at 12:36 PM Wencong Liu > wrote: > > >

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

2023-09-14 Thread Xintong Song
Sorry to join the discussion late. Overall, I think it's a good idea to support dynamically switching the operator algorithms between Streaming (optimized towards low latency + checkpointing supports) and Batch (optimized towards throughput). This is indeed a big and complex topic, and I really

Re: [DISCUSS] FLIP-331: Support EndOfStreamWindows and isOutputOnEOF operator attribute to optimize task deployment

2023-09-14 Thread Xintong Song
Thanks for preparing this FLIP, Dong & Jinhao. I'm overall +1 to this proposal. This is helpful for some cases that we are dealing with. - Wencong and I are preparing guidelines for migrating from DataSet API to DataStream API. We noticed that users have to define a custom trigger in order to

Re: [VOTE] FLIP-361: Improve GC Metrics

2023-09-13 Thread Xintong Song
+1 (binding) Best, Xintong On Thu, Sep 14, 2023 at 2:40 AM Samrat Deb wrote: > +1 ( non binding) > > These improved GC metrics will be a great addition. > > Bests, > Samrat > > On Wed, 13 Sep 2023 at 7:58 PM, ConradJam wrote: > > > +1 (non-binding) > > gc metrics help with autoscale tuning

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Xintong Song
; > Gyula > > On Wed, Sep 6, 2023 at 10:21 AM Xintong Song > wrote: > > > Thank you :) > > > > Best, > > > > Xintong > > > > > > > > On Wed, Sep 6, 2023 at 4:17 PM Gyula Fóra wrote: > > > > > Makes sense Xintong

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Xintong Song
Thank you :) Best, Xintong On Wed, Sep 6, 2023 at 4:17 PM Gyula Fóra wrote: > Makes sense Xintong, I am happy to extend the proposal with the average gc > time metric +1 > > Gyula > > On Wed, Sep 6, 2023 at 10:09 AM Xintong Song > wrote: > > > > > > &

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Xintong Song
? > > .TimePerGc or .AverageTime would make sense. > > AverageTime may be a bit nicer :) > > > > My only concern is how useful this will be in reality. If there are only > > (or several) long pauses then the msPerSec metrics will show it already, > > and if there is a

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Xintong Song
Thanks for bringing this up, Gyula. The proposed changes make sense to me. +1 for them. In addition to the proposed changes, I wonder if we should also add something like timePerGc? This would help understand whether there are long pauses, due to GC STW, that may lead to rpc unresponsiveness and

Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

2023-09-01 Thread Xintong Song
+1 Best, Xintong On Fri, Sep 1, 2023 at 1:11 PM Dong Lin wrote: > Thanks Wencong for initiating the discussion. > > +1 for the proposal. > > On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu wrote: > > > Hi devs, > > > > I would like to start a discussion on FLIP-357: Deprecate Iteration API >

Re: [DISCUSS][2.0] Deprecating Accumulator in favor of MetricsGroup

2023-08-28 Thread Xintong Song
Thanks for bringing this up, Matthias. One thing that a user may achieve with an accumulator but not with a metric group is to programmatically fetch the job execution result, rather than outputting the results to an external sink, in attached mode. This can also be achieved by using CollectSink,

Re: [DISCUSS][2.0] Evaluation of imperative API related TBD work items

2023-08-21 Thread Xintong Song
As no concerns are raised, I'll update the wiki page. Best, Xintong On Thu, Aug 17, 2023 at 5:31 PM Xintong Song wrote: > Hi devs, > > We have been evaluating the 3 TBD items from the Imperative API section of > the release 2.0 work items [1]. > 1. Refactor the API module

[DISCUSS][2.0] Evaluation of imperative API related TBD work items

2023-08-17 Thread Xintong Song
Hi devs, We have been evaluating the 3 TBD items from the Imperative API section of the release 2.0 work items [1]. 1. Refactor the API modules 2. Refactor context-like APIs 3. Convert user-facing concrete classes into interfaces. *We propose to remove 1 & 3 from the list *(3 seems subsumed by

Re: [DISCUSS] Update Flink Roadmap

2023-08-14 Thread Xintong Song
Thanks for driving this, Jark. The current draft looks good to me. I think it is good to open a PR with it. And if there are other comments, we can discuss them during the PR review. I also added a few minor comments in the draft regarding the feature radar. Those can also be discussed on the

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-14 Thread Xintong Song
tus regardless. > > For the reasons above, I prefer introducing the configuration as is, and > change it later with the a deprecation process or migration process. What > do you think? > > Best, > Xuannan > On Aug 14, 2023, 14:09 +0800, Xintong Song , wrote: > > Thanks

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-14 Thread Xintong Song
se let me know if you have further questions. > > Best, > Xuannan > On Aug 10, 2023, 20:23 +0800, Xintong Song , wrote: > > Thanks for preparing the FLIP, Xuannan. > > > > +1 in general. > > > > A quick question, could you explain why we are relying on the wa

Re: [DISCUSS] FLIP-330: Support specifying record timestamp requirement

2023-08-11 Thread Xintong Song
rformance for batch-only jobs in the long term. > > What do you think? > > Best, > Dong > > > On Thu, Aug 10, 2023 at 9:16 PM Xintong Song > wrote: > > > Hi Yunfeng, > > > > Thanks for preparing this FLIP. I'm respectful for the efforts you > al

Re: [DISCUSS] FLIP-330: Support specifying record timestamp requirement

2023-08-10 Thread Xintong Song
Hi Yunfeng, Thanks for preparing this FLIP. I'm respectful for the efforts you already put into the PoC implementation and benchmarks. However, I have to say I'm quite concerned about this proposal. 1. The FLIP is based on the assumption that in non-timestamp scenarios StreamRecord is the only

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-10 Thread Xintong Song
Thanks for preparing the FLIP, Xuannan. +1 in general. A quick question, could you explain why we are relying on the watermark for emitting the record attribute? Why not use timestamps in the records? I don't see any concern in using watermarks. Just wondering if there's any deep considerations

Re: [DISCUSS] How about adding OLAP to Flink Roadmap?

2023-08-08 Thread Xintong Song
Thanks for bringing this up, Shammon. In general, I'd be +1 to improve Flink's ability to serve as an OLAP engine. I see a great value in Flink becoming a unified Large-scale Data Processing / Analysis tool. Through my interactions with users (Alibaba internal users, external users on Alibaba

[ANNOUNCE] New Apache Flink Committer - Weihua Hu

2023-08-03 Thread Xintong Song
Hi everyone, On behalf of the PMC, I'm very happy to announce Weihua Hu as a new Flink Committer! Weihua has been consistently contributing to the project since May 2022. He mainly works in Flink's distributed coordination areas. He is the main contributor of FLIP-298 and many other improvements

[ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

2023-08-03 Thread Xintong Song
Hi everyone, On behalf of the PMC, I'm very happy to announce that Matthias Pohl has joined the Flink PMC! Matthias has been consistently contributing to the project since Sep 2020, and became a committer in Dec 2021. He mainly works in Flink's distributed coordination and high availability

Re: [VOTE] FLIP-333: Redesign Apache Flink website

2023-08-03 Thread Xintong Song
+1 (binding) Best, Xintong On Fri, Aug 4, 2023 at 3:05 AM Hong Liang wrote: > +1 (binding) > > Thanks Deepthi! > > > > On Thu, Aug 3, 2023 at 7:44 PM Danny Cranmer > wrote: > > > +1 (binding) > > > > Thanks Deepthi > > > > > > On Thu, 3 Aug 2023, 12:03 Rui Fan, <1996fan...@gmail.com>

Release 2.0 Status Updates - 08/01/2023

2023-07-31 Thread Xintong Song
Hi devs, As the release 2.0 must-have items being voted and approved, the release managers recently had another discussion on how to move forward. Here's the summary. ## Follow-up of previous discussions - As a follow-up of deciding the must-have items [1], Martijn will help draft an

[RESULT][VOTE] Release 2.0 must-have work items - Round 2

2023-07-27 Thread Xintong Song
Hi devs, I'm glad to announce that the must-have work items for release 2.0 [1] have been approved. The voting thread is [2] and the discussions can be found in [3][4]. There are 11 approving votes, 7 of which are binding: - Xintong Song (binding) - Yuan Mei (binding) - Yu Li (binding) - Leonard

Re: [VOTE] Release 2.0 must-have work items - Round 2

2023-07-27 Thread Xintong Song
Thanks all for participating. I'm closing this vote in another thread. Best, Xintong On Thu, Jul 27, 2023 at 11:37 AM Jiabao Sun wrote: > + 1 (non-binding) > > Thanks Xintong for driving this. > > Best, > Jiabao > > > On 2023/07/20 09:22:46 Xintong Song wrote: &

Re: [DISCUSS] Proposing an LTS Release for the 1.x Line

2023-07-26 Thread Xintong Song
t; released? > > > > > > 2. How long are we going to support 1.x LTS? 1 year? 2 years? As > > long > > > > as > > > > > > the issues that block users from migrating to 2.0 are not solved, > > we > > >

Re: [VOTE][2.0] FLIP-343: Remove parameter in WindowAssigner#getDefaultTrigger()

2023-07-26 Thread Xintong Song
t; Jing > > On Wed, Jul 26, 2023 at 5:49 PM Matthias Pohl > wrote: > > > +1 (binding) > > > > On Wed, Jul 26, 2023 at 10:11 AM Yuxin Tan > wrote: > > > > > +1 (non-binding) > > > > > > Best, > > > Yuxin > >

Re: [DISCUSS][2.0] FLIP-351: REST API normalizes +/-Inf / NaN to 0

2023-07-26 Thread Xintong Song
rest/messages/job/metrics/IOMetricsInfo.java > > > > > > > > > ./flink-runtime/src/main/java/org/apache/flink/runtime/rest/messages/JobVertexBackPressureInfo.java > > > > > > > > > ./flink-runtime/src/main/java/org/apache/flink/runtime/rest/message

Re: Re: Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-26 Thread Xintong Song
gt; > > > > >> > > > > >> At 2023-07-21 18:05:26, "Jing Ge" > > wrote: > > > > >> >Hi Wencong, > > > > >> > > > > > >> >Just out of curiosity, will the newly introduced > > > > >> >deseriali

Re: [VOTE][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-26 Thread Xintong Song
+1 (binding) Best, Xintong On Wed, Jul 26, 2023 at 2:29 PM Wencong Liu wrote: > Hi dev, > > > I'd like to start a vote on FLIP-347. > > > Discussion thread: > https://lists.apache.org/thread/3gcxhnqpsvb85golnlxf9tv5p43xkjgj > FLIP: >

Re: [VOTE][2.0] FLIP-344: Remove parameter in RichFunction#open

2023-07-26 Thread Xintong Song
+1 (binding) Best, Xintong On Wed, Jul 26, 2023 at 2:26 PM Wencong Liu wrote: > Hi dev, > > > I'd like to start a vote on FLIP-344. > > > Discussion thread: > https://lists.apache.org/thread/5lyjrrdtwkngkol2t541r4xwoh7133km > FLIP: >

Re: [VOTE][2.0] FLIP-343: Remove parameter in WindowAssigner#getDefaultTrigger()

2023-07-26 Thread Xintong Song
+1 (binding) Best, Xintong On Wed, Jul 26, 2023 at 3:35 PM Yuepeng Pan wrote: > +1 (non-binding) > > Thanks. > > Yuepeng Pan. > At 2023-07-26 14:26:04, "Wencong Liu" wrote: > >Hi dev, > > > > > >I'd like to start a vote on FLIP-343. > > > > > >Discussion thread: >

Re: [DISCUSS] Feature freeze and deprecation work for 2.0

2023-07-25 Thread Xintong Song
lease cycle, considering FLIP-321 and 2.0 working >> items are finalized quite nearing the feature freeze of 1.18? >> >> Best, >> Qingsheng, Jing, Konstantin and Sergey >> >> On Fri, Jul 21, 2023 at 5:28 PM Xintong Song >> wrote: >> >>> Good

Re: [VOTE] Release 2.0 must-have work items - Round 2

2023-07-25 Thread Xintong Song
ei wrote: > > > > > +1 (binding) > > > > > > Thanks for driving the discussion through and for all the efforts in > > > resolving the complexities :-) > > > > > > Best > > > Yuan > > > > > > On Thu, Jul 20, 2023 a

  1   2   3   4   5   6   7   8   9   10   >