Re: [DISCUSS] FLIP-445: Support dynamic parallelism inference for HiveSource

2024-04-17 Thread Lijie Wang
Thanks for driving the discussion. +1 for the proposal and +1 for the `InferMode.NONE` option. Best, Lijie Ron liu 于2024年4月18日周四 11:36写道: > Hi, Xia > > Thanks for driving this FLIP. > > This proposal looks good to me overall. However, I have the following minor > questions: > > 1. FLIP

[jira] [Created] (FLINK-35151) Flink mysql cdc will stuck when suspend binlog split and ChangeEventQueue is full

2024-04-17 Thread Xin Gong (Jira)
Xin Gong created FLINK-35151: Summary: Flink mysql cdc will stuck when suspend binlog split and ChangeEventQueue is full Key: FLINK-35151 URL: https://issues.apache.org/jira/browse/FLINK-35151 Project:

Re: [DISCUSS] FLIP-445: Support dynamic parallelism inference for HiveSource

2024-04-17 Thread Ron liu
Hi, Xia Thanks for driving this FLIP. This proposal looks good to me overall. However, I have the following minor questions: 1. FLIP introduced `table.exec.hive.infer-source-parallelism.mode` as a new parameter, and the value is the enum class `InferMode`, I think the InferMode class should

[jira] [Created] (FLINK-35150) The specified upload does not exist. The upload ID may be invalid

2024-04-17 Thread qyw (Jira)
qyw created FLINK-35150: --- Summary: The specified upload does not exist. The upload ID may be invalid Key: FLINK-35150 URL: https://issues.apache.org/jira/browse/FLINK-35150 Project: Flink Issue Type:

Re: [VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Yun Tang
+1 (binding) Best, Yun Tang From: Jark Wu Sent: Thursday, April 18, 2024 9:54 To: dev@flink.apache.org Subject: Re: [VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines +1 (binding) Best, Jark On Wed, 17 Apr 2024 at 20:52,

Re:[VOTE] Release flink-connector-jdbc v3.2.0, release candidate #1

2024-04-17 Thread Yuepeng Pan
+1 (non-binding) - Checked source-build for source code tag v3.2.0-rc1 - Did test for some examples with mysql 5.8. - Checked release note page. Thanks for driving it! Best, Yuepeng Pan At 2024-04-17 18:02:06, "Danny Cranmer" wrote: >Hi everyone, > >Please review and vote on the release

[jira] [Created] (FLINK-35149) Fix DataSinkTranslator#sinkTo ignoring pre-write topology if not TwoPhaseCommittingSink

2024-04-17 Thread Hongshun Wang (Jira)
Hongshun Wang created FLINK-35149: - Summary: Fix DataSinkTranslator#sinkTo ignoring pre-write topology if not TwoPhaseCommittingSink Key: FLINK-35149 URL: https://issues.apache.org/jira/browse/FLINK-35149

Re: [VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Jark Wu
+1 (binding) Best, Jark On Wed, 17 Apr 2024 at 20:52, Leonard Xu wrote: > +1(binding) > > Best, > Leonard > > > 2024年4月17日 下午8:31,Lincoln Lee 写道: > > > > +1(binding) > > > > Best, > > Lincoln Lee > > > > > > Ferenc Csaky 于2024年4月17日周三 19:58写道: > > > >> +1 (non-binding) > >> > >> Best, > >>

[jira] [Created] (FLINK-35148) Improve InstantiationUtil for checking nullary public constructor

2024-04-17 Thread Mingliang Liu (Jira)
Mingliang Liu created FLINK-35148: - Summary: Improve InstantiationUtil for checking nullary public constructor Key: FLINK-35148 URL: https://issues.apache.org/jira/browse/FLINK-35148 Project: Flink

[jira] [Created] (FLINK-35147) SinkMaterializer throws StateMigrationException when widening the field type in the output table

2024-04-17 Thread Sharon Xie (Jira)
Sharon Xie created FLINK-35147: -- Summary: SinkMaterializer throws StateMigrationException when widening the field type in the output table Key: FLINK-35147 URL: https://issues.apache.org/jira/browse/FLINK-35147

[jira] [Created] (FLINK-35146) CompileAndExecuteRemotePlanITCase.testCompileAndExecutePlan

2024-04-17 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-35146: --- Summary: CompileAndExecuteRemotePlanITCase.testCompileAndExecutePlan Key: FLINK-35146 URL: https://issues.apache.org/jira/browse/FLINK-35146 Project: Flink

Re: [VOTE] Release flink-connector-aws v4.3.0, release candidate #1

2024-04-17 Thread Danny Cranmer
Hey, Thanks for root causing this. I agree this needs to be fixed. -1 binding Thanks, Danny On Wed, 17 Apr 2024, 15:43 Aleksandr Pilipenko, wrote: > Hi Danny, > > There is a blocker bug reported in Kinesis connector [1] > We should pause the release until it is resolved. > > [1] >

Re: [VOTE] Release flink-connector-aws v4.3.0, release candidate #1

2024-04-17 Thread Aleksandr Pilipenko
Hi Danny, There is a blocker bug reported in Kinesis connector [1] We should pause the release until it is resolved. [1] https://issues.apache.org/jira/browse/FLINK-35115 Thanks, Aleksandr

[jira] [Created] (FLINK-35145) Add timeout for cluster termination

2024-04-17 Thread Zhanghao Chen (Jira)
Zhanghao Chen created FLINK-35145: - Summary: Add timeout for cluster termination Key: FLINK-35145 URL: https://issues.apache.org/jira/browse/FLINK-35145 Project: Flink Issue Type:

Re: [VOTE] Release flink-connector-mongodb v1.2.0, release candidate #1

2024-04-17 Thread Jeyhun Karimov
Thanks for driving this. +1 (non-binding) - Validated checksum hash - Verified signature - Tag is present - Reviewed web PR Regards, Jeyhun On Wed, Apr 17, 2024 at 3:26 PM gongzhongqiang wrote: > +1 (non-binding) > > - Flink website pr reviewed > - Check source code without binary files > -

Re: [VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-17 Thread Jeyhun Karimov
+1 (non binding) Regards, Jeyhun On Wed, Apr 17, 2024 at 2:22 PM Zhu Zhu wrote: > +1 (binding) > > Thanks, > Zhu > > Yuxin Tan 于2024年4月17日周三 18:36写道: > > > +1 (non-binding) > > > > Best, > > Yuxin > > > > > > Zakelly Lan 于2024年4月17日周三 16:51写道: > > > > > +1 binding > > > > > > > > > Best, > >

Re: [ANNOUNCE] New Apache Flink Committer - Zakelly Lan

2024-04-17 Thread gongzhongqiang
Congratulations, Zakelly! Best, Zhongqiang Gong Yuan Mei 于2024年4月15日周一 10:51写道: > Hi everyone, > > On behalf of the PMC, I'm happy to let you know that Zakelly Lan has become > a new Flink Committer! > > Zakelly has been continuously contributing to the Flink project since 2020, > with a

Re: [VOTE] Release flink-connector-mongodb v1.2.0, release candidate #1

2024-04-17 Thread gongzhongqiang
+1 (non-binding) - Flink website pr reviewed - Check source code without binary files - Validated checksum hash and signature - CI passed with tag v1.2.0-rc1 Best, Zhongqiang Gong Danny Cranmer 于2024年4月17日周三 18:44写道: > Hi everyone, > > Please review and vote on the release candidate #1 for

Re: [VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Leonard Xu
+1(binding) Best, Leonard > 2024年4月17日 下午8:31,Lincoln Lee 写道: > > +1(binding) > > Best, > Lincoln Lee > > > Ferenc Csaky 于2024年4月17日周三 19:58写道: > >> +1 (non-binding) >> >> Best, >> Ferenc >> >> >> >> >> On Wednesday, April 17th, 2024 at 10:26, Ahmed Hamdy >> wrote: >> >>> >>> >>>

Re: Question around Flink's AdaptiveBatchScheduler

2024-04-17 Thread Junrui Lee
Thanks Venkata and Xia for providing further clarification. I think your example illustrates the significance of this proposal very well. Please feel free go ahead and address the concerns. Best, Junrui Venkatakrishnan Sowrirajan 于2024年4月16日周二 07:01写道: > Thanks for adding your thoughts to this

Re: [VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Lincoln Lee
+1(binding) Best, Lincoln Lee Ferenc Csaky 于2024年4月17日周三 19:58写道: > +1 (non-binding) > > Best, > Ferenc > > > > > On Wednesday, April 17th, 2024 at 10:26, Ahmed Hamdy > wrote: > > > > > > > + 1 (non-binding) > > > > Best Regards > > Ahmed Hamdy > > > > > > On Wed, 17 Apr 2024 at 08:28,

Re: [VOTE] Release flink-connector-mongodb v1.2.0, release candidate #1

2024-04-17 Thread Jiabao Sun
Thanks Danny for driving this. +1 (non-binding) - Validated checksum hash - Verified signature - Tag is present - Build successful with jdk8, jdk11 and jdk17 - Checked the dist jar was built by jdk8 - Reviewed web PR Best, Jiabao Danny Cranmer 于2024年4月17日周三 18:39写道: > Hi everyone, > > Please

Re: [VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-17 Thread Zhu Zhu
+1 (binding) Thanks, Zhu Yuxin Tan 于2024年4月17日周三 18:36写道: > +1 (non-binding) > > Best, > Yuxin > > > Zakelly Lan 于2024年4月17日周三 16:51写道: > > > +1 binding > > > > > > Best, > > Zakelly > > > > On Wed, Apr 17, 2024 at 2:05 PM Rui Fan <1996fan...@gmail.com> wrote: > > > > > +1(binding) > > > > >

Re: [VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Ferenc Csaky
+1 (non-binding) Best, Ferenc On Wednesday, April 17th, 2024 at 10:26, Ahmed Hamdy wrote: > > > + 1 (non-binding) > > Best Regards > Ahmed Hamdy > > > On Wed, 17 Apr 2024 at 08:28, Yuepeng Pan panyuep...@apache.org wrote: > > > +1(non-binding). > > > > Best, > > Yuepeng Pan > > > >

[jira] [Created] (FLINK-35143) Expose newly added tables capture in mysql pipeline connector

2024-04-17 Thread Hongshun Wang (Jira)
Hongshun Wang created FLINK-35143: - Summary: Expose newly added tables capture in mysql pipeline connector Key: FLINK-35143 URL: https://issues.apache.org/jira/browse/FLINK-35143 Project: Flink

[VOTE] Release flink-connector-mongodb v1.2.0, release candidate #1

2024-04-17 Thread Danny Cranmer
Hi everyone, Please review and vote on the release candidate #1 for v1.2.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) This release supports Flink 1.18 and 1.19. The complete staging area is available for your review, which

[jira] [Created] (FLINK-35144) Support multi source sync for FlinkCDC

2024-04-17 Thread Congxian Qiu (Jira)
Congxian Qiu created FLINK-35144: Summary: Support multi source sync for FlinkCDC Key: FLINK-35144 URL: https://issues.apache.org/jira/browse/FLINK-35144 Project: Flink Issue Type:

Re: [DISCUSS] Connector releases for Flink 1.19

2024-04-17 Thread Ferenc Csaky
Thank you Danny and Sergey for pushing this! I can help with the HBase connector if necessary, will comment the details to the relevant Jira ticket. Best, Ferenc On Wednesday, April 17th, 2024 at 11:17, Danny Cranmer wrote: > > > Hello all, > > I have created a parent Jira to cover the

[jira] [Created] (FLINK-35141) Release flink-connector-pulsar vX.X.X for Flink 1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35141: - Summary: Release flink-connector-pulsar vX.X.X for Flink 1.19 Key: FLINK-35141 URL: https://issues.apache.org/jira/browse/FLINK-35141 Project: Flink Issue

[jira] [Created] (FLINK-35134) Release flink-connector-elasticsearch vX.X.X for Flink 1.18/1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35134: - Summary: Release flink-connector-elasticsearch vX.X.X for Flink 1.18/1.19 Key: FLINK-35134 URL: https://issues.apache.org/jira/browse/FLINK-35134 Project: Flink

[VOTE] Release flink-connector-jdbc v3.2.0, release candidate #1

2024-04-17 Thread Danny Cranmer
Hi everyone, Please review and vote on the release candidate #1 for the version 3.2.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) This release supports Flink 1.18 and 1.19. The complete staging area is available for your review,

[jira] [Created] (FLINK-35136) Release flink-connector-hbase vX.X.X for Flink 1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35136: - Summary: Release flink-connector-hbase vX.X.X for Flink 1.19 Key: FLINK-35136 URL: https://issues.apache.org/jira/browse/FLINK-35136 Project: Flink Issue

[jira] [Created] (FLINK-35142) Release flink-connector-rabbitmq vX.X.X for Flink 1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35142: - Summary: Release flink-connector-rabbitmq vX.X.X for Flink 1.19 Key: FLINK-35142 URL: https://issues.apache.org/jira/browse/FLINK-35142 Project: Flink

[jira] [Created] (FLINK-35140) Release flink-connector-opensearch vX.X.X for Flink 1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35140: - Summary: Release flink-connector-opensearch vX.X.X for Flink 1.19 Key: FLINK-35140 URL: https://issues.apache.org/jira/browse/FLINK-35140 Project: Flink

Re: [VOTE] Release flink-connector-aws v4.3.0, release candidate #1

2024-04-17 Thread Ahmed Hamdy
Hi Danny, Thanks for driving this. +1 (non-binding) - Checksum verified - Signature and Keys match - Licenses checked - Build from source - Run Kinesis example - Reviewed web PR Best Regards Ahmed Hamdy On Tue, 16 Apr 2024 at 13:01, Jeyhun Karimov wrote: > +1 (non-binding) > > - Verified

Re: [DISCUSS] Connector releases for Flink 1.19

2024-04-17 Thread Danny Cranmer
Hello all, I have created a parent Jira to cover the releases [1]. I have assigned AWS and MongoDB to myself and OpenSearch to Sergey. Please assign the relevant issue to yourself as you pick up the tasks. Thanks! [1] https://issues.apache.org/jira/browse/FLINK-35131 On Tue, Apr 16, 2024 at

[jira] [Created] (FLINK-35133) Release flink-connector-cassandra v4.3.0 for Flink 1.18/1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35133: - Summary: Release flink-connector-cassandra v4.3.0 for Flink 1.18/1.19 Key: FLINK-35133 URL: https://issues.apache.org/jira/browse/FLINK-35133 Project: Flink

Re: [VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-17 Thread Yuxin Tan
+1 (non-binding) Best, Yuxin Zakelly Lan 于2024年4月17日周三 16:51写道: > +1 binding > > > Best, > Zakelly > > On Wed, Apr 17, 2024 at 2:05 PM Rui Fan <1996fan...@gmail.com> wrote: > > > +1(binding) > > > > Best, > > Rui > > > > On Wed, Apr 17, 2024 at 1:02 PM Xuannan Su > wrote: > > > > > Hi

[jira] [Created] (FLINK-35139) Release flink-connector-mongodb vX.X.X for Flink 1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35139: - Summary: Release flink-connector-mongodb vX.X.X for Flink 1.19 Key: FLINK-35139 URL: https://issues.apache.org/jira/browse/FLINK-35139 Project: Flink

[jira] [Created] (FLINK-35138) Release flink-connector-kafka vX.X.X for Flink 1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35138: - Summary: Release flink-connector-kafka vX.X.X for Flink 1.19 Key: FLINK-35138 URL: https://issues.apache.org/jira/browse/FLINK-35138 Project: Flink Issue

[jira] [Created] (FLINK-35137) Release flink-connector-jdbc vX.X.X for Flink 1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35137: - Summary: Release flink-connector-jdbc vX.X.X for Flink 1.19 Key: FLINK-35137 URL: https://issues.apache.org/jira/browse/FLINK-35137 Project: Flink Issue

[jira] [Created] (FLINK-35135) Release flink-connector-gcp-pubsub vX.X.X for Flink 1.18/1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35135: - Summary: Release flink-connector-gcp-pubsub vX.X.X for Flink 1.18/1.19 Key: FLINK-35135 URL: https://issues.apache.org/jira/browse/FLINK-35135 Project: Flink

[jira] [Created] (FLINK-35132) Release flink-connector-aws v4.3.0 for Flink 1.18/1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35132: - Summary: Release flink-connector-aws v4.3.0 for Flink 1.18/1.19 Key: FLINK-35132 URL: https://issues.apache.org/jira/browse/FLINK-35132 Project: Flink

[jira] [Created] (FLINK-35131) Support and Release Connectors for Flink 1.19

2024-04-17 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-35131: - Summary: Support and Release Connectors for Flink 1.19 Key: FLINK-35131 URL: https://issues.apache.org/jira/browse/FLINK-35131 Project: Flink Issue Type:

Re: [VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-17 Thread Xintong Song
+1 (binding) Best, Xintong On Wed, Apr 17, 2024 at 4:47 PM Zakelly Lan wrote: > +1 binding > > > Best, > Zakelly > > On Wed, Apr 17, 2024 at 2:05 PM Rui Fan <1996fan...@gmail.com> wrote: > > > +1(binding) > > > > Best, > > Rui > > > > On Wed, Apr 17, 2024 at 1:02 PM Xuannan Su > wrote: > >

[DISCUSS][QUESTION] Drop jdk 8 support for Flink connector Opensearch

2024-04-17 Thread Sergey Nuyanzin
Hi everyone I'm working on support for Opensearch v2.x for Flink connector Opensearch[1]. Unfortunately after several breaking changes (e.g. [2], [3]) on Opensearch side it is not possible anymore to use the same connector built for both Opensearch v1 and v2. This makes us to go in a similar way

Re: [VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-17 Thread Zakelly Lan
+1 binding Best, Zakelly On Wed, Apr 17, 2024 at 2:05 PM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > On Wed, Apr 17, 2024 at 1:02 PM Xuannan Su wrote: > > > Hi everyone, > > > > Thanks for all the feedback about the FLIP-442: General Improvement to > >

[jira] [Created] (FLINK-35130) Simplify AvailabilityNotifierImpl to support speculative scheduler and improve performance

2024-04-17 Thread Yuxin Tan (Jira)
Yuxin Tan created FLINK-35130: - Summary: Simplify AvailabilityNotifierImpl to support speculative scheduler and improve performance Key: FLINK-35130 URL: https://issues.apache.org/jira/browse/FLINK-35130

Re: [VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Ahmed Hamdy
+ 1 (non-binding) Best Regards Ahmed Hamdy On Wed, 17 Apr 2024 at 08:28, Yuepeng Pan wrote: > +1(non-binding). > > > > > Best, > Yuepeng Pan > > At 2024-04-17 14:27:27, "Ron liu" wrote: > >Hi Dev, > > > >Thank you to everyone for the feedback on FLIP-435: Introduce a New > >Materialized

Re:[VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Yuepeng Pan
+1(non-binding). Best, Yuepeng Pan At 2024-04-17 14:27:27, "Ron liu" wrote: >Hi Dev, > >Thank you to everyone for the feedback on FLIP-435: Introduce a New >Materialized Table for Simplifying Data Pipelines[1][2]. > >I'd like to start a vote for it. The vote will be open for at least 72

Re: [VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Rui Fan
+1(binding) Best, Rui On Wed, Apr 17, 2024 at 2:45 PM Feng Jin wrote: > +1 (non-binding) > > Best, > Feng Jin > > On Wed, Apr 17, 2024 at 2:28 PM Ron liu wrote: > > > +1(binding) > > > > Best, > > Ron > > > > Ron liu 于2024年4月17日周三 14:27写道: > > > > > Hi Dev, > > > > > > Thank you to everyone

Re: [VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Feng Jin
+1 (non-binding) Best, Feng Jin On Wed, Apr 17, 2024 at 2:28 PM Ron liu wrote: > +1(binding) > > Best, > Ron > > Ron liu 于2024年4月17日周三 14:27写道: > > > Hi Dev, > > > > Thank you to everyone for the feedback on FLIP-435: Introduce a New > > Materialized Table for Simplifying Data

Re: [VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Ron liu
+1(binding) Best, Ron Ron liu 于2024年4月17日周三 14:27写道: > Hi Dev, > > Thank you to everyone for the feedback on FLIP-435: Introduce a New > Materialized Table for Simplifying Data Pipelines[1][2]. > > I'd like to start a vote for it. The vote will be open for at least 72 > hours unless there is

[VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-17 Thread Ron liu
Hi Dev, Thank you to everyone for the feedback on FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines[1][2]. I'd like to start a vote for it. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1]

Re: [VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-17 Thread Rui Fan
+1(binding) Best, Rui On Wed, Apr 17, 2024 at 1:02 PM Xuannan Su wrote: > Hi everyone, > > Thanks for all the feedback about the FLIP-442: General Improvement to > Configuration for Flink 2.0 [1] [2]. > > I'd like to start a vote for it. The vote will be open for at least 72 > hours(excluding