[DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-22 Thread yue ma
Hi Flink devs, I would like to start a discussion on FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0 https://cwiki.apache.org/confluence/display/FLINK/FLIP-447%3A+Upgrade+FRocksDB+from+6.20.3++to+8.10.0 This FLIP proposes upgrading the version of FRocksDB in the Flink Project from 6.20.3 to

[RESULT][VOTE] FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-22 Thread Ron Liu
Hi, Dev I'm happy to announce that FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines[1] has been accepted with 13 approving votes (8 binding) [2] - Ron Liu(binding) - Feng Jin - Rui Fan(binding) - Yuepeng Pan - Ahmed Hamdy - Ferenc Csaky - Lincoln Lee(binding) - Leonard

[DISCUSS] FLIP-448: Introduce Pluggable Workflow Scheduler Interface for Materialized Table

2024-04-22 Thread Ron Liu
Hi, Dev I would like to start a discussion about FLIP-448: Introduce Pluggable Workflow Scheduler Interface for Materialized Table. In FLIP-435[1], we proposed Materialized Table, which has two types of data refresh modes: Full Refresh & Continuous Refresh Mode. In Full Refresh mode, the

[jira] [Created] (FLINK-35191) Support alter materialized table related syntaxes: suspend, resume, refresh, set and reset

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35191: - Summary: Support alter materialized table related syntaxes: suspend, resume, refresh, set and reset Key: FLINK-35191 URL: https://issues.apache.org/jira/browse/FLINK-35191

[jira] [Created] (FLINK-35194) Support describe job syntax

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35194: - Summary: Support describe job syntax Key: FLINK-35194 URL: https://issues.apache.org/jira/browse/FLINK-35194 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35195) Support the execution of materialized table in continuous mode

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35195: - Summary: Support the execution of materialized table in continuous mode Key: FLINK-35195 URL: https://issues.apache.org/jira/browse/FLINK-35195 Project: Flink

[jira] [Created] (FLINK-35205) Support the execution of alter materialized table reset options

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35205: - Summary: Support the execution of alter materialized table reset options Key: FLINK-35205 URL: https://issues.apache.org/jira/browse/FLINK-35205 Project: Flink

[jira] [Created] (FLINK-35196) bouncycastle class not found for flink pulsar connector

2024-04-22 Thread Wenbing Shen (Jira)
Wenbing Shen created FLINK-35196: Summary: bouncycastle class not found for flink pulsar connector Key: FLINK-35196 URL: https://issues.apache.org/jira/browse/FLINK-35196 Project: Flink

Re: [VOTE] Release flink-connector-aws v4.3.0, release candidate #2

2024-04-22 Thread Ahmed Hamdy
Thanks Danny, +1 (non-binding) - Verified Checksums - Verified Signatures - No binaries exists in source archive - Built source - Reviewed Web PR - Run basic Kinesis example Best Regards Ahmed Hamdy On Sun, 21 Apr 2024 at 14:25, Hang Ruan wrote: > +1 (non-binding) > > - Validated checksum

[jira] [Created] (FLINK-35193) Support drop materialized table syntax and execution

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35193: - Summary: Support drop materialized table syntax and execution Key: FLINK-35193 URL: https://issues.apache.org/jira/browse/FLINK-35193 Project: Flink Issue Type:

[jira] [Created] (FLINK-35208) Respect pipeline.cached-files during processing Python dependencies

2024-04-22 Thread Dian Fu (Jira)
Dian Fu created FLINK-35208: --- Summary: Respect pipeline.cached-files during processing Python dependencies Key: FLINK-35208 URL: https://issues.apache.org/jira/browse/FLINK-35208 Project: Flink

[jira] [Created] (FLINK-35190) Support the syntaxes of Create, Suspend, Resume, Refresh and Drop materialized table

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35190: - Summary: Support the syntaxes of Create, Suspend, Resume, Refresh and Drop materialized table Key: FLINK-35190 URL: https://issues.apache.org/jira/browse/FLINK-35190

[jira] [Created] (FLINK-35201) Support the execution of drop materialized table in full refresh mode

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35201: - Summary: Support the execution of drop materialized table in full refresh mode Key: FLINK-35201 URL: https://issues.apache.org/jira/browse/FLINK-35201 Project: Flink

[jira] [Created] (FLINK-35200) Support the execution of suspend, resume materialized table in full refresh mode

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35200: - Summary: Support the execution of suspend, resume materialized table in full refresh mode Key: FLINK-35200 URL: https://issues.apache.org/jira/browse/FLINK-35200 Project:

[jira] [Created] (FLINK-35203) Support the execution of alter materialized table set refresh mode

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35203: - Summary: Support the execution of alter materialized table set refresh mode Key: FLINK-35203 URL: https://issues.apache.org/jira/browse/FLINK-35203 Project: Flink

[jira] [Created] (FLINK-35192) operator oom

2024-04-22 Thread chenyuzhi (Jira)
chenyuzhi created FLINK-35192: - Summary: operator oom Key: FLINK-35192 URL: https://issues.apache.org/jira/browse/FLINK-35192 Project: Flink Issue Type: Bug Components: Kubernetes

[jira] [Created] (FLINK-35199) Support the execution of create materialized table in full refreshmode

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35199: - Summary: Support the execution of create materialized table in full refreshmode Key: FLINK-35199 URL: https://issues.apache.org/jira/browse/FLINK-35199 Project: Flink

[jira] [Created] (FLINK-35204) Support the execution of alter materialized table set options

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35204: - Summary: Support the execution of alter materialized table set options Key: FLINK-35204 URL: https://issues.apache.org/jira/browse/FLINK-35204 Project: Flink

[jira] [Created] (FLINK-35186) Create State V2 from new StateDescriptor

2024-04-22 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-35186: --- Summary: Create State V2 from new StateDescriptor Key: FLINK-35186 URL: https://issues.apache.org/jira/browse/FLINK-35186 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35188) Introduce CatalogMaterializedTable and related interface to support materialized table

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35188: - Summary: Introduce CatalogMaterializedTable and related interface to support materialized table Key: FLINK-35188 URL: https://issues.apache.org/jira/browse/FLINK-35188

[jira] [Created] (FLINK-35207) Kubernetes session E2E test fails to fetch packages.

2024-04-22 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-35207: --- Summary: Kubernetes session E2E test fails to fetch packages. Key: FLINK-35207 URL: https://issues.apache.org/jira/browse/FLINK-35207 Project: Flink Issue

[jira] [Created] (FLINK-35189) Introduce test-filesystem Catalog based on FileSystem Connector to support materialized table

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35189: - Summary: Introduce test-filesystem Catalog based on FileSystem Connector to support materialized table Key: FLINK-35189 URL: https://issues.apache.org/jira/browse/FLINK-35189

[jira] [Created] (FLINK-35198) Support the execution of refresh materialized table

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35198: - Summary: Support the execution of refresh materialized table Key: FLINK-35198 URL: https://issues.apache.org/jira/browse/FLINK-35198 Project: Flink Issue Type:

[jira] [Created] (FLINK-35197) Support the execution of suspend, resume materialized table in continuous mode

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35197: - Summary: Support the execution of suspend, resume materialized table in continuous mode Key: FLINK-35197 URL: https://issues.apache.org/jira/browse/FLINK-35197 Project:

[DISCUSS] FLIP-XXX Support currentFetchEventTimeLag and processingLag metrics

2024-04-22 Thread jialiang tan
Hello devs, I would like to start a discussion about FLIP-XXX: Support currentFetchEventTimeLag and processingLag metrics [1]. The main motivation for this change was that I had some difficulties in implementing the currentFetchEventTimeLag metrics for KafkaSource [2]. So I proposed to let the

[jira] [Created] (FLINK-35187) FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35187: - Summary: FLIP-435: Introduce a New Materialized Table for Simplifying Data Pipelines Key: FLINK-35187 URL: https://issues.apache.org/jira/browse/FLINK-35187 Project: Flink

[jira] [Created] (FLINK-35202) Support the execution of set freshness for materialized table

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35202: - Summary: Support the execution of set freshness for materialized table Key: FLINK-35202 URL: https://issues.apache.org/jira/browse/FLINK-35202 Project: Flink

[jira] [Created] (FLINK-35206) Desc table syntax support materialized table

2024-04-22 Thread dalongliu (Jira)
dalongliu created FLINK-35206: - Summary: Desc table syntax support materialized table Key: FLINK-35206 URL: https://issues.apache.org/jira/browse/FLINK-35206 Project: Flink Issue Type: Sub-task

FW: RE: [DISCUSS] FLIP-XXX Apicurio-avro format

2024-04-22 Thread David Radley
Hi Martijn, A gentle nudge, is this ok for you or one of the PMC or committers to create a Flip now? Kind regards, David. From: David Radley Date: Monday, 15 April 2024 at 12:29 To: dev@flink.apache.org Subject: Re: [EXTERNAL] RE: [DISCUSS] FLIP-XXX Apicurio-avro format Hi Martijn,

[VOTE] Release flink-connector-kafka v3.2.0, release candidate #1

2024-04-22 Thread Danny Cranmer
Hi everyone, Please review and vote on release candidate #1 for flink-connector-kafka v3.2.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) This release supports Flink 1.18 and 1.19. The complete staging area is available for your

Re: [VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-22 Thread Xuannan Su
Hi all, Thank you all! Closing the vote. The result will be announced in a separate email. Best regards, Xuannan On Fri, Apr 19, 2024 at 10:58 AM gongzhongqiang wrote: > > +1 (non-binding) > > > Best, > Zhongqiang Gong > > Xuannan Su 于2024年4月17日周三 13:02写道: > > > Hi everyone, > > > > Thanks

Re: [DISCUSS] FLIP-XXX Support currentFetchEventTimeLag and processingLag metrics

2024-04-22 Thread jialiang tan
Sorry all, it seems bad formatting in my email message, now I send it again gently and hope it work. I would like to start a discussion about FLIP-XXX: SupportcurrentFetchEventTimeLag and processingLag metrics [1]. The main motivation for this change was that I had some difficulties

[RESULT][VOTE] FLIP-442: General Improvement to Configuration for Flink 2.0

2024-04-22 Thread Xuannan Su
Hi devs, I'm happy to announce that FLIP-442: General Improvement to Configuration for Flink 2.0[1] has been accepted with 9 approving votes (4 bindings) [2]: - Rui Fan (binding) - Zakelly Lan (binding) - Xintong Song (binding) - Yuxin Tan (non-binding) - Zhu Zhu (binding) - Jeyhun Karimov

[VOTE] Release flink-connector-cassandra v3.2.0, release candidate #1

2024-04-22 Thread Danny Cranmer
Hi everyone, Please review and vote on release candidate #1 for flink-connector-cassandra v3.2.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) This release supports Flink 1.18 and 1.19. The complete staging area is available for

[jira] [Created] (FLINK-35209) Add a DeserializationSchema decorator that counts deserialize errors

2024-04-22 Thread Nicolas Perrin (Jira)
Nicolas Perrin created FLINK-35209: -- Summary: Add a DeserializationSchema decorator that counts deserialize errors Key: FLINK-35209 URL: https://issues.apache.org/jira/browse/FLINK-35209 Project:

[jira] [Created] (FLINK-35210) Give the option to set automatically the parallelism of the KafkaSource to the number of kafka partitions

2024-04-22 Thread Nicolas Perrin (Jira)
Nicolas Perrin created FLINK-35210: -- Summary: Give the option to set automatically the parallelism of the KafkaSource to the number of kafka partitions Key: FLINK-35210 URL:

[jira] [Created] (FLINK-35211) when synchronize LOB fields using oracle cdc, an error occurs

2024-04-22 Thread wangsw (Jira)
wangsw created FLINK-35211: -- Summary: when synchronize LOB fields using oracle cdc, an error occurs Key: FLINK-35211 URL: https://issues.apache.org/jira/browse/FLINK-35211 Project: Flink Issue

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-22 Thread Roman Khachatryan
Hi, Thanks for writing the proposal and preparing the upgrade. FRocksDB definitely needs to be kept in sync with the upstream and the new APIs are necessary for faster rescaling. We're already using a similar version internally. I reviewed the FLIP and it looks good to me (disclaimer: I took

[jira] [Created] (FLINK-35213) A potential inconsistent table structure issue

2024-04-22 Thread LvYanquan (Jira)
LvYanquan created FLINK-35213: - Summary: A potential inconsistent table structure issue Key: FLINK-35213 URL: https://issues.apache.org/jira/browse/FLINK-35213 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-22 Thread Rui Fan
Thanks Yue for driving this proposal! This upgrade makes sense, +1 for that. Best, Rui On Tue, Apr 23, 2024 at 12:06 PM Roman Khachatryan wrote: > Hi, > > Thanks for writing the proposal and preparing the upgrade. > > FRocksDB definitely needs to be kept in sync with the upstream and the new

[jira] [Created] (FLINK-35215) The performance of serializerKryo and serializerKryoWithoutRegistration are regressed

2024-04-22 Thread Rui Fan (Jira)
Rui Fan created FLINK-35215: --- Summary: The performance of serializerKryo and serializerKryoWithoutRegistration are regressed Key: FLINK-35215 URL: https://issues.apache.org/jira/browse/FLINK-35215 Project:

[jira] [Created] (FLINK-35214) Update result partition id for remote input channel when unknown input channel is updated

2024-04-22 Thread Yuxin Tan (Jira)
Yuxin Tan created FLINK-35214: - Summary: Update result partition id for remote input channel when unknown input channel is updated Key: FLINK-35214 URL: https://issues.apache.org/jira/browse/FLINK-35214

Re: [VOTE] Release flink-connector-kafka v3.2.0, release candidate #1

2024-04-22 Thread Muhammet Orazov
Thanks Danny, +1 (non-binding) - Checked 512 hash - Checked gpg signature - Reviewed pr - Built the source with JDK 11 & 8 Best, Muhammet On 2024-04-22 13:55, Danny Cranmer wrote: Hi everyone, Please review and vote on release candidate #1 for flink-connector-kafka v3.2.0, as follows: [ ]

[jira] [Created] (FLINK-35212) PyFlink thread mode process just can run once in standalonesession mode

2024-04-22 Thread Wei Yuan (Jira)
Wei Yuan created FLINK-35212: Summary: PyFlink thread mode process just can run once in standalonesession mode Key: FLINK-35212 URL: https://issues.apache.org/jira/browse/FLINK-35212 Project: Flink

Re: [VOTE] Release flink-connector-cassandra v3.2.0, release candidate #1

2024-04-22 Thread Muhammet Orazov
Thanks! +1 (non-binding) - Checked 512 hash - Checked gpg signature - Reviewed web pr & release notes - Built the source with JDK 11 & 8 Best, Muhammet On 2024-04-22 13:04, Danny Cranmer wrote: Hi everyone, Please review and vote on release candidate #1 for flink-connector-cassandra v3.2.0,

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-22 Thread Yanfei Lei
Hi Yue & Roman, Thanks for initiating this FLIP and all the efforts for the upgrade. 8.10.0 introduces some new features, making it possible for Flink to implement some new exciting features, and the upgrade also makes FRocksDB easier to maintain, +1 for upgrading. I read the FLIP and have a

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-22 Thread Yun Tang
Hi Yue, Thanks for driving this work. It has been three years since last major upgrade of FRocksDB. And it would be great improvement of Flink's state-backend with this upgrade. +1 for this work. Best Yun Tang From: Yanfei Lei Sent: Tuesday, April 23, 2024