Re: [ANNOUNCE] Apache Flink 1.19.0 released

2024-03-20 Thread Yanquan Lv
Congratulations! Thanks to all contributors. Best, Yanquan Shawn Huang 于2024年3月19日周二 15:12写道: > Congratulations! > > Best, > Shawn Huang > > > Xuannan Su 于2024年3月19日周二 14:40写道: > > > Congratulations! Thanks for all the great work! > > > > Best regards, > > Xuannan > > > > On Tue, Mar 19, 2024

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-03-20 Thread Feng Jin
Hi Ron and Lincoln Thanks for driving this discussion. I believe it will greatly improve the convenience of managing user real-time pipelines. I have some questions. *Regarding Limitations of Dynamic Table:* > Does not support modifying the select statement after the dynamic table is created.

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-20 Thread weijie guo
Congratulations! Well done. Best regards, Weijie Feng Jin 于2024年3月21日周四 11:40写道: > Congratulations! > > > Best, > Feng > > > On Thu, Mar 21, 2024 at 11:37 AM Ron liu wrote: > > > Congratulations! > > > > Best, > > Ron > > > > Jark Wu 于2024年3月21日周四 10:46写道: > > > > > Congratulations and

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-20 Thread Zakelly Lan
Congratulations! Best, Zakelly On Thu, Mar 21, 2024 at 12:05 PM weijie guo wrote: > Congratulations! Well done. > > > Best regards, > > Weijie > > > Feng Jin 于2024年3月21日周四 11:40写道: > >> Congratulations! >> >> >> Best, >> Feng >> >> >> On Thu, Mar 21, 2024 at 11:37 AM Ron liu wrote: >> >> >

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-20 Thread Hang Ruan
Congrattulations! Best, Hang Lincoln Lee 于2024年3月21日周四 09:54写道: > > Congrats, thanks for the great work! > > > Best, > Lincoln Lee > > > Peter Huang 于2024年3月20日周三 22:48写道: > >> Congratulations >> >> >> Best Regards >> Peter Huang >> >> On Wed, Mar 20, 2024 at 6:56 AM Huajie Wang wrote: >>

Re: [NOTICE] Flink CDC is importing GitHub issues into Apache Jira

2024-03-20 Thread Hongshun Wang
Congratulations. Thanks to Qingsheng for driving this job forward. Best, Hongshun On Wed, Mar 20, 2024 at 9:08 PM Qingsheng Ren wrote: > FYI: The auto-import has completed. Totally 137 issues are migrated from > GitHub issue to Apache Jira. Sorry again for flushing your inbox! > > Best, >

[jira] [Created] (FLINK-34900) Check compatibility for classes in flink-core-api that skip japicmp

2024-03-20 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-34900: -- Summary: Check compatibility for classes in flink-core-api that skip japicmp Key: FLINK-34900 URL: https://issues.apache.org/jira/browse/FLINK-34900 Project: Flink

[jira] [Created] (FLINK-34898) Cannot create named STRUCT with a single field

2024-03-20 Thread Chloe He (Jira)
Chloe He created FLINK-34898: Summary: Cannot create named STRUCT with a single field Key: FLINK-34898 URL: https://issues.apache.org/jira/browse/FLINK-34898 Project: Flink Issue Type: Bug

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-20 Thread Ron liu
Congratulations! Best, Ron Jark Wu 于2024年3月21日周四 10:46写道: > Congratulations and welcome! > > Best, > Jark > > On Thu, 21 Mar 2024 at 10:35, Rui Fan <1996fan...@gmail.com> wrote: > > > Congratulations! > > > > Best, > > Rui > > > > On Thu, Mar 21, 2024 at 10:25 AM Hang Ruan > wrote: > > > > >

Re: [ANNOUNCE] Apache Flink 1.19.0 released

2024-03-20 Thread Hongshun Wang
Congratulations! Best, Hongshun On Tue, Mar 19, 2024 at 3:12 PM Shawn Huang wrote: > Congratulations! > > Best, > Shawn Huang > > > Xuannan Su 于2024年3月19日周二 14:40写道: > > > Congratulations! Thanks for all the great work! > > > > Best regards, > > Xuannan > > > > On Tue, Mar 19, 2024 at 1:31 PM

Re: [VOTE] FLIP-433: State Access on DataStream API V2

2024-03-20 Thread Rui Fan
+1(binding) Thanks to Weijie for driving this proposal, which solves the problem that I raised in FLIP-359. Best, Rui On Thu, Mar 21, 2024 at 10:10 AM Hangxiang Yu wrote: > +1 (binding) > > On Thu, Mar 21, 2024 at 10:04 AM Xintong Song > wrote: > > > +1 (binding) > > > > Best, > > > >

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-20 Thread Lincoln Lee
Congrats, thanks for the great work! Best, Lincoln Lee Peter Huang 于2024年3月20日周三 22:48写道: > Congratulations > > > Best Regards > Peter Huang > > On Wed, Mar 20, 2024 at 6:56 AM Huajie Wang wrote: > >> >> Congratulations >> >> >> >> Best, >> Huajie Wang >> >> >> >> Leonard Xu 于2024年3月20日周三

Re: [VOTE] FLIP-433: State Access on DataStream API V2

2024-03-20 Thread Hangxiang Yu
+1 (binding) On Thu, Mar 21, 2024 at 10:04 AM Xintong Song wrote: > +1 (binding) > > Best, > > Xintong > > > > On Wed, Mar 20, 2024 at 8:30 PM weijie guo > wrote: > > > Hi everyone, > > > > > > Thanks for all the feedback about the FLIP-433: State Access on > > DataStream API V2 [1]. The

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-20 Thread Rui Fan
Congratulations! Best, Rui On Thu, Mar 21, 2024 at 10:25 AM Hang Ruan wrote: > Congrattulations! > > Best, > Hang > > Lincoln Lee 于2024年3月21日周四 09:54写道: > >> >> Congrats, thanks for the great work! >> >> >> Best, >> Lincoln Lee >> >> >> Peter Huang 于2024年3月20日周三 22:48写道: >> >>>

Re:Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-20 Thread Xuyang
Cheers! -- Best! Xuyang 在 2024-03-21 10:28:45,"Rui Fan" <1996fan...@gmail.com> 写道: >Congratulations! > >Best, >Rui > >On Thu, Mar 21, 2024 at 10:25 AM Hang Ruan wrote: > >> Congrattulations! >> >> Best, >> Hang >> >> Lincoln Lee 于2024年3月21日周四 09:54写道: >> >>> >>> Congrats, thanks

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-20 Thread Feng Jin
Congratulations! Best, Feng On Thu, Mar 21, 2024 at 11:37 AM Ron liu wrote: > Congratulations! > > Best, > Ron > > Jark Wu 于2024年3月21日周四 10:46写道: > > > Congratulations and welcome! > > > > Best, > > Jark > > > > On Thu, 21 Mar 2024 at 10:35, Rui Fan <1996fan...@gmail.com> wrote: > > > > >

[jira] [Created] (FLINK-34899) Remove all classes that skip the japicmp check for flink-core-api

2024-03-20 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-34899: -- Summary: Remove all classes that skip the japicmp check for flink-core-api Key: FLINK-34899 URL: https://issues.apache.org/jira/browse/FLINK-34899 Project: Flink

Re: [VOTE] FLIP-433: State Access on DataStream API V2

2024-03-20 Thread Xintong Song
+1 (binding) Best, Xintong On Wed, Mar 20, 2024 at 8:30 PM weijie guo wrote: > Hi everyone, > > > Thanks for all the feedback about the FLIP-433: State Access on > DataStream API V2 [1]. The discussion thread is here [2]. > > > The vote will be open for at least 72 hours unless there is an

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-20 Thread Jark Wu
Congratulations and welcome! Best, Jark On Thu, 21 Mar 2024 at 10:35, Rui Fan <1996fan...@gmail.com> wrote: > Congratulations! > > Best, > Rui > > On Thu, Mar 21, 2024 at 10:25 AM Hang Ruan wrote: > > > Congrattulations! > > > > Best, > > Hang > > > > Lincoln Lee 于2024年3月21日周四 09:54写道: > > >

Re: [ANNOUNCE] Apache Flink 1.19.0 released

2024-03-20 Thread Jacky Lau
Congratulations! Best, Jacky Lau Hongshun Wang 于2024年3月21日周四 10:32写道: > Congratulations! > > Best, > Hongshun > > On Tue, Mar 19, 2024 at 3:12 PM Shawn Huang wrote: > > > Congratulations! > > > > Best, > > Shawn Huang > > > > > > Xuannan Su 于2024年3月19日周二 14:40写道: > > > > > Congratulations!

Re: [VOTE] FLIP-436: Introduce Catalog-related Syntax

2024-03-20 Thread Benchao Li
+1 (binding) gongzhongqiang 于2024年3月20日周三 11:40写道: > > +1 (non-binding) > > Best, > Zhongqiang Gong > > Yubin Li 于2024年3月19日周二 18:03写道: > > > Hi everyone, > > > > Thanks for all the feedback, I'd like to start a vote on the FLIP-436: > > Introduce Catalog-related Syntax [1]. The discussion

Re: Re: [DISCUSS] FLIP-436: Introduce "SHOW CREATE CATALOG" Syntax

2024-03-20 Thread Hang Ruan
Hi. Yubin. Thanks for your update. LGTM. Best, Hang Yubin Li 于2024年3月20日周三 11:56写道: > Hi Hang, > > I have updated FLIP as you suggested, thanks for your valuable feedback! > > Best, > Yubin > > On Wed, Mar 20, 2024 at 11:15 AM Hang Ruan wrote: > > > > Hi, Yubin, > > > > I found a little

[jira] [Created] (FLINK-34854) [Feature] Optimizing the last chunk with a large amount of data to avoid memory overflow.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34854: -- Summary: [Feature] Optimizing the last chunk with a large amount of data to avoid memory overflow. Key: FLINK-34854 URL:

[jira] [Created] (FLINK-34795) expected to support flink-connector-db2-cd('scan.startup.mode'='specific_offsets')

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34795: -- Summary: expected to support flink-connector-db2-cd('scan.startup.mode'='specific_offsets') Key: FLINK-34795 URL:

[jira] [Created] (FLINK-34892) Nightly AWS connectors build fails on running python tests

2024-03-20 Thread Aleksandr Pilipenko (Jira)
Aleksandr Pilipenko created FLINK-34892: --- Summary: Nightly AWS connectors build fails on running python tests Key: FLINK-34892 URL: https://issues.apache.org/jira/browse/FLINK-34892 Project:

[jira] [Created] (FLINK-34880) [Feature][Pipeline] SchemeRegistry saves the routing rules used to restore the TransformDataOperator

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34880: -- Summary: [Feature][Pipeline] SchemeRegistry saves the routing rules used to restore the TransformDataOperator Key: FLINK-34880 URL:

[jira] [Created] (FLINK-34839) Flink CDC 3.1.0 Plan

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34839: -- Summary: Flink CDC 3.1.0 Plan Key: FLINK-34839 URL: https://issues.apache.org/jira/browse/FLINK-34839 Project: Flink Issue Type:

[jira] [Created] (FLINK-34809) [mysql] Add notifications to Slack when the Snapshot phase ends or Binlog stream phase begins

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34809: -- Summary: [mysql] Add notifications to Slack when the Snapshot phase ends or Binlog stream phase begins Key: FLINK-34809 URL:

[jira] [Created] (FLINK-34875) [Feature] CDC Action supports update and delete record (logical deletion)

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34875: -- Summary: [Feature] CDC Action supports update and delete record (logical deletion) Key: FLINK-34875 URL: https://issues.apache.org/jira/browse/FLINK-34875

[jira] [Created] (FLINK-34866) [Bug] Meet NoClassDefFoundError when add new table in mysql cdc source

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34866: -- Summary: [Bug] Meet NoClassDefFoundError when add new table in mysql cdc source Key: FLINK-34866 URL: https://issues.apache.org/jira/browse/FLINK-34866

[jira] [Created] (FLINK-34871) Oracle add data has error

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34871: -- Summary: Oracle add data has error Key: FLINK-34871 URL: https://issues.apache.org/jira/browse/FLINK-34871 Project: Flink Issue Type:

Re: [DISCUSS] Manual savepoint triggering in flink-kubernetes-operator

2024-03-20 Thread Yang Wang
Using a separate CR for managing the savepoints is really a good idea. After then managing savepoints will be easier and we will not leak any unusable savepoints on the object storage. Best, Yang On Wed, Mar 13, 2024 at 4:40 AM Gyula Fóra wrote: > That would be great Mate! If you could draw

[VOTE] FLIP-433: State Access on DataStream API V2

2024-03-20 Thread weijie guo
Hi everyone, Thanks for all the feedback about the FLIP-433: State Access on DataStream API V2 [1]. The discussion thread is here [2]. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. Best regards, Weijie [1]

[jira] [Created] (FLINK-34801) [Bug] An insufficient number of arguments were supplied for the procedure or function cdc.fn_cdc_get_all_changes_

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34801: -- Summary: [Bug] An insufficient number of arguments were supplied for the procedure or function cdc.fn_cdc_get_all_changes_ Key: FLINK-34801 URL:

[jira] [Created] (FLINK-34858) Support modifying partition keys and primary keys of Schema.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34858: -- Summary: Support modifying partition keys and primary keys of Schema. Key: FLINK-34858 URL: https://issues.apache.org/jira/browse/FLINK-34858

[jira] [Created] (FLINK-34847) MySQL database using the compress function, can provide analytic function

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34847: -- Summary: MySQL database using the compress function, can provide analytic function Key: FLINK-34847 URL: https://issues.apache.org/jira/browse/FLINK-34847

[jira] [Created] (FLINK-34826) flink-connector-mysql-cdc-2.4.2 when a multi-column primary key index starts with a varchar column, a snapshot chunk can potentially return a large number of rows and ca

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34826: -- Summary: flink-connector-mysql-cdc-2.4.2 when a multi-column primary key index starts with a varchar column, a snapshot chunk can potentially return a large number of rows and cause jvm to

[jira] [Created] (FLINK-34814) [Bug] java.lang.NoClassDefFoundError: Could not initialize class org.apache.kafka.connect.json.JsonConverterConfig

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34814: -- Summary: [Bug] java.lang.NoClassDefFoundError: Could not initialize class org.apache.kafka.connect.json.JsonConverterConfig Key: FLINK-34814 URL:

Re: [VOTE] FLIP-402: Extend ZooKeeper Curator configurations

2024-03-20 Thread Yang Wang
+1 (binding) since ZK HA is still widely used. Best, Yang On Thu, Mar 14, 2024 at 6:27 PM Matthias Pohl wrote: > Nothing to add from my side. Thanks, Alex. > > +1 (binding) > > On Thu, Mar 7, 2024 at 4:09 PM Alex Nitavsky > wrote: > > > Hi everyone, > > > > I'd like to start a vote on

[jira] [Created] (FLINK-34867) [Bug] [StarRocks] [cdc master branch] Unsupported CDC data type BYTES/VARBINARY

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34867: -- Summary: [Bug] [StarRocks] [cdc master branch] Unsupported CDC data type BYTES/VARBINARY Key: FLINK-34867 URL:

[jira] [Created] (FLINK-34853) [Feature] submit Flink CDC pipeline job to k8s cluster.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34853: -- Summary: [Feature] submit Flink CDC pipeline job to k8s cluster. Key: FLINK-34853 URL: https://issues.apache.org/jira/browse/FLINK-34853 Project:

[jira] [Created] (FLINK-34848) refactor: some class in flink-connector-mysql-cdc do not inherit from the base class in flink-cdc-base

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34848: -- Summary: refactor: some class in flink-connector-mysql-cdc do not inherit from the base class in flink-cdc-base Key: FLINK-34848 URL:

[jira] [Created] (FLINK-34822) [Bug] oracle clob

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34822: -- Summary: [Bug] oracle clob Key: FLINK-34822 URL: https://issues.apache.org/jira/browse/FLINK-34822 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-34830) [3.1][cdc-composer] verify the options of Context for Factory.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34830: -- Summary: [3.1][cdc-composer] verify the options of Context for Factory. Key: FLINK-34830 URL: https://issues.apache.org/jira/browse/FLINK-34830

[jira] [Created] (FLINK-34886) Support Sybase database

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34886: -- Summary: Support Sybase database Key: FLINK-34886 URL: https://issues.apache.org/jira/browse/FLINK-34886 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-34812) [Bug] During the snapshot phase, projection is not being performed according to the user defined schema.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34812: -- Summary: [Bug] During the snapshot phase, projection is not being performed according to the user defined schema. Key: FLINK-34812 URL:

[jira] [Created] (FLINK-34850) [test] Support to execute the (mysql) test cases with custom database deployment

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34850: -- Summary: [test] Support to execute the (mysql) test cases with custom database deployment Key: FLINK-34850 URL:

[jira] [Created] (FLINK-34878) [Feature][Pipeline] Flink CDC pipeline transform supports CASE WHEN

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34878: -- Summary: [Feature][Pipeline] Flink CDC pipeline transform supports CASE WHEN Key: FLINK-34878 URL: https://issues.apache.org/jira/browse/FLINK-34878

[jira] [Created] (FLINK-34874) [MongoDB] Support initial.snapshotting.pipeline related configs in table api

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34874: -- Summary: [MongoDB] Support initial.snapshotting.pipeline related configs in table api Key: FLINK-34874 URL: https://issues.apache.org/jira/browse/FLINK-34874

[DISCUSS] FLIP-XXX Apicurio-avro format

2024-03-20 Thread David Radley
Hi, As per the FLIP process I would like to raise a FLIP, but do not have authority, so have created a google doc for the Flip to introduce a new Apicurio Avro format. The document is https://docs.google.com/document/d/14LWZPVFQ7F9mryJPdKXb4l32n7B0iWYkcOdEd1xTC7w/edit?usp=sharing I have

[jira] [Created] (FLINK-34883) Error on Postgres-CDC using incremental snapshot with UUID column as PK

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34883: -- Summary: Error on Postgres-CDC using incremental snapshot with UUID column as PK Key: FLINK-34883 URL: https://issues.apache.org/jira/browse/FLINK-34883

[jira] [Created] (FLINK-34889) Flink CDC may occur binlog can't be find when dynamic added table repeatedly

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34889: -- Summary: Flink CDC may occur binlog can't be find when dynamic added table repeatedly Key: FLINK-34889 URL: https://issues.apache.org/jira/browse/FLINK-34889

[jira] [Created] (FLINK-34888) How to do MySQL to MySQL pipeline

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34888: -- Summary: How to do MySQL to MySQL pipeline Key: FLINK-34888 URL: https://issues.apache.org/jira/browse/FLINK-34888 Project: Flink Issue

[jira] [Created] (FLINK-34894) Migrate JoinDependentConditionDerivationRule

2024-03-20 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-34894: --- Summary: Migrate JoinDependentConditionDerivationRule Key: FLINK-34894 URL: https://issues.apache.org/jira/browse/FLINK-34894 Project: Flink Issue

[jira] [Created] (FLINK-34893) Bump Checkstyle to 9+

2024-03-20 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-34893: --- Summary: Bump Checkstyle to 9+ Key: FLINK-34893 URL: https://issues.apache.org/jira/browse/FLINK-34893 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-34895) Migrate FlinkRewriteSubQueryRule

2024-03-20 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-34895: --- Summary: Migrate FlinkRewriteSubQueryRule Key: FLINK-34895 URL: https://issues.apache.org/jira/browse/FLINK-34895 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-34808) [Bug] Mysql cdc connector connection pool

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34808: -- Summary: [Bug] Mysql cdc connector connection pool Key: FLINK-34808 URL: https://issues.apache.org/jira/browse/FLINK-34808 Project: Flink

[jira] [Created] (FLINK-34846) [Bug][cdc-connectors][cdc-base] Transaction log of high_watermark read twice

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34846: -- Summary: [Bug][cdc-connectors][cdc-base] Transaction log of high_watermark read twice Key: FLINK-34846 URL: https://issues.apache.org/jira/browse/FLINK-34846

[jira] [Created] (FLINK-34865) [enhancement] [StarRocks] When synchronizing tables using Flink CDC 3.x, is it possible to include the comments of the source table's fields when creating the target tab

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34865: -- Summary: [enhancement] [StarRocks] When synchronizing tables using Flink CDC 3.x, is it possible to include the comments of the source table's fields when creating the target table?

[jira] [Created] (FLINK-34870) [Bug] MongoCDC checkpoint error

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34870: -- Summary: [Bug] MongoCDC checkpoint error Key: FLINK-34870 URL: https://issues.apache.org/jira/browse/FLINK-34870 Project: Flink Issue

[jira] [Created] (FLINK-34885) mysql to sr pipeline, float value bug

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34885: -- Summary: mysql to sr pipeline, float value bug Key: FLINK-34885 URL: https://issues.apache.org/jira/browse/FLINK-34885 Project: Flink

[jira] [Created] (FLINK-34882) mysql cdc to doris: TIME type converting fail

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34882: -- Summary: mysql cdc to doris: TIME type converting fail Key: FLINK-34882 URL: https://issues.apache.org/jira/browse/FLINK-34882 Project: Flink

[jira] [Created] (FLINK-34864) [enhancement] [ cdc-connector] [ignore no primary key table] When using regular expressions to synchronize the entire database with Flink CDC, skip tables that do not ha

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34864: -- Summary: [enhancement] [ cdc-connector] [ignore no primary key table] When using regular expressions to synchronize the entire database with Flink CDC, skip tables that do not have a primary

[jira] [Created] (FLINK-34876) [Feature][Pipeline] Flink CDC pipeline transform supports UDF

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34876: -- Summary: [Feature][Pipeline] Flink CDC pipeline transform supports UDF Key: FLINK-34876 URL: https://issues.apache.org/jira/browse/FLINK-34876

[jira] [Created] (FLINK-34798) [Bug] flink cdc mysql, use datastream to synchronize multiple tables, read full amount of data in a loop

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34798: -- Summary: [Bug] flink cdc mysql, use datastream to synchronize multiple tables, read full amount of data in a loop Key: FLINK-34798 URL:

[jira] [Created] (FLINK-34806) [Feature][Postgres] Support automatically identify newly added tables

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34806: -- Summary: [Feature][Postgres] Support automatically identify newly added tables Key: FLINK-34806 URL: https://issues.apache.org/jira/browse/FLINK-34806

[jira] [Created] (FLINK-34843) [Bug] BinlogSplitReader#pollSplitRecords return finishedSplit when exception occurs

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34843: -- Summary: [Bug] BinlogSplitReader#pollSplitRecords return finishedSplit when exception occurs Key: FLINK-34843 URL:

[jira] [Created] (FLINK-34829) When will opengauss or gaussdb be supported?

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34829: -- Summary: When will opengauss or gaussdb be supported? Key: FLINK-34829 URL: https://issues.apache.org/jira/browse/FLINK-34829 Project: Flink

[jira] [Created] (FLINK-34856) [Bug] timeout when Invocating Remote RPC.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34856: -- Summary: [Bug] timeout when Invocating Remote RPC. Key: FLINK-34856 URL: https://issues.apache.org/jira/browse/FLINK-34856 Project: Flink

[jira] [Created] (FLINK-34828) sql server Limited date synchronization

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34828: -- Summary: sql server Limited date synchronization Key: FLINK-34828 URL: https://issues.apache.org/jira/browse/FLINK-34828 Project: Flink

Re: [DISCUSS] FLIP-426: Grouping Remote State Access

2024-03-20 Thread Jinzhong Li
Hi Yue, Thanks for your comments. I get your point.I think there are two possible ways we can support multiGet in synchronization models in future: 1) Implement the multiGet interface in the existing Rocksdb State interface, then integrate this interface to the synchronous Operators layer. 2)

[jira] [Created] (FLINK-34891) RemoteStorageScannerTest causes exit 239

2024-03-20 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-34891: --- Summary: RemoteStorageScannerTest causes exit 239 Key: FLINK-34891 URL: https://issues.apache.org/jira/browse/FLINK-34891 Project: Flink Issue Type: Bug

RE: Additional metadata available for Kafka serdes

2024-03-20 Thread David Radley
Hi Balint, Excellent, I have just put up a discussion thread on the dev list for a new Fli top add this, please review and feedback and +1 if this is what you are looking for. At the moment, I have not got the topic name in there. I wonder where we would use the topic name during

[jira] [Created] (FLINK-34879) [Feature][Pipeline] Flink CDC pipeline transform supports the strategy of schema evolution

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34879: -- Summary: [Feature][Pipeline] Flink CDC pipeline transform supports the strategy of schema evolution Key: FLINK-34879 URL:

[jira] [Created] (FLINK-34881) [Feature] Flink CDC Oracle support tables without primary keys and scan newly added tables

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34881: -- Summary: [Feature] Flink CDC Oracle support tables without primary keys and scan newly added tables Key: FLINK-34881 URL:

[jira] [Created] (FLINK-34859) [Bug] Oracle cdc in table api does no support server-time-zone option

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34859: -- Summary: [Bug] Oracle cdc in table api does no support server-time-zone option Key: FLINK-34859 URL: https://issues.apache.org/jira/browse/FLINK-34859

[jira] [Created] (FLINK-34877) [Feature][Pipeline] Flink CDC pipeline transform supports type conversion

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34877: -- Summary: [Feature][Pipeline] Flink CDC pipeline transform supports type conversion Key: FLINK-34877 URL: https://issues.apache.org/jira/browse/FLINK-34877

[jira] [Created] (FLINK-34811) [Bug] [sqlserver cdc] the field "snapshot" is always "false"

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34811: -- Summary: [Bug] [sqlserver cdc] the field "snapshot" is always "false" Key: FLINK-34811 URL: https://issues.apache.org/jira/browse/FLINK-34811

[jira] [Created] (FLINK-34835) [cdc-common] Add default value of pipeline parallelism.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34835: -- Summary: [cdc-common] Add default value of pipeline parallelism. Key: FLINK-34835 URL: https://issues.apache.org/jira/browse/FLINK-34835 Project:

[jira] [Created] (FLINK-34868) [Bug] oracle-cdc cannot read oracle multitenant pdb binlog: ORA-00942: table or view does not exist (LOG_MINING_FLUSH)

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34868: -- Summary: [Bug] oracle-cdc cannot read oracle multitenant pdb binlog: ORA-00942: table or view does not exist (LOG_MINING_FLUSH) Key: FLINK-34868 URL:

[jira] [Created] (FLINK-34873) [Bug] After starting Streaming ELT from MySQL to StarRocks using Flink CDC 3.0, the newly created tables are not being synchronized.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34873: -- Summary: [Bug] After starting Streaming ELT from MySQL to StarRocks using Flink CDC 3.0, the newly created tables are not being synchronized. Key: FLINK-34873

[jira] [Created] (FLINK-34844) [3.1][pipeline-connectors] Add pipeline Sink for Pulsar.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34844: -- Summary: [3.1][pipeline-connectors] Add pipeline Sink for Pulsar. Key: FLINK-34844 URL: https://issues.apache.org/jira/browse/FLINK-34844 Project:

[jira] [Created] (FLINK-34849) Flink CDC3.0 Demo Mysql binlog to Starrocks Exception amount of AddColumnEvent is already existed[Bug]

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34849: -- Summary: Flink CDC3.0 Demo Mysql binlog to Starrocks Exception amount of AddColumnEvent is already existed[Bug] Key: FLINK-34849 URL:

[jira] [Created] (FLINK-34860) [Bug] Jackson version conflicts among MySQL/TiDB/MongoDB connectors

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34860: -- Summary: [Bug] Jackson version conflicts among MySQL/TiDB/MongoDB connectors Key: FLINK-34860 URL: https://issues.apache.org/jira/browse/FLINK-34860

[jira] [Created] (FLINK-34861) [Bug] Inconsistent Kafka shading among cdc connectors

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34861: -- Summary: [Bug] Inconsistent Kafka shading among cdc connectors Key: FLINK-34861 URL: https://issues.apache.org/jira/browse/FLINK-34861 Project:

[jira] [Created] (FLINK-34825) [flink-cdc-pipeline-connectors] Add Implementation of DataSource in MongoDB

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34825: -- Summary: [flink-cdc-pipeline-connectors] Add Implementation of DataSource in MongoDB Key: FLINK-34825 URL: https://issues.apache.org/jira/browse/FLINK-34825

[jira] [Created] (FLINK-34869) [Bug][mysql] Remove all previous table and add new added table will throw Exception.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34869: -- Summary: [Bug][mysql] Remove all previous table and add new added table will throw Exception. Key: FLINK-34869 URL:

[jira] [Created] (FLINK-34832) [Bug] BinlogOffset's comparative method may be cause incremental snapshot scan data lose

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34832: -- Summary: [Bug] BinlogOffset's comparative method may be cause incremental snapshot scan data lose Key: FLINK-34832 URL:

[jira] [Created] (FLINK-34862) [Bug] oracle-cdc ORA-01371: Complete LogMiner dictionary not found

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34862: -- Summary: [Bug] oracle-cdc ORA-01371: Complete LogMiner dictionary not found Key: FLINK-34862 URL: https://issues.apache.org/jira/browse/FLINK-34862

[jira] [Created] (FLINK-34855) [Improve] When using miniCluster, don't check 'FLINK_HOME'.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34855: -- Summary: [Improve] When using miniCluster, don't check 'FLINK_HOME'. Key: FLINK-34855 URL: https://issues.apache.org/jira/browse/FLINK-34855

[jira] [Created] (FLINK-34890) io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource :.java.sql.SQLException: ORA-01291: missing log file

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34890: -- Summary: io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource :.java.sql.SQLException: ORA-01291: missing log file Key: FLINK-34890 URL:

Re: [DISCUSS] FLIP-428: Fault Tolerance/Rescale Integration for Disaggregated State

2024-03-20 Thread Jinzhong Li
Hi Yue, Thanks for your feedback! > 1. If we choose Option-3 for ForSt , how would we handle Manifest File > ? Should we take a snapshot of the Manifest during the synchronization phase? IIUC, the GetLiveFiles() API in Option-3 can also catch the fileInfo of Manifest files, and this api also

[jira] [Created] (FLINK-34752) "legacy-flink-cdc-sources" Page of TIDB for Flink CDC Chinese Documentation.

2024-03-20 Thread LvYanquan (Jira)
LvYanquan created FLINK-34752: - Summary: "legacy-flink-cdc-sources" Page of TIDB for Flink CDC Chinese Documentation. Key: FLINK-34752 URL: https://issues.apache.org/jira/browse/FLINK-34752 Project:

Re: [DISCUSS] FLIP-428: Fault Tolerance/Rescale Integration for Disaggregated State

2024-03-20 Thread yue ma
Hi Jinzhong Thank you for initiating this FLIP. I have just some minor question: 1. If we choice Option-3 for ForSt , how would we handle Manifest File ? Should we take snapshot of the Manifest during the synchronization phase? Otherwise, may the Manifest and MetaInfo information be

[jira] [Created] (FLINK-34767) flink-sql-connector-mysql-cdc cannot read incremental data in initial mode

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34767: -- Summary: flink-sql-connector-mysql-cdc cannot read incremental data in initial mode Key: FLINK-34767 URL: https://issues.apache.org/jira/browse/FLINK-34767

[jira] [Created] (FLINK-34770) Unify download links of legacy connector to fat jar

2024-03-20 Thread LvYanquan (Jira)
LvYanquan created FLINK-34770: - Summary: Unify download links of legacy connector to fat jar Key: FLINK-34770 URL: https://issues.apache.org/jira/browse/FLINK-34770 Project: Flink Issue Type:

[jira] [Created] (FLINK-34774) Supports synchronizing data from the oracle standby database

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34774: -- Summary: Supports synchronizing data from the oracle standby database Key: FLINK-34774 URL: https://issues.apache.org/jira/browse/FLINK-34774

[jira] [Created] (FLINK-34786) [Bug] [tidb cdc]a error:java.lang.IllegalArgumentException: Invalid range: [+INF..{xxx}) at org.apache.flink.shaded.guava30.com.google.common.collect.Range.(Range.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34786: -- Summary: [Bug] [tidb cdc]a error:java.lang.IllegalArgumentException: Invalid range: [+INF..{xxx}) at org.apache.flink.shaded.guava30.com.google.common.collect.Range.(Range.java:358)

[jira] [Created] (FLINK-34791) Add docs for the experimental features.

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34791: -- Summary: Add docs for the experimental features. Key: FLINK-34791 URL: https://issues.apache.org/jira/browse/FLINK-34791 Project: Flink

[jira] [Created] (FLINK-34813) [Bug] Using tidb-cdc to connect to the data source cpu usage is always high

2024-03-20 Thread Flink CDC Issue Import (Jira)
Flink CDC Issue Import created FLINK-34813: -- Summary: [Bug] Using tidb-cdc to connect to the data source cpu usage is always high Key: FLINK-34813 URL: https://issues.apache.org/jira/browse/FLINK-34813

  1   2   >