Re: Postgresql cdc for Flink 2.0

2025-09-18 Thread George
Hi all / Hong. Slight contradiction picked up, currently Flink CDC 3.4 (Postgres) is not compatible with Flink 2.x https://nightlies.apache.org/flink/flink-cdc-docs-release-3.4/docs/connectors/flink-sources/overview/#supported-flink-versions So the setting above wont make a difference. G On Thu

Re: Postgresql cdc for Flink 2.0

2025-09-18 Thread George
Hi Wang As per previous, this is already how my CDC is configured. any other ideas. here is my table create. CREATE OR REPLACE TABLE postgres_catalog.inbound.adults ( id BIGINT -- This is a postgresql Serial generated field ,nationalid VARCHAR(14) -- NOT NULL ,data STRING -- JSONB Payload ,creat

Re: Postgresql cdc for Flink 2.0

2025-09-17 Thread George
Flink 1.20.x does not allow me to unpack/repack the json as a string into the array of rows as a complex structure... my inbound json payload is packed into a column called "data" as type jsonb... with the 1.20.x functionality I can't unpack jsonify it... and the functionality needed is only in 2

Re: Postgresql cdc for Flink 2.0

2025-09-17 Thread George
Hi there Had a look was/is using this setting on the CDC source, see below. CREATE OR REPLACE TABLE postgres_catalog.inbound.adults ( id BIGINT -- This is a postgresql Serial generated field ,nationalid VARCHAR(14) -- NOT NULL ,data STRING -- JSONB Payload ,created_at TIMESTAMP_LTZ(3) ,WATERMARK

Re: Postgresql cdc for Flink 2.0

2025-09-17 Thread George Leonard
Thank you.Will give this a try and see if using 2.0 gives me the extra functionality to unpack my payload. Although if I follow the docs the extra functionality is only part of 2.2.Sent from my iPhoneGeorge Leonard__george...@gmail.com+27 82 655 2466On 18 Sep 2025, at 03:55,

Re: Postgresql cdc for Flink 2.0

2025-09-17 Thread Hongshun Wang
Hi George, Please use ,'scan.incremental.snapshot.enabled'= 'true' . The old SouceFunction has been removed in flink 2.0. Otherwise, you can use flink 1.20. Best, Hongshun On Sun, Sep 14, 2025 at 11:48 PM George wrote: > Hi all... > > Below is the jars included in my flink 2.0 build. and th