[
https://issues.apache.org/jira/browse/FLINK-37371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leonard Xu resolved FLINK-37371.
--------------------------------
Fix Version/s: cdc-3.4.0
Resolution: Fixed
Fixed via master: da40727d29c8e3d3c637a52d102b3ed636ebf13a
> Postgres CDC incremental source fails to handle upper-case table and column
> names
> ---------------------------------------------------------------------------------
>
> Key: FLINK-37371
> URL: https://issues.apache.org/jira/browse/FLINK-37371
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.2.0
> Reporter: Sergei Morozov
> Assignee: Sergei Morozov
> Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.4.0
>
>
> Example table schema:
> {code:java}
> create table "Customers" (
> "Id" int not null primary key,
> "Name" varchar(32) not null
> );
> {code}
> Snapshotting fails with the following exception:
> {noformat}
> java.sql.SQLException: No result returned after running query [SELECT
> reltuples::bigint FROM pg_class WHERE oid = to_regclass('public.Customers')]
> at
> org.apache.flink.cdc.connectors.postgres.source.utils.PostgresQueryUtils.lambda$queryApproximateRowCnt$1(PostgresQueryUtils.java:81)
> at
> org.apache.flink.cdc.connectors.shaded.io.debezium.jdbc.JdbcConnection.queryAndMap(JdbcConnection.java:642)
> at
> org.apache.flink.cdc.connectors.shaded.io.debezium.jdbc.JdbcConnection.queryAndMap(JdbcConnection.java:510)
> at
> org.apache.flink.cdc.connectors.postgres.source.utils.PostgresQueryUtils.queryApproximateRowCnt(PostgresQueryUtils.java:76)
> at
> org.apache.flink.cdc.connectors.postgres.source.PostgresChunkSplitter.queryApproximateRowCnt(PostgresChunkSplitter.java:77)
> at
> org.apache.flink.cdc.connectors.base.source.assigner.splitter.JdbcSourceChunkSplitter.splitTableIntoChunks(JdbcSourceChunkSplitter.java:295)
> at
> org.apache.flink.cdc.connectors.base.source.assigner.splitter.JdbcSourceChunkSplitter.generateSplits(JdbcSourceChunkSplitter.java:81)
> ... 14 common frames omitted
> {noformat}
> Also:
> {noformat}
> org.apache.flink.util.FlinkRuntimeException: Read split
> SnapshotSplit{tableId=public.Customers, splitId='public.Customers:0',
> splitKeyType=[`Id` INT NOT NULL], splitStart=null, splitEnd=[8097],
> highWatermark=null} error due to org.apache.flink.util.FlinkRuntimeException:
> Snapshotting of table public.Customers failed.
> at
> org.apache.flink.cdc.connectors.base.source.reader.external.IncrementalSourceScanFetcher.checkReadException(IncrementalSourceScanFetcher.java:181)
> at
> org.apache.flink.cdc.connectors.base.source.reader.external.IncrementalSourceScanFetcher.pollSplitRecords(IncrementalSourceScanFetcher.java:128)
> at
> org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.pollSplitRecords(IncrementalSourceSplitReader.java:161)
> at
> org.apache.flink.cdc.connectors.base.source.reader.IncrementalSourceSplitReader.fetch(IncrementalSourceSplitReader.java:98)
> at
> org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58)
> at
> org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:142)
> at
> org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
> at java.base/java.lang.Thread.run(Thread.java:831)
> Caused by:
> org.apache.flink.cdc.connectors.shaded.io.debezium.DebeziumException:
> org.apache.flink.util.FlinkRuntimeException: Snapshotting of table
> public.Customers failed
> at
> org.apache.flink.cdc.connectors.shaded.io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
> at
> org.apache.flink.cdc.connectors.postgres.source.fetch.PostgresScanFetchTask.executeDataSnapshot(PostgresScanFetchTask.java:112)
> at
> org.apache.flink.cdc.connectors.base.source.reader.external.AbstractScanFetchTask.execute(AbstractScanFetchTask.java:71)
> at
> org.apache.flink.cdc.connectors.postgres.source.fetch.PostgresScanFetchTask.execute(PostgresScanFetchTask.java:86)
> at
> org.apache.flink.cdc.connectors.base.source.reader.external.IncrementalSourceScanFetcher.lambda$submitTask$1(IncrementalSourceScanFetcher.java:99)
> ... 3 common frames omitted
> Caused by: org.apache.flink.util.FlinkRuntimeException: Snapshotting of table
> public.Customers failed
> at
> org.apache.flink.cdc.connectors.postgres.source.fetch.PostgresScanFetchTask$PostgresSnapshotSplitReadTask.createDataEventsForTable(PostgresScanFetchTask.java:350)
> at
> org.apache.flink.cdc.connectors.postgres.source.fetch.PostgresScanFetchTask$PostgresSnapshotSplitReadTask.createDataEvents(PostgresScanFetchTask.java:265)
> at
> org.apache.flink.cdc.connectors.postgres.source.fetch.PostgresScanFetchTask$PostgresSnapshotSplitReadTask.doExecute(PostgresScanFetchTask.java:255)
> at
> org.apache.flink.cdc.connectors.postgres.source.fetch.PostgresScanFetchTask$PostgresSnapshotSplitReadTask.doExecute(PostgresScanFetchTask.java:211)
> at
> org.apache.flink.cdc.connectors.shaded.io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
> ... 7 common frames omitted
> Caused by: org.postgresql.util.PSQLException: ERROR: column "id" does not
> exist
> Hint: Perhaps you meant to reference the column "Customers.Id".
> Position: 45
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2553)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2285)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:323)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:481)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:401)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:164)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:114)
> at
> org.apache.flink.cdc.connectors.postgres.source.fetch.PostgresScanFetchTask$PostgresSnapshotSplitReadTask.createDataEventsForTable(PostgresScanFetchTask.java:313)
> ... 11 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
