Jubin Soni created FLINK-39950:
----------------------------------
Summary: [sqlserver] scan.incremental.snapshot.chunk.key-column
only searches primary key columns instead of all table columns
Key: FLINK-39950
URL: https://issues.apache.org/jira/browse/FLINK-39950
Project: Flink
Issue Type: Bug
Components: Flink CDC
Reporter: Jubin Soni
*Issue:*
The {{scan.incremental.snapshot.chunk.key-column}} option is documented and
exposed for the SQL Server CDC connector, but it is broken when the specified
column is not part of the primary key.
In {{{}SqlServerUtils.getSplitColumn(){}}}, when {{chunkKeyColumn}} is set, the
code searches only {{table.primaryKeyColumns()}} instead of
{{{}table.columns(){}}}. This means:
# Any non-primary-key chunk key column silently fails with a misleading error:
{quote}{{Chunk key column 'X' doesn't exist in the primary key [...]}}
{quote}
# Tables with no primary key cannot use {{chunkKeyColumn}} at all — the method
throws before even checking it.
The same feature works correctly in MySQL, Postgres, and Oracle connectors, all
of which search {{{}table.columns(){}}}.
*File location:*
[https://github.com/apache/flink-cdc/blob/master/flink-cdc-connect/flink-cdc-source-connectors/flink-connector-sqlserver-cdc/src/main/java/org/apache/flink/cdc/connectors/sqlserver/source/utils/SqlServerUtils.java]
*Steps to reproduce:*
# Have a SQL Server table with columns {{{}(id INT PRIMARY KEY, name VARCHAR,
created_at BIGINT){}}}.
# Configure the connector with {{{}scan.incremental.snapshot.chunk.key-column
= created_at{}}}.
# Start the job — it immediately throws {{{}ValidationException: Chunk key
column 'created_at' doesn't exist in the primary key [id]{}}}.
*Expected behavior:*
The connector should use {{created_at}} as the chunk key column, consistent
with MySQL/Postgres/Oracle behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
