yuminn-k opened a new pull request, #13279:
URL: https://github.com/apache/seatunnel/pull/13279

   ### Purpose of this pull request
   
   Fixes #13207
   
   This is a contribution to Apache SeaTunnel. It lets SqlServer-CDC read 
tables from a database whose name contains `.`, such as `LIS6.0`.
   
   With `database-names = ["LIS6.0"]` and `table-names = 
["LIS6.0.dbo.LIS_ITEM"]`, source creation fails today with `Cannot get split 
'LIS6.0.dbo.LIS_ITEM' to get databaseName and tableName`, because 
`TablePath.of` accepts at most three dot-separated parts.
   
   Following the maintainer guidance in the issue, the global `TablePath` 
syntax is not changed. The connector resolves the name itself, using 
`database-names` as the source of truth.
   
   Changes:
   
   1. `SqlServerIncrementalSourceFactory#toTablePath` (connector): a name with 
more than three parts is split as `<database>.<schema>.<table>`, where schema 
and table are the last two parts and the database is everything before them. 
The database must be listed in `database-names`. Otherwise it throws 
`IllegalArgumentException` instead of guessing, so ambiguous input (for example 
a dot inside a schema or table name, or a database not listed in 
`database-names`) is rejected with a clear message. Names with three or fewer 
parts still go through `TablePath.of` unchanged. The same function is used for 
`table-names-config`.
   2. `SqlServerIncrementalSourceFactory#getCatalogTables` (connector): 
`CatalogTableUtil.getCatalogTables` / `Catalog#getTables` parse `table-names` 
with `TablePath.of`, so when a name has more than three parts the factory 
builds the `TablePath`s with `toTablePath` and looks them up in the SQL Server 
catalog directly. When every name has three or fewer parts, the existing 
`CatalogTableUtil.getCatalogTables` path is used as before.
   3. `SqlServerSourceConfigFactory` (connector): `table.include.list` for 
Debezium used everything after the first `.`, which turned 
`LIS6.0.dbo.LIS_ITEM` into `0.dbo.LIS_ITEM`. It now keeps the last two parts 
(`dbo.LIS_ITEM`). For ordinary `db.schema.table` names the result is the same 
as before.
   4. `ConnectorMetricsCalcContext#normalizeTableName` (Zeta engine): with only 
the connector fixed, the job still fails at runtime in the metrics path. 
`updateMetrics` calls `normalizeTableName`, which calls `TablePath.of(tableId)` 
on the row's table id and throws the same `Cannot get split 
'LIS6.0.dbo.LIS_ITEM'` error. This code came in with #10100 and is part of 
3.0.0. It now falls back to the original table id when `TablePath.of` throws 
`IllegalArgumentException`. For such an id, `TablePath.of("LIS6.0", "dbo", 
"LIS_ITEM").getFullName()` is the same string, so per-table metrics keep the 
key they were registered with. This is why the PR touches the engine as well as 
the connector.
   
   `SqlServerSnapshotFetchTask` is not changed: in the local run below the 
snapshot phase worked without changes there.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes. A SqlServer-CDC job whose database name contains `.` previously failed 
during source creation. It now starts when the database is listed in 
`database-names` and the table is written as `<database>.<schema>.<table>`. A 
name with more than three parts whose database prefix is not in 
`database-names` fails with a message explaining the requirement. Behavior for 
names with three or fewer parts is unchanged.
   
   ### How was this patch tested?
   
   Unit tests:
   
   - `SqlServerIncrementalSourceFactoryTest` (new cases): dotted database name 
resolves to `TablePath("LIS6.0", "dbo", "LIS_ITEM")`; unresolvable dotted names 
are rejected with `IllegalArgumentException`; `table.include.list` is 
`dbo.LIS_ITEM` for a dotted database name.
     `./mvnw -pl seatunnel-connectors-v2/connector-cdc/connector-cdc-sqlserver 
test -Dtest=SqlServerIncrementalSourceFactoryTest`: `Tests run: 29, Failures: 
0, Errors: 0, Skipped: 0`
   - `ConnectorMetricsCalcContextTest#testUpdateMetricsWithDotInDatabaseName`: 
`updateMetrics` with table id `LIS6.0.dbo.LIS_ITEM` counts the row in the total 
and per-table counters. It fails without the engine change 
(`IllegalArgumentException: Cannot get split 'LIS6.0.dbo.LIS_ITEM' to get 
databaseName and tableName`) and passes with it.
     `./mvnw -pl seatunnel-engine/seatunnel-engine-server test 
-Dtest=ConnectorMetricsCalcContextTest`: `Tests run: 3, Failures: 0, Errors: 0, 
Skipped: 0`
   
   Local run against Azure SQL Edge (arm64) with the source settings from the 
issue (`database-names = ["LIS6.0"]`, `table-names = ["LIS6.0.dbo.LIS_ITEM", 
"LIS6.0.dbo.LIS_HISITEM"]`, `startup.mode = "initial"`):
   
   - Before this change the job failed with `Cannot get split 
'LIS6.0.dbo.LIS_ITEM'`, first in the source factory and, with only the 
connector changes applied, in the engine metrics path.
   - Snapshot: the Zeta job read the 5 snapshot rows (3 from 
`LIS6.0.dbo.LIS_ITEM`, 2 from `LIS6.0.dbo.LIS_HISITEM`).
   - Incremental: Azure SQL Edge has no CLR, so the SQL Server CDC LSN 
functions the connector relies on are not available there. The incremental 
phase was checked only with a test-only LSN shim: 5 change rows were read with 
the expected table ids (`LIS6.0.dbo.LIS_ITEM`: INSERT, UPDATE_BEFORE, 
UPDATE_AFTER; `LIS6.0.dbo.LIS_HISITEM`: DELETE, INSERT). The shim is not part 
of this PR.
   
   ### Check list
   
   * [ ] If any new Jar binary package adding in your PR, please add License 
Notice according
     [New License 
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
   * [ ] If necessary, please update the documentation to describe the new 
feature. https://github.com/apache/seatunnel/tree/dev/docs
   * [ ] If necessary, please update `incompatible-changes.md` to describe the 
incompatibility caused by this PR.
   * [ ] If you are contributing the connector code, please check that the 
following files are updated:
     1. Update 
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
 and add new connector information in it
     2. Update the pom file of 
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
     3. Add ci label in 
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
     4. Add e2e testcase in 
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
     5. Update connector 
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to