corgy-w opened a new pull request, #11826:
URL: https://github.com/apache/seatunnel/pull/11826
### Purpose of this pull request
Fix the issue of out-of-order fields in CDC when `column.include.list` (or
other column-subsetting) is configured.
In `MySqlSnapshotSplitReadTask`, the snapshot reader picked the column
definition with `table.columns().get(i)`, which returns columns in the **table
DDL order**. When Debezium's `column.include.list` filters/orders the columns,
the actual column order in the query result set differs from the table order,
causing field values to be written into the wrong positions.
The fix uses `columnArray.getColumns()[i]`, which reflects the actual order
of the columns in the executed query (matching Debezium's own
`JdbcConnection.rowToArray` behavior).
This is a re-work of the previously closed PR #9838, rebased on the latest
`dev`.
### Does this PR introduce _any_ user-facing change?
No. It fixes incorrect field ordering when reading CDC data with
`column.include.list`.
### How was this patch tested?
- Added E2E test `testMysqlCdcWithColumnIncludeList` in
`AbstractMysqlCDCITBase`: configures `column.include.list` with 10 columns
(excluding others) in a MySQL CDC source, verifies snapshot data and then
stream-stage insert/update/delete data are correctly written to the sink in the
right column order.
- New job config: `mysqlcdc_to_mysql_with_column_include_list.conf`.
- Verified locally: `spotless:apply` clean, cdc-mysql module installs and
`connector-cdc-mysql-e2e` test code compiles (the E2E test itself will run in
CI).
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [ ] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the
following files are updated:
1. Update
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
and add new connector information in it
2. Update the pom file of
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
