SEPURI-SAI-KRISHNA opened a new pull request, #13308:
URL: https://github.com/apache/seatunnel/pull/13308

   Yes, glad to contribute to Apache SeaTunnel.
   
   ### Purpose of this pull request
   
   Closes #13307.
   
   With `schema-changes.enabled = true`, MySQL-CDC ignored `ALTER TABLE ... 
RENAME COLUMN old TO new`, although the MySQL-CDC page lists `rename column` as 
supported. `CustomAlterTableParserListener` had no handler for `RENAME COLUMN`, 
so the statement gave no event (`Unsupported resolve schemaChangeEvent ..., 
just skip.`). Debezium applies the rename to its own table model, so the 
following change records carry the new name, while the SeaTunnel schema keeps 
the old one and the row converter sets the missing field to NULL. In 
`MysqlCDCWithSchemaChangeIT` this fails the JDBC sink with `Column 'add_column' 
cannot be null`; a nullable column gets NULL instead of its values without an 
error.
   
   Changes:
   
   - `CustomAlterTableParserListener.enterAlterByRenameColumn` emits 
`AlterTableChangeColumnEvent.change(table, old, column)`, where the column 
carries only the new name, as the Oracle-CDC listener does for its `RENAME 
COLUMN` (#9314). `RENAME COLUMN` cannot change the definition, and 
`AbstractSchemaChangeResolver.completeSchemaChangeEvents` already fills in the 
definition of the old column from the current catalog table, so 
`connector-cdc-base` needs no change.
   - A rename that changes only the case (`name TO NAME`) gives no event: 
Debezium keeps the existing name in that case, and its records keep carrying it.
   - Docs: the `change.column` row of the event filter table on the MySQL-CDC 
page (EN and ZH) names both `CHANGE COLUMN` and `RENAME COLUMN`.
   
   Not covered here:
   
   - A rename that names the old column in another case (`RENAME COLUMN NAME TO 
full_name` for `name`) fails while completing the event, like `CHANGE COLUMN 
NAME ...` on dev. #13297 resolves such names before completion and covers both.
   - Swapping or cycling names in one statement (`RENAME COLUMN a TO b, RENAME 
COLUMN b TO a`): MySQL resolves every clause against the original columns, 
while Debezium 1.9.8 applies them in order and loses a column in its own model.
   - Renaming a primary key column keeps the old name in the schema's primary 
key, because `AlterTableSchemaEventHandler` copies the primary key unchanged. 
`CHANGE COLUMN` has the same limit on dev; it is a separate fix.
   
   I searched open and closed PRs and issues for `RENAME COLUMN` and checked 
the diffs of the open PRs that touch MySQL-CDC parsing or DDL files (#12014, 
#11162, #12284, #12258, #13297): none handles `RENAME COLUMN` (#11162 only 
mentions it in the schema evolution guide). This PR and #13297 change different 
lines of the MySQL-CDC page and no common file otherwise.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes. With `schema-changes.enabled = true`, `RENAME COLUMN` is applied 
downstream as a column rename, like `CHANGE COLUMN`, and the renamed column 
keeps its values. It can be filtered as `change.column`. No option, default or 
API changes.
   
   ### How was this patch tested?
   
   - `CustomMySqlAntlrDdlParserTest#testParseAlterTableRenameColumn`: `RENAME 
COLUMN c_name TO c_full_name, ADD COLUMN c_note ...` gives a change event with 
the old and new names followed by the add event. 
`#testParseAlterTableRenameColumnOnlyInCaseHasNoEvent`: `c_name TO C_NAME` and 
`c_name TO c_name` give no event. With the listener of dev the first test fails 
(1 event instead of 2), and with a case-sensitive comparison the second one 
fails.
   - Locally, against Debezium's `MySqlAntlrDdlParser` table model, the 
resulting schema matched for a plain rename (type and length kept), a case-only 
rename, a rename with `ADD ... AFTER` the new name, a rename with `MODIFY` of 
the new name, and backquoted names with spaces.
   - `connector-cdc-base` 143 and `connector-cdc-mysql` 61 unit tests pass.
   - E2E: `ddl/change_columns.sql` gains a `RENAME COLUMN add_column TO 
add_column_renamed` step followed by rows. The column keeps the new name: the 
sink may apply schema changes later than the source, and a name used only in 
between would be missing from the sink while the source no longer has it. 
`comment_changes.sql` documents the new table state.
     - `MysqlCDCWithSchemaChangeIT#testMysqlCdcWithSchemaEvolutionCase` and 
`#testMysqlCdcWithSchemaEvolutionCaseExactlyOnce` (Zeta) pass; each sink ran 
`CHANGE COLUMN add_column add_column_renamed INT NOT NULL DEFAULT 1`. With the 
listener of dev both fail: the sink keeps `add_column` and fails with `Column 
'add_column' cannot be null`.
     - `MysqlCDCWithFlinkSchemaChangeIT` passes on Flink 1.13.6, 1.15.3, 1.18.0 
and 1.20.1 (8 runs).
   - `spotless:check`, `MarkdownTest`, `git diff --check`.
   
   ### Check list
   
   * [x] If any new Jar binary package adding in your PR, please add License 
Notice according
     [New License 
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
 (no new jar)
   * [x] If necessary, please update the documentation to describe the new 
feature. https://github.com/apache/seatunnel/tree/dev/docs
   * [x] If necessary, please update `incompatible-changes.md` to describe the 
incompatibility caused by this PR. (not needed: no incompatible change)
   * [ ] If you are contributing the connector code, please check that the 
following files are updated: (not a new connector)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to