wuhainan opened a new issue, #13157: URL: https://github.com/apache/seatunnel/issues/13157
### Search before asking - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement. ### Description ### Background SeaTunnel supports schema evolution for CDC pipelines. However, when a pipeline runs on the Flink engine and includes a SQL Transform, schema change events are not currently handled by the transform as schema changes. This makes it difficult to use data transformation and automatic schema evolution together in a Flink-based CDC pipeline. ### Feature request Support schema evolution through SQL Transform when running SeaTunnel jobs on the Flink engine. When a source emits a schema change event, the pipeline should: 1. Deliver the schema change to the transform and update its runtime schema. 2. Propagate the resulting output schema change to the sink when needed. 3. Keep transformed records consistent with the sink schema before and after the change. 4. Fail with a clear error when a schema change cannot be safely handled. The behavior should also be verified with parallel execution and checkpoint/savepoint recovery. This request focuses on the Flink execution path and can build on the SQL Transform schema handling in #12284. ### Usage Scenario A MySQL CDC job runs on the Flink engine and synchronizes data to TiDB through the JDBC sink. The job uses a SQL Transform to mask sensitive fields, such as phone numbers, before writing them to TiDB. When the source table schema changes, we want the pipeline to handle supported schema changes without manually stopping and rebuilding the job. For example, if a source column used by the transform changes type, the pipeline should update the transform and sink schemas as appropriate. If a change cannot be applied safely, the job should fail with a clear error. The behavior for newly added columns should follow the SQL projection: columns that are not part of the transform output should not be written to the sink automatically. ### Related issues - #8253 — SQL Transform support for schema changes. - #12284 — SQL Transform schema change handling for the Zeta engine. - #11752 — Schema change propagation to parallel sink writers on Flink. - #11402 — Discussion of schema change ordering and recovery. ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
