jubins opened a new pull request, #4625: URL: https://github.com/apache/flink-cdc/pull/4625
## What is the purpose of this pull request? Fixes [FLINK-41498](https://issues.apache.org/jira/browse/FLINK-41498). `OceanBaseMetadataApplier` does not override `MetadataApplier#setAcceptedSchemaEvolutionTypes` or `#acceptsSchemaEvolutionType`, so it falls back to the interface defaults: the configured set is discarded and every schema change type is accepted. `SchemaDerivator#normalizeSchemaChangeEvents` filters events only through `acceptsSchemaEvolutionType`, so the types resolved from `include.schema.changes` / `exclude.schema.changes` are ignored by the OceanBase sink. The worst case is the default `schema.change.behavior: lenient`. In lenient mode the YAML parser excludes `drop.table` and `truncate.table` by default, but the OceanBase sink still applies them, calling `OceanBaseCatalog#dropTable` / `#truncateTable` on the target. A `DROP TABLE` or `TRUNCATE TABLE` on the source then drops or empties the target table, losing data the user configured the pipeline to keep. Setting `exclude.schema.changes: [drop.table, truncate.table]` explicitly has no effect either. [FLINK-41107](https://issues.apache.org/jira/browse/FLINK-41107) (#4599) fixed the same gap in the Hudi and MaxCompute sinks. The OceanBase sink was not covered there. ## Brief change log - `OceanBaseMetadataApplier`: store the accepted schema evolution types, defaulting to the supported set; override `setAcceptedSchemaEvolutionTypes`, `acceptsSchemaEvolutionType` and `getSupportedSchemaEvolutionTypes`. The supported set is every `SchemaChangeEventType`, because `applySchemaChange` handles all eight event types (create/drop table, truncate table, add/drop/rename column, alter column type, alter table comment). This follows `DorisMetadataApplier` and the Hudi/MaxCompute fix in #4599. - `OceanBaseMetadataApplierTest`: add `testAcceptedSchemaEvolutionTypes`. --- ## Verifying this change This change added tests and can be verified as follows: - Added `OceanBaseMetadataApplierTest#testAcceptedSchemaEvolutionTypes`. It checks that `getSupportedSchemaEvolutionTypes()` returns all event types and that `drop.table` is accepted by default. After `setAcceptedSchemaEvolutionTypes` is called with every type except `drop.table` and `truncate.table`, it checks that `acceptsSchemaEvolutionType` returns `false` for `DROP_TABLE` and `TRUNCATE_TABLE` and `true` for `ADD_COLUMN`. Without the fix, the `DROP_TABLE` / `TRUNCATE_TABLE` assertions fail because the default implementation returns `true`. - Ran locally against an OceanBase container: without the fix the new test fails with `Expecting value to be false but was true` (the `DROP_TABLE` assertion); with the fix it passes. - `OceanBaseMetadataApplierTest` starts an OceanBase container through Testcontainers, so it needs Docker. Run it with: `mvn -pl flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-oceanbase -am -Dtest='OceanBaseMetadataApplierTest#testAcceptedSchemaEvolutionTypes' -Dsurefire.failIfNoSpecifiedTests=false test` - The other module tests (`OceanBaseMySQLCatalogTest`, `OceanBaseDataSinkFactoryTest`, `OceanBaseEventSerializationSchemaTest`) pass. `mvn spotless:check` and checkstyle pass for the module. ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable. The sink now honors the existing `include.schema.changes` / `exclude.schema.changes` options and lenient-mode defaults. --- ##### Was generative AI tooling used to co-author this PR? <!-- If generative AI tooling has been used in the process of authoring this PR, please change the checkbox below to `[X]` followed by the name of the tool, and uncomment the "Generated-by" line. See the ASF Generative Tooling Guidance for details: https://www.apache.org/legal/generative-tooling.html You are responsible for the quality and correctness of every change in this PR regardless of the tooling used. Low-effort AI-generated PRs will be closed. --> - [X] Yes (please specify the tool below) Claude Code Generated-by: Claude Code (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
