[ 
https://issues.apache.org/jira/browse/FLINK-41498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-41498:
-----------------------------------
    Labels: pull-request-available  (was: )

> OceanBase pipeline sink ignores lenient mode and exclude.schema.changes, 
> dropping and truncating target tables
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-41498
>                 URL: https://issues.apache.org/jira/browse/FLINK-41498
>             Project: Flink
>          Issue Type: Bug
>          Components: Flink CDC
>    Affects Versions: cdc-3.7.0, cdc-3.8.0
>            Reporter: Jubin Soni
>            Priority: Critical
>              Labels: pull-request-available
>
> h2. Bug / gap
> OceanBaseMetadataApplier does not override 
> MetadataApplier#setAcceptedSchemaEvolutionTypes, #acceptsSchemaEvolutionType 
> or #getSupportedSchemaEvolutionTypes. The interface defaults discard the 
> configured event types and accept every schema change type. 
> SchemaDerivator#normalizeSchemaChangeEvents filters schema change events only 
> through metadataApplier.acceptsSchemaEvolutionType(...), so for the OceanBase 
> sink nothing is filtered. With the default schema.change.behavior (lenient), 
> the parser excludes drop.table and truncate.table, but upstream DROP TABLE / 
> TRUNCATE TABLE events are still forwarded to 
> OceanBaseMetadataApplier#applySchemaChange, which calls catalog.dropTable / 
> catalog.truncateTable on the target table. The same happens when a user 
> explicitly sets sink.exclude.schema.changes: [drop.table, truncate.table], or 
> uses include.schema.changes.
> h2. Reproduction
> Pipeline (default lenient behavior, no include/exclude options):
> {code}
> source:
>   type: mysql
>   …
> sink:
>   type: oceanbase
>   url: jdbc:mysql://<host>:2881/test
>   username: root@test
>   password: ‘'
> {code}
> Resolving the accepted types with YamlPipelineDefinitionParser, passing them 
> to OceanBaseMetadataApplier#setAcceptedSchemaEvolutionTypes, and running 
> SchemaDerivator#normalizeSchemaChangeEvents on a TruncateTableEvent and a 
> DropTableEvent (released 3.7.0-1.20 jars; the four files involved are 
> unchanged on master):
> {code}
> resolved accepted types = [ADD_COLUMN, ALTER_COLUMN_TYPE, CREATE_TABLE, 
> DROP_COLUMN, RENAME_COLUMN, ALTER_TABLE_COMMENT]
> acceptsSchemaEvolutionType(DROP_TABLE)     = true
> acceptsSchemaEvolutionType(TRUNCATE_TABLE) = true
> events forwarded to applySchemaChange = 
> [TruncateTableEvent{tableId=app_db.orders}, 
> DropTableEvent{tableId=app_db.orders}]
> {code}
> Expected:
> {code}
> acceptsSchemaEvolutionType(DROP_TABLE)     = false
> acceptsSchemaEvolutionType(TRUNCATE_TABLE) = false
> events forwarded to applySchemaChange = []
> {code}
> h2. Why it matters
> The schema evolution docs state that in lenient mode (the default) 
> "TruncateTableEvent and DropTableEvent will not be sent to downstream to 
> avoid unexpected data loss". For the OceanBase sink this does not hold: a 
> TRUNCATE TABLE or DROP TABLE on the MySQL source deletes the data or the 
> whole table in OceanBase, and per-event-type exclusion cannot prevent it.
> h2. Related work
> FLINK-41107 (PR #4599, fixed in cdc-3.8.0) fixed the same gap for 
> HudiMetadataApplier and MaxComputeMetadataApplier. OceanBaseMetadataApplier 
> has the same gap and was not covered. All other pipeline sinks (Doris, 
> StarRocks, Paimon, Iceberg, Fluss, Kafka, Values) already override these 
> methods. Searched JIRA for "oceanbase", "acceptsSchemaEvolutionType", 
> "setAcceptedSchemaEvolutionTypes", "exclude.schema.changes", and Flink CDC 
> issues mentioning MetadataApplier / drop.table / truncate.table, and GitHub 
> PRs for OceanBaseMetadataApplier and the same method names, on 2026-11-14: no 
> existing ticket or PR for OceanBase.
> h2. Proposed fix
> Keep the enabled schema evolution types in OceanBaseMetadataApplier 
> (initialised to its supported types), override 
> setAcceptedSchemaEvolutionTypes / acceptsSchemaEvolutionType to honour them, 
> and override getSupportedSchemaEvolutionTypes with the types the applier 
> actually implements, following DorisMetadataApplier and the FLINK-41107 
> change.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to