[
https://issues.apache.org/jira/browse/FLINK-41498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18144962#comment-18144962
]
Jubin Soni commented on FLINK-41498:
------------------------------------
I'm working on this.
> OceanBase pipeline sink ignores lenient mode and exclude.schema.changes,
> dropping and truncating target tables
> --------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-41498
> URL: https://issues.apache.org/jira/browse/FLINK-41498
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.7.0, cdc-3.8.0
> Reporter: Jubin Soni
> Priority: Critical
>
> h2. Bug / gap
> OceanBaseMetadataApplier does not override
> MetadataApplier#setAcceptedSchemaEvolutionTypes, #acceptsSchemaEvolutionType
> or #getSupportedSchemaEvolutionTypes. The interface defaults discard the
> configured event types and accept every schema change type.
> SchemaDerivator#normalizeSchemaChangeEvents filters schema change events only
> through metadataApplier.acceptsSchemaEvolutionType(...), so for the OceanBase
> sink nothing is filtered. With the default schema.change.behavior (lenient),
> the parser excludes drop.table and truncate.table, but upstream DROP TABLE /
> TRUNCATE TABLE events are still forwarded to
> OceanBaseMetadataApplier#applySchemaChange, which calls catalog.dropTable /
> catalog.truncateTable on the target table. The same happens when a user
> explicitly sets sink.exclude.schema.changes: [drop.table, truncate.table], or
> uses include.schema.changes.
> h2. Reproduction
> Pipeline (default lenient behavior, no include/exclude options):
> {code}
> source:
> type: mysql
> …
> sink:
> type: oceanbase
> url: jdbc:mysql://<host>:2881/test
> username: root@test
> password: ‘'
> {code}
> Resolving the accepted types with YamlPipelineDefinitionParser, passing them
> to OceanBaseMetadataApplier#setAcceptedSchemaEvolutionTypes, and running
> SchemaDerivator#normalizeSchemaChangeEvents on a TruncateTableEvent and a
> DropTableEvent (released 3.7.0-1.20 jars; the four files involved are
> unchanged on master):
> {code}
> resolved accepted types = [ADD_COLUMN, ALTER_COLUMN_TYPE, CREATE_TABLE,
> DROP_COLUMN, RENAME_COLUMN, ALTER_TABLE_COMMENT]
> acceptsSchemaEvolutionType(DROP_TABLE) = true
> acceptsSchemaEvolutionType(TRUNCATE_TABLE) = true
> events forwarded to applySchemaChange =
> [TruncateTableEvent{tableId=app_db.orders},
> DropTableEvent{tableId=app_db.orders}]
> {code}
> Expected:
> {code}
> acceptsSchemaEvolutionType(DROP_TABLE) = false
> acceptsSchemaEvolutionType(TRUNCATE_TABLE) = false
> events forwarded to applySchemaChange = []
> {code}
> h2. Why it matters
> The schema evolution docs state that in lenient mode (the default)
> "TruncateTableEvent and DropTableEvent will not be sent to downstream to
> avoid unexpected data loss". For the OceanBase sink this does not hold: a
> TRUNCATE TABLE or DROP TABLE on the MySQL source deletes the data or the
> whole table in OceanBase, and per-event-type exclusion cannot prevent it.
> h2. Related work
> FLINK-41107 (PR #4599, fixed in cdc-3.8.0) fixed the same gap for
> HudiMetadataApplier and MaxComputeMetadataApplier. OceanBaseMetadataApplier
> has the same gap and was not covered. All other pipeline sinks (Doris,
> StarRocks, Paimon, Iceberg, Fluss, Kafka, Values) already override these
> methods. Searched JIRA for "oceanbase", "acceptsSchemaEvolutionType",
> "setAcceptedSchemaEvolutionTypes", "exclude.schema.changes", and Flink CDC
> issues mentioning MetadataApplier / drop.table / truncate.table, and GitHub
> PRs for OceanBaseMetadataApplier and the same method names, on 2026-11-14: no
> existing ticket or PR for OceanBase.
> h2. Proposed fix
> Keep the enabled schema evolution types in OceanBaseMetadataApplier
> (initialised to its supported types), override
> setAcceptedSchemaEvolutionTypes / acceptsSchemaEvolutionType to honour them,
> and override getSupportedSchemaEvolutionTypes with the types the applier
> actually implements, following DorisMetadataApplier and the FLINK-41107
> change.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)