[
https://issues.apache.org/jira/browse/FLINK-41434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-41434:
-----------------------------------
Labels: pull-request-available (was: )
> Fluss CDC source rebuilds the column mapping for every record with an older
> schema
> ----------------------------------------------------------------------------------
>
> Key: FLINK-41434
> URL: https://issues.apache.org/jira/browse/FLINK-41434
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Reporter: Xiaobing Fang
> Priority: Major
> Labels: pull-request-available
>
> After a column is appended to a Fluss table, a source reader can see the new
> schema and continue receiving records written with the previous schema.
> To reproduce, use a 453-column table, append one nullable column, and let the
> same reader process a 454-column record followed by multiple 453-column
> records.
> For every older record, FlussRecordDeserializer rebuilds both Fluss Schema
> objects and recomputes the same column mapping. Building the schemas also
> repeats duplicate-name checks whose cost grows quadratically with the number
> of columns. In a local benchmark alternating the two versions,
> deserialization consumed 551 µs of thread CPU per record, compared with 3.26
> µs when all records used the latest version.
> The reader should compute the mapping once for each source and target schema
> version pair and reuse it for subsequent records. Older records should
> continue to emit null for the appended column.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
