LuciferYang opened a new issue, #10258: URL: https://github.com/apache/paimon/issues/10258
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master ### Compute Engine Flink (CDC write) ### Minimal reproduce step `MergeTreeWriter`'s restore constructor replays the pending increment's data files, changelog and compaction files but ignores the index file changes carried in `DataIncrement.newIndexFiles/deletedIndexFiles` and `CompactIncrement.newIndexFiles/deletedIndexFiles`. `StoreSinkWriteImpl.replace(FileStoreTable)` is called from the CDC write operators (for example `CdcRecordStoreWriteOperator`) when a record forces a schema refresh. It runs `write.checkpoint()` then `write.restore(states)` in process, with no commit in between. During `checkpoint()`, the primary-key index maintainer finishes and accepts a pending build and merges the resulting index file additions and deletions into the increment stored in the writer state, advancing its own state past that build. On `restore()` the same increment is handed to the new `MergeTreeWriter`, whose constructor drops the index file changes, and the maintainer no longer has a pending build to re-produce them. ### What doesn't meet your expectations? The accepted index payloads should reach the index manifest. Instead they are silently dropped, so a primary-key full-text (or vector or sorted) index misses the rows covered by that payload, and the payloads they replaced are never deleted, staying as zombie manifest entries. ### Anything else? _No response_ ### Are you willing to submit a PR? - [X] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
