LuciferYang opened a new pull request, #10260:
URL: https://github.com/apache/paimon/pull/10260

   ### Purpose
   
   `MergeTreeWriter`'s restore constructor replayed the pending increment's 
data files, changelog and compaction files but dropped the index file changes 
in `DataIncrement` and `CompactIncrement`.
   
   On the CDC schema-change path, `StoreSinkWriteImpl.replace` runs 
`checkpoint()` then `restore()` in process. During `checkpoint()` the 
primary-key index maintainer finishes and accepts a pending build and merges 
its index file additions and deletions into the writer state increment, 
advancing its own state past that build. On `restore()` the new writer dropped 
those changes and the maintainer had nothing left to re-emit, so the accepted 
payloads never reached the index manifest and the replaced payloads stayed as 
zombie entries.
   
   This carries `newIndexFiles` and `deletedIndexFiles` from both the data and 
compact increments through the restore replay and re-emits them on the next 
`prepareCommit`.
   
   This closes #10258.
   
   ### Tests
   
   Added `MergeTreeWriterCloseFailureTest.testRestoreReplayKeepsIndexFiles`: it 
builds a restore increment carrying a new payload in the data increment and a 
removed payload in the compact increment, restores a writer from it, and 
asserts the next `prepareCommit` re-emits both. The assertion fails against the 
pre-fix constructor and passes with the fix.
   
   ### API and Format
   
   No.
   
   ### Documentation
   
   No.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to