junmuz opened a new pull request, #11072:
URL: https://github.com/apache/paimon/pull/11072

   Notify commit callbacks after successful manifest-list replacement and 
reconcile Iceberg metadata against the committed snapshot when no file delta is 
available.
   
   Handle retry recovery and deletion-vector refresh, with regression coverage 
for append and primary-key repairs, metadata retries, and deletion-vector index 
replacement.
   
   ### Purpose
   
   remove_unexisting_manifests repairs a Paimon table by replacing its manifest 
lists without recording a file-change delta. This bypassed commit callbacks, 
leaving Iceberg metadata referencing files excluded from the repaired Paimon 
snapshot.
   
   Add a callback that runs after successful manifest replacement, outside the 
commit lock. The Iceberg implementation compares the repaired snapshot’s 
visible files against existing Iceberg manifests and publishes updated metadata 
immediately, without requiring another write or compaction.
   
   The change also handles empty-delta overwrite snapshots during retries, 
refreshes deletion-vector manifests, and updates changed-partition statistics.
   
   ### Tests
   • IcebergCompatibilityTest: 69 tests, including seven new regression cases 
covering partial and complete manifest removal, primary-key repair without 
further compaction, retry recovery with or without previous Iceberg metadata, 
and retained or cleared deletion-vector indexes.
   
   • RemoveUnexistingManifestsTest: 3 tests.
   
   • Targeted FileStoreCommitTest methods: 5 cases covering legacy snapshots 
and lease-loss protection.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to