zhoulii commented on code in PR #11272: URL: https://github.com/apache/paimon/pull/11272#discussion_r4649605152
########## docs/docs/multimodal-table/data-evolution-maintenance.mdx: ########## @@ -89,6 +89,20 @@ Ordinary compaction **does not physically remove rows hidden by deletion vectors**. Removing their positions would change the alignment with untouched column files. +### Recovering Files After a Failed Rewrite + +Flink and Spark rewrite DV indexes on workers. A failed worker attempt cleans +up the new DV files it has tracked. Files from tasks that already succeeded, +or from a lost worker process, can remain unreferenced if the overall rewrite +fails before committing. + +After the failed job and its retries have stopped, use +[Remove Orphan Files](../maintenance/manage-snapshots#remove-orphan-files) Review Comment: Clarified that `remove_orphan_files` does not scan external index roots and documented manual recovery with ownership and retained-reference checks. Extended the failed Spark job test to verify this limitation and recovery of recorded orphan paths. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
