terrytlu opened a new pull request, #17254: URL: https://github.com/apache/iceberg/pull/17254
When two Iceberg tables share the same underlying location (for example a test table created by copying SHOW CREATE TABLE but forgetting to change the LOCATION), running orphan-file cleanup for one table walks the whole location and deletes every file not reachable from that single table's metadata, silently corrupting the other table. Add a locationConflictMode(LocationConflictMode) option to DeleteOrphanFiles, defaulting to ERROR. Before deleting anything, it detects whether another table shares the metadata location by comparing the immutable table-uuid stored in each metadata.json (also handling .metadata.json.gz). A missing or unreadable uuid (legacy file without a uuid, corrupt, or compressed) is always treated as a conflict (fail-safe). In ERROR mode it aborts when a conflict is detected; in IGNORE mode the cleanup is skipped entirely and the other table's files are preserved; in DELETE mode cleanup proceeds even if a conflict is detected (legacy behavior), and should be used only when you are certain no other table shares the location. The detection helper lives in core (OrphanFileUtils.hasOtherTableInLocation) and is wired into DeleteOrphanFilesSparkAction for Spark 3.5/4.0/4.1. The Spark remove_orphan_files procedure also exposes a location_conflict_mode parameter, and docs/spark-procedures.md documents it. Core and Spark procedure tests cover the conflict / no-conflict / null-uuid / compressed / corrupt cases. Fix #17213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
