rahulsmahadev opened a new pull request, #17622: URL: https://github.com/apache/iceberg/pull/17622
Spark implementation of the `RepairTable` action whose API was added in #17399. `repairFileMetrics()` compares the metrics recorded in each live manifest entry (record count, file size, column bounds/null/nan/value counts) against the underlying data and delete files, and rewrites only the manifests that contain an incorrect entry. Rewritten entries carry through their original snapshot id and sequence numbers, so the repair does not change which delete files apply. `dryRun()` reports what would be repaired without committing. Scoped to Spark 4.1 for now; happy to backport once this lands. Tests in `TestRepairTableAction` cover: no-op on correct stats, record-count/file-size/column-metric repair, dry run, partitioned tables, and concurrency (concurrent append, a conflicting manifest rewrite, commit-state-unknown, and cleanup of manifests written by a failed or dry-run repair). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
