fallintoplace opened a new issue, #2413: URL: https://github.com/apache/iceberg-go/issues/2413
Track cleanup and compaction of delete files. ## In progress - [ ] Remove whole dangling position and equality delete files: #2408 - [ ] Rewrite v2 Parquet position delete files: #2409 ## Follow-ups - [ ] Support position delete files with row payloads and files written under older partition specs. - [ ] Consider position delete formats beyond Parquet and support for v3 tables. - [ ] Explore safe cleanup of dangling deletion vectors. - [ ] Reduce memory needed to sort position delete rows. The proposed rewrite holds decoded rows in memory and limits input file sizes per group. - [ ] Expand interoperability coverage with other Iceberg implementations. The PRs are still under review, so their scope may change. ## References - [Iceberg scan planning](https://iceberg.apache.org/spec/#scan-planning) and [deletion vectors](https://iceberg.apache.org/spec/#deletion-vectors) - [Java maintenance guide](https://iceberg.apache.org/docs/latest/maintenance/#rewrite-position-delete-files) - Java: [remove dangling delete files](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/actions/RemoveDanglingDeleteFilesAction.java), [rewrite position delete files API](https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/actions/RewritePositionDeleteFiles.java), and [Spark rewrite action](https://github.com/apache/iceberg/blob/main/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
