hotcache opened a new pull request, #3046:
URL: https://github.com/apache/iceberg-rust/pull/3046
## Which issue does this PR close?
Part of #1607.
This is the first in a series of PRs to add compaction/rewrite support
to iceberg-rust. Follow-up PRs will add manifest caching, OverwriteFiles,
RowDelta, and conflict detection.
## What changes are included in this PR?
Introduces `MergingSnapshotProducer` and `RewriteFilesAction` — the Rust
equivalents of Java's `MergingSnapshotProducer` and `BaseRewriteFiles`.
**MergingSnapshotProducer** (`merging.rs`) is a shared engine for
snapshot-producing operations that both add and delete data files:
- `ManifestFilterManager` rewrites existing manifests to remove deleted
file entries, supporting partition-evolved tables (non-default specs)
- Writes new manifests for added files
- Computes snapshot summaries accounting for both additions and removals
- Delegates final snapshot/manifest-list creation to `SnapshotProducer`
**RewriteFilesAction** (`rewrite.rs`) is a thin action on top of
`MergingSnapshotProducer` that replaces data files (compaction). It
validates that at least one file is deleted and one added, and commits
with `Operation::Replace`.
### Design
As discussed in #2620, `MergingSnapshotProducer` uses composition
rather than inheritance — each action owns an instance and configures it
with its own operation type and validation rules. This aligns with
@JanKaul's [recent
analysis](https://github.com/apache/iceberg-rust/pull/2620#issuecomment-5366874140)
that the MergeCache-per-action approach converges to the same struct
field-for-field.
### Planned follow-ups
- **PR2**: Manifest cache for retry optimization
- **PR3**: `OverwriteFilesAction` with explicit file mode
- **PR4**: `overwrite_by_row_filter(Expression)` for retry-safe overwrites
- **PR5**: `RowDeltaAction` for position/equality delete files
- **PR6**: Snapshot validation (conflict detection)
## Are these changes tested?
7 tests covering:
- E2E compaction (3 files → 1 merged)
- Partial rewrite (2 of 3 files)
- Missing delete target → error
- Empty table rewrite → error
- Empty manifest omission
- Validation: no deletes → error, no adds → error
## AI Disclosure
AI (Claude) was used to assist with code review, test generation, and
PR description drafting. All code was reviewed and validated by the author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
