twuebi opened a new pull request, #1033: URL: https://github.com/apache/iceberg-go/pull/1033
Decompose the atomic rewrite-snapshot of `Transaction.RewriteDataFiles` which assumes in-process compaction to enable out-of-process distributed compaction, by exposing a builder similar to what iceberg-java has with `Table.newRewrite()`. Mirrors iceberg-java's RewriteFiles (Table.newRewrite()), sitting beside RowDelta. Suppresses the overwrite producer's default validator and queues a rewrite-specific conflict validator internally, so RewriteDataFiles and external callers (distributed compaction coordinators) commit the same shape through one indivisible operation. Transaction.NewRewrite(snapshotProps) returns *RewriteFiles with DeleteFile / AddDataFile / Apply / Commit. ExecuteCompactionGroup is the worker-side read+write step, returning a plain-data CompactionGroupResult. RewriteDataFiles drives the builder internally. Partial-progress mode now commits per group inside the loop, fixing a latent bug where a mid-loop failure left staged groups uncovered by the rewrite validator. Tag rewrite snapshots OpReplace instead of OpOverwrite (matching iceberg-java's BaseRewriteFiles.operation()) — gated on cfg.rewriteSemantics so generic ReplaceFiles callers, where row content can change, keep OpOverwrite. This also addresses #841 and should address the comments made in #867 since only the unexported `cfg.rewriteSemantics` makes OpOverwrite -> OpReplace and `RewriteFiles.Commit()` is the only caller of `withRewriteSemantics()`. Black-box coverage: equivalence with RewriteDataFiles, safe pos-delete expunge, concurrent eq-delete rejection under refresh-and-replay. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
