Hi, Andrei.
Many thanks for your review, extremely useful! I pushed code updates for three of your points: 1. Content-based op selection (you were right, we had overwrite hardcoded — and it was ~15 lines) 2. Write-time rejection of empty/unknown/float-double equality keys, and an explicit note on the action that BaseRowDelta's conflict-validation surface is deferred. 3. On #2786, we documented the 204-reload contract and the 409-vs-5xx atomicity split, and gave 503 an explicit commit-state-unknown arm. Your "retrying those blindly double-applies" line is now paraphrased in the doc comment. Some of your feedback could be happily checked off as done. The strict-greater sequence rule was already in the Rust reader, and the PR's round-trip test (test_row_delta_write_and_read_back) turns out to exercise your delete-and-reinsert-same-PK case specifically, through the scan path with equality deletes applied. We hit the same worry about position-only tests hiding stubbed equality reads. equality_ids is int on our side, and the null-key reader fix landed in the base branch just before our PR. One design divergence worth flagging: our prepare_commit consumes the transaction rather than inspecting it. Rust's ownership makes that the safer default — a prepared transaction can't also be committed directly, and the build-N-then-batch pattern still works. Curious whether that caused you any friction in practice on the Go side. I'd take you up on the diffs — the partition-scoped conflict detection especially, since that's the piece we deferred, and the dangling-equality-deletes-after-compaction cleanup, which we'll need downstream. Which repo/branches should I look at? Best, Steve From: Andrei Tserakhau via dev <[email protected]> Sent: Thursday, July 9, 2026 1:10 PM To: [email protected] Cc: Andrei Tserakhau <[email protected]> Subject: Re: [DISCUSS] Row-delta commits and multi-table transactions in iceberg-rust ? patterns from a CDC producer Hi Steve, I built this same thing in iceberg-go last year, RowDelta commit plus the REST multi-table commit, so here's what bit us. Most of it showed up after the initial API merged, in the read path and the edge cases, so worth knowing up front. On the RowDelta PR (#2785): The op-type selection is basically the whole trick and it's cheap, maybe 15 lines. Data only is append, deletes only is delete, both is overwrite. Same as Java's BaseRowDelta. The sequence-number rule is where correctness lives. Equality deletes apply only to data files with a strictly lower seq number (position deletes use >=, equality use >). That's what keeps the deletes in a RowDelta from killing the new rows added in the same commit. We test it directly: insert PK=3, then in one RowDelta delete PK=3 and re-insert it, scan should still see PK=3. Easy to get wrong and it fails silently. The commit side is small, ~250 lines. The read side is where the weeks go. We shipped the commit API well before the scanner could actually apply equality deletes (that's a hash-based anti-join, the hardest piece by far). That's fine for interop testing against Lakekeeper/Spark, but watch out: a round-trip test that only uses position deletes will pass while equality reads are still stubbed, so it hides the gap. Conflict validation you can defer, just say so in the comment. Flink skips most of it anyway since seq numbers carry correctness. It came back for us later as partition-scoped conflict detection, because the first cut used an always-true filter that serialized every concurrent writer. Things you'll hit next, all post-merge fixes for us: - null keys in equality delete files (null vs absent in the key encoding) - reject empty equality-key sets at write time - reject float/double equality columns (NaN and +-0 make equality undefined) - match deletes to data files by partition transform, not raw value - drop dangling equality deletes after compaction or they leak forever - equality_ids avro element type is int, not long (breaks cross-engine reads) On the multi-table commit PR (#2786): Keep it a separate optional trait, don't widen the base Catalog. Only REST implements it and callers type-assert. Split building the commit from committing it. We have a TableCommit() that pulls {identifier, requirements, updates} out of a transaction without consuming it, so you build N per-table txns and then batch the payloads. Your prepare_commit() looks like the same idea, just keep it inspect-only. Watch the 204: no refreshed metadata comes back and post-commit hooks can't run, so callers have to reload afterward. And the error mapping is the atomicity surface, 409 is commit-failed and retryable, but 500/502/503/504 are commit-state-unknown since the commit may or may not have applied. Retrying those blindly double-applies. Happy to point at the specific diffs if any of this is useful. Thanks, Andrei On Wed, Jul 8, 2026 at 7:04 PM <[email protected] <mailto:[email protected]> > wrote: Hi all, I'm building an open-source Postgres→Iceberg CDC pipeline on iceberg-rust, and two capabilities came up that felt broadly useful, so I've opened them as draft PRs to discuss: 1. RowDeltaAction (apache/iceberg-rust#2785 <https://github.com/apache/iceberg-rust/pull/2785> ) — the commit-side equivalent of Java's Table.newRowDelta(): data files + delete files (equality/position) in one overwrite snapshot. The crate already reads delete files; this adds the missing write/commit side. It relates to #2218 (it's the commit-side complement of a DeltaWriter) and #2186. Includes an end-to-end merge-on-read round-trip test. 2. Multi-table transaction commits (apache/iceberg-rust#2786 <https://github.com/apache/iceberg-rust/pull/2786> , issue #2784) — Transaction::prepare_commit() plus RestCatalog::commit_transaction(Vec) driving the REST spec's /v1/{prefix}/transactions/commit. Our use case: committing per-cadence row deltas to many tables plus a changelog table atomically, validated against Lakekeeper. Both are drafts and I'm happy to adapt the APIs to wherever the merge-on-read work in #2186 is heading. Feedback very welcome. Thanks, Steve
