SergeyTimofeev1989 opened a new issue, #18608:
URL: https://github.com/apache/iceberg/issues/18608
### Problem / Motivation
The Configuration page documents three isolation-level table properties:
- `write.delete.isolation-level`
- `write.update.isolation-level`
- `write.merge.isolation-level`
There is no explicit note that these properties are scoped **only** to
row-level operations
(DELETE, UPDATE, MERGE) and have **no effect** on append/INSERT operations.
As a result, users
reasonably assume that setting one of these properties can reduce commit
conflicts or commit
duration for append-heavy workloads. This is a common source of
misunderstanding when
diagnosing 409 `CommitFailedException` responses from a REST catalog.
### Real-world scenario that motivated this request
We run 5 Iceberg tables, each written by ~30 independent Flink append jobs
with a 1-minute
checkpoint interval (≈27 commits/minute per table), committed through a REST
catalog
(Lakekeeper). We observed frequent 409 responses from the catalog
(`CommitFailedException`,
`assert-ref-snapshot-id` not satisfied), with commit retries.
Our team set `write.delete.isolation-level = snapshot` on these tables,
expecting it to reduce
commit conflicts or commit duration. It had **no effect** on the append
path, because:
1. The Flink append path commits via `AppendFiles` (`table.newAppend()`),
which does **not**
invoke `validateNoConflictingData` or any data-conflict validation.
2. The `write.*.isolation-level` properties are read by Spark's DELETE /
UPDATE / MERGE paths,
not by append.
3. A 409 from the REST catalog is a **metadata-pointer conflict**, resolved
by retry regardless
of isolation level (see the "Concurrent write operations" / "Retry
validation" sections on
the Reliability page).
### Proposed change
Add an explicit note to the documentation stating that:
- `write.delete.isolation-level`, `write.update.isolation-level`, and
`write.merge.isolation-level`
apply only to row-level operations (DELETE, UPDATE, MERGE).
- Append/INSERT has **no isolation level**; concurrent append conflicts are
resolved at the
metadata level via retry, and there is no data-conflict check to configure.
Suggested locations (either or both):
- **Configuration page** → Write properties section, next to the three
isolation-level rows.
- **Reliability page** → "Concurrent write operations" / "Retry validation"
subsection, as a
short cross-reference clarifying that retry-based conflict resolution
applies to append.
### Additional context
- Iceberg versions in use: 1.8.x (Spark maintenance) and 1.11.0 (Flink
2.1.2, `iceberg-flink-runtime-2.1`).
- Catalog: REST (Lakekeeper), S3 storage with remote signing.
- The confusion is not about a missing feature — it is about the docs not
making the
append/row-level scoping of the `write.*.isolation-level` properties
explicit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
