abjain123 opened a new issue, #18713:
URL: https://github.com/apache/iceberg/issues/18713
### Apache Iceberg version
1.11.0
### Query engine
None
### Please describe the bug 🐞
When Write-Audit-Publish (WAP) is used via spark.wap.id on a format-version
3 table where deletion vectors (DVs) are in use, WAP snapshot isolation is
broken in two related ways:
**Failure mode 1 — WAP APPEND (INSERT):**
Staged data becomes visible on the main branch before cherry-pick. Reading
the table from the main branch after a WAP INSERT (without cherry-picking)
incorrectly returns the staged rows. Expected: main branch should not
include any data from a staged WAP snapshot.
**Failure mode 2 — WAP DELETE with write.delete.mode=merge-on-read:**
The staged WAP snapshot does not appear in <table>.snapshots filtered by
its wap.id. SELECT snapshot_id FROM <table>.snapshots WHERE summary['wap.id'] =
'<wap-id>' returns 0 rows, making any subsequent cherrypick_snapshot call
impossible.
**Steps to Reproduce**
Using any Spark + Iceberg catalog (Hadoop, Hive, REST). Verified with Spark
3.5.x + Iceberg 1.10.1.
**Failure mode (WAP DELETE):**
_-- Step 1: Create format-version 3 table with MOR DELETE
CREATE TABLE test_wap_dv (id INT, name STRING, dept STRING)
USING iceberg
TBLPROPERTIES (
'format-version' = '3',
'write.delete.mode' = 'merge-on-read'
);
-- Step 2: Insert rows
INSERT INTO test_wap_dv
VALUES (1, 'Alice', 'Eng'), (2, 'Bob', 'HR'), (3, 'Carol', 'Eng');
-- Step 3: Enable WAP
ALTER TABLE test_wap_dv
SET TBLPROPERTIES ('write.wap.enabled' = 'true');
-- Step 4: Stage a DELETE under WAP
SET spark.wap.id = 'wap_stage_001';
DELETE FROM test_wap_dv WHERE dept = 'HR';
-- Succeeds with no exception
-- Step 5: Find staged snapshot (BUG: returns 0 rows)
SELECT snapshot_id, summary
FROM test_wap_dv.snapshots
WHERE summary['wap.id'] = 'wap_stage_001'
ORDER BY committed_at DESC LIMIT 1;
-- Expected: 1 row containing the staged snapshot ID
-- Actual: 0 rows
-- Step 6: Cherry-pick fails (no snapshot ID to use)
-- CALL <catalog>.system.cherrypick_snapshot(
-- table => '<db>.test_wap_dv',
-- snapshot_id => <id from step 5>);
-- Fails with IndexError / NullPointerException because step 5 returned
nothing_
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [x] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
