yadavay-amzn opened a new issue, #17206:
URL: https://github.com/apache/iceberg/issues/17206
### Apache Iceberg version
main (development)
### Query engine
Spark
### Please describe the bug 🐞
**Summary:** Once a table ends up with more than one deletion vector (DV)
referencing the same data file in a snapshot, **every scan of that table fails
during planning** with:
```
Can't index multiple DVs for <data-file>: <dv1> and <dv2>
```
thrown from `DeleteFileIndex.Builder.add(Map<String, DeleteFile> dvByPath,
DeleteFile dv)` (`core/src/main/java/org/apache/iceberg/DeleteFileIndex.java`,
~L529-533). The table becomes effectively unreadable and there is no built-in
way to recover it. This is core scan-planning code, so it affects every query
engine, not just Spark.
**Why it throws:** The DV index is built by iterating all live
delete-manifest entries in the snapshot (`DeleteFileIndex.Builder.build()` ->
`loadDeleteFiles()` -> the loop at ~L501). For each DV, `add(...)` does
`dvByPath.putIfAbsent(dv.referencedDataFile(), dv)` and throws a
`ValidationException` if an entry already exists for that path. There is no
deduplication or merge.
**Spec context:** The spec forbids more than one DV per data file per
snapshot ("there can be at most one deletion vector for a given data file in a
snapshot"; "Writers must ensure that there is at most one deletion vector per
data file and must merge new deletes with existing vectors or position delete
files"). So two DVs for one data file is always metadata corruption, and the
`ValidationException` is a *correct* detection of an invalid state - but it
leaves the table permanently unreadable.
**How a table reaches this state:** This is the read-side manifestation of a
write-side conflict-detection gap (see #15966). When two concurrent row-level
writers use a `conflictDetectionFilter` that is narrower than the set of data
files they actually touch, `validateAddedDVs` can miss the conflict and allow
two DVs for the same data file to be committed. #15966 fixes the *write* side
and prevents future corruption, but it cannot heal tables that are **already**
corrupted - those remain bricked.
**Steps to reproduce:** Produce a snapshot containing two live DV entries
that reference the same data file (for example via the concurrent
narrow-`conflictDetectionFilter` path described in #15966, or by crafting two
delete manifests that each contain a DV for the same data file), then run any
scan. Planning fails with `Can't index multiple DVs`.
**Expected behavior:** A reader should be able to recover from an
already-corrupted table without data loss. Rather than throwing (and leaving
the table unreadable), `DeleteFileIndex` should:
1. **Byte-identical duplicates** (same `location()`, `contentOffset()`,
`contentSizeInBytes()`) - the same DV referenced from two manifest entries
(e.g. a manifest-rewrite race): deduplicate silently, logging at DEBUG.
2. **Genuinely different DVs for the same data file**: **merge (union) their
position bitmaps** by reading both Puffin blobs and combining them, analogous
to the merge already performed on the write path, and log a WARNING about the
spec violation. This preserves **all** deletes.
3. **Never silently pick one DV**, which would drop the other's deletes and
resurrect deleted rows - a correctness violation worse than a scan failure.
**Proposed fix (I can contribute):** In `DeleteFileIndex.Builder.build()`,
collect DVs per referenced data file, and in a post-pass deduplicate
byte-identical DVs and union the position bitmaps of any genuinely different
DVs (reusing the existing DV merge utility), logging a WARNING; index only the
resulting merged DV. `FileIO` is available on the Builder. This is a read-side
recovery/self-heal that complements the write-side fix in #15966. I would
include a reproducer test.
### Willingness to contribute
- [X] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
