JingsongLi opened a new pull request, #11036:
URL: https://github.com/apache/paimon/pull/11036
### Purpose
DV conflict detection for bucket-unaware tables currently materializes every
index-manifest entry before checking a commit. Limit that lookup to the exact
(partition, bucket) pairs touched by its data-file and index changes, using the
existing binary-row filtering in `scanBucketEntries` before constructing
unrelated DV range maps.
Collect scopes from both ADD and DELETE entries, including DV-only updates
and removal-only changes. Each retry reads the latest snapshot, preserving the
existing file-existence, DV-version, and Data Evolution row-ID checks. The
index manifest is still traversed as a single file.
### Tests
- 247 tests passed on JDK 8, with the normal Maven checkstyle, Spotless and
enforcer checks enabled.
- Added 16 latch-controlled concurrent publication/retry cases across
ordinary append and Data Evolution tables: disjoint partitions/buckets,
first-DV creation, stale replacements, shared-DV inputs, and
file-deletion/DV-update races.
- Added focused checks for the union of data/index scopes, data-file-only
and DV-only changes, DELETE-only index changes, and empty/missing-manifest
cases.
```sh
mvn -o -B -pl paimon-core -am -DwildcardSuites=none -DfailIfNoTests=false \
'-Dtest=ConflictDetectionTest,DeletionVectorConflictDetectionTest,IndexFileHandlerTest,FileStoreCommitTest,DataEvolutionDeletionVectorTest,RowIdColumnConflictCheckerTest,RowIdRangeConflictCheckerTest'
test
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]