JingsongLi opened a new pull request, #11036:
URL: https://github.com/apache/paimon/pull/11036

   ### Purpose
   
   DV conflict detection for bucket-unaware tables currently materializes every 
index-manifest entry before checking a commit. Limit that lookup to the exact 
(partition, bucket) pairs touched by its data-file and index changes, using the 
existing binary-row filtering in `scanBucketEntries` before constructing 
unrelated DV range maps.
   
   Collect scopes from both ADD and DELETE entries, including DV-only updates 
and removal-only changes. Each retry reads the latest snapshot, preserving the 
existing file-existence, DV-version, and Data Evolution row-ID checks. The 
index manifest is still traversed as a single file.
   
   ### Tests
   
   - 247 tests passed on JDK 8, with the normal Maven checkstyle, Spotless and 
enforcer checks enabled.
   - Added 16 latch-controlled concurrent publication/retry cases across 
ordinary append and Data Evolution tables: disjoint partitions/buckets, 
first-DV creation, stale replacements, shared-DV inputs, and 
file-deletion/DV-update races.
   - Added focused checks for the union of data/index scopes, data-file-only 
and DV-only changes, DELETE-only index changes, and empty/missing-manifest 
cases.
   
   ```sh
   mvn -o -B -pl paimon-core -am -DwildcardSuites=none -DfailIfNoTests=false \
     
'-Dtest=ConflictDetectionTest,DeletionVectorConflictDetectionTest,IndexFileHandlerTest,FileStoreCommitTest,DataEvolutionDeletionVectorTest,RowIdColumnConflictCheckerTest,RowIdRangeConflictCheckerTest'
 test
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to