lilei1128 opened a new pull request, #16980:
URL: https://github.com/apache/iceberg/pull/16980
When multiple worker threads processed manifests in parallel via
Tasks.range(), expression-matched files were written directly to the
non-thread-safe deleteFiles set, causing data races and incorrect
deleted-file counts.
Close #16978
Fix by collecting expression-deleted files into a ConcurrentLinkedQueue
during parallel filtering, then merging into deleteFiles on the calling
thread after all workers complete. Also convert duplicateDeleteCount
from int to AtomicInteger to safely handle concurrent increments across
worker threads.
Add a regression test that appends 4 files in separate commits (one
manifest each) and deletes all via overwriteByRowFilter with a 4-thread
pool, asserting the snapshot summary reports exactly 4 deleted files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
