FrankChen021 commented on code in PR #20182:
URL: https://github.com/apache/druid/pull/20182#discussion_r3880818951
##########
processing/src/main/java/org/apache/druid/segment/file/PartialSegmentFileMapperV10.java:
##########
@@ -988,6 +1054,12 @@ public void evictContainer(int containerIndex)
checkClosed();
containerLocks[containerIndex].lock();
try {
+ if (containerFetchesInFlight[containerIndex] > 0) {
+ // A fetch is writing into this container. Deleting the file now would
leave it writing to a null File, so
+ // hand the eviction to whichever fetch finishes last rather than
blocking here.
+ containerEvictionPending[containerIndex] = true;
Review Comment:
[P2] Eviction can clear a newer fetch's residency
The new in-flight counter only protects fetches that have already entered
beginContainerFetch. Once this branch releases containerLocks, eviction clears
downloadedFiles and bitmap bits later in the method outside that lock. A new
fetch can enter during that gap, initialize the replacement container, write
its bytes, and mark the files downloaded; the old eviction then removes those
entries and bits, so the fetch returns with inconsistent residency
(isFullyDownloaded() becomes false and mapFile can fail). Keep the clear and
generation work under the same gate, or prevent new fetches until eviction
finishes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
