kfaraz commented on PR #14616:
URL: https://github.com/apache/druid/pull/14616#issuecomment-1648483808
> When would this occur?
Only if the client (Coordinator) sends a LOAD request and then sends a DROP
request for the same segment while the load is still in progress. There is
already a list of `segmentsToDrop` but that might be ineffective in some cases.
A typical case might look like this:
- Coordinator asks historical to load segment
- Historical queues LOAD task on executor
- Coordinator asks historical to drop segment. The Historical then
- unannounces the segment immediately
- adds an entry to `segmentsToDrop`
- schedules DROP task on executor for 30s later
- LOAD task starts and removes this segment from `segmentsToDrop` and
proceeds with downloading the segment
- LOAD task finishes and announces the segment but is not able to update
success status since the latest request is a DROP.
- DROP task does not do anything since the segment is not present in
`segmentsToDrop` anymore. Thus the result holder remains stuck in pending state.
- All subsequent DROP requests will be answered with `pending` (at least
until the cache entry expires) without ever actually trying to drop the segment.
- Only a subsequent LOAD request will clear this state
I think a simple fix might be to start the LOAD task only if the latest
request is a load.
---
I think a DROP followed by a LOAD should be okay. Since the LOAD task clears
the entry from `segmentsToDrop`, the DROP would be ignored.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
