FrankChen021 commented on code in PR #20805:
URL: https://github.com/apache/druid/pull/20805#discussion_r4429602491
##########
multi-stage-query/src/main/java/org/apache/druid/msq/querykit/ReadableInputQueue.java:
##########
@@ -331,47 +352,123 @@ private ListenableFuture<NumberedReadableInput>
nextChannelInput()
@Nullable
private ListenableFuture<NumberedReadableInput> nextSegmentInput()
{
- // Pick a loadahead future, preferring ones that are already loaded.
- ListenableFuture<NumberedReadableInput> selectedLoadaheadFuture = null;
synchronized (this) {
- for (ListenableFuture<NumberedReadableInput> f : pendingNextInputs) {
- if (selectedLoadaheadFuture == null || f.isDone()) {
- selectedLoadaheadFuture = f;
- if (f.isDone()) {
- break;
- }
- }
- }
-
- if (selectedLoadaheadFuture != null) {
- pendingNextInputs.remove(selectedLoadaheadFuture);
+ while (!pendingNextInputs.isEmpty()) {
+ final ListenableFuture<NumberedReadableInput> selectedFuture =
selectPendingInput();
+ final LoadableSegment selectedSegment =
pendingNextInputs.remove(selectedFuture);
Review Comment:
[P2] Release pruned load-ahead futures
**Finding:** When a pending future's segment becomes prunable, the future is
removed from pendingNextInputs without being canceled or drained. A completed
future leaves its SegmentReferenceHolder in loadedSegments, while an in-flight
future remains owned by loadingSegments; both are retained until
ReadableInputQueue.close(), potentially accumulating large segment references
during the stage.
**Suggestion:** Add a discard path that cancels or closes the associated
acquisition and drains any completed SegmentReferenceHolder before continuing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
