LakshSingla commented on code in PR #14117:
URL: https://github.com/apache/druid/pull/14117#discussion_r1179964567
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerMemoryParameters.java:
##########
@@ -309,8 +316,12 @@ public static WorkerMemoryParameters createInstance(
)
);
- // Apportion max frames to all processors equally, then subtract one to
account for an output frame.
- final int superSorterMaxChannelsPerProcessor = maxNumFramesForSuperSorter
/ superSorterMaxActiveProcessors - 1;
+ final int isSmallWorker = usableMemoryInJvm <
SMALL_WORKER_CAPACITY_THRESHOLD_BYTES ? 1 : 0;
+ // Apportion max frames to all processors equally, then subtract one to
account for an output frame and one to account
+ // for the durable storage's output frame in the supersorter
Review Comment:
Updated with tests and the explanation in the comments. Regarding 2nd point,
I have mentioned that we need to be conservative with the heap space in case of
smaller heaps hence the subtraction is going on there and not in the case of
larger heaps where we can have more parallelization without the risk of OOMing
due to these temporary allocator factories.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
