gianm commented on code in PR #18765:
URL: https://github.com/apache/druid/pull/18765#discussion_r2587654421
##########
multi-stage-query/src/main/java/org/apache/druid/msq/exec/std/StandardShuffleOperations.java:
##########
@@ -382,8 +383,18 @@ public <T2> FrameProcessor<T2> decorate(FrameProcessor<T2>
processor)
executionContext.makeIntermediateOutputChannelFactory(
StringUtils.format("hash-parts-super-sort-%06d",
channel.getPartitionNumber())),
executionContext.frameContext().frameWriterSpec().getRowBasedFrameType(),
- 1,
- 2,
+
+ // Use full parallelism, since at the time this sorter
runs, it is the only sorter running.
+ //
+ // Typically, nothing else is running at all. Whenever
there is more than one output partition,
+ // the step prior to this localSort (typically
hashPartition) should use buffered output channels
+ // and therefore would have exited.
+ //
+ // In the case where there is one output partition, the
step prior to this localSort may run
+ // concurrently with the sorter, but in that case it
will only have one output channel so won't
+ // have many frames buffered.
+ memoryParameters.getSuperSorterConcurrentProcessors(),
+ memoryParameters.getSuperSorterMaxChannelsPerMerger(),
Review Comment:
Yes, it'll always be at least 2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
