jihoonson commented on a change in pull request #6629: Add support parallel
combine in brokers
URL: https://github.com/apache/incubator-druid/pull/6629#discussion_r240883297
##########
File path:
server/src/main/java/org/apache/druid/client/CachingClusteredClient.java
##########
@@ -285,12 +301,72 @@ public CachingClusteredClient(
List<Sequence<T>> sequencesByInterval = new
ArrayList<>(alreadyCachedResults.size() + segmentsByServer.size());
addSequencesFromCache(sequencesByInterval, alreadyCachedResults);
addSequencesFromServer(sequencesByInterval, segmentsByServer);
- return Sequences
- .simple(sequencesByInterval)
- .flatMerge(seq -> seq, query.getResultOrdering());
+ return merge(sequencesByInterval);
});
}
+ private Sequence<T> merge(List<Sequence<T>> sequencesByInterval)
+ {
+ final int numParallelCombineThreads =
QueryContexts.getNumBrokerParallelCombineThreads(query);
Review comment:
Would you elaborate more details about what's the benefit of this?
My intention was, as noted
[here](https://github.com/apache/incubator-druid/pull/6629#discussion_r240882331),
that users should choose the number of threads for parallel combine carefully
and Druid guarantees that users can always expect the benefit of using parallel
combine if the query runs. If users just want a sort of best effort, they can
set `numParallelCombineThreads` to `NUM_CURRENT_AVAILABLE_THREADS`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
