leventov opened a new issue #8356: Query execution interleaving URL: https://github.com/apache/incubator-druid/issues/8356 The idea of query execution interleaving first popped up in Druid discussions in the context of parallel result combining in Brokers in the discussion between me and @jihoonson, see https://github.com/apache/incubator-druid/pull/6629#discussion_r241089247. In general, this is not a new idea to reduce head-of-line blocking effects. See [The Tail at Scale](https://blog.acolyer.org/2015/01/15/the-tail-at-scale/), Dean and Barroso, 2013. In Druid, interleaving may be implemented inside `PrioritizedExecutorService`. On Historicals, the element of processing is naturally a segment, when a Broker sends a request to a Historical to process many segments for a large query. On Brokers, interleaving may be implemented as discussed in https://github.com/apache/incubator-druid/pull/6629#discussion_r241089247. Interleaving will play especially well together with online aggregation (https://github.com/apache/incubator-druid/issues/7087) but would be useful even if implemented on its own independently of online aggregation.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
