[
https://issues.apache.org/jira/browse/DRILL-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15838336#comment-15838336
]
Jinfeng Ni commented on DRILL-5223:
-----------------------------------
The following link has some preliminary analysis of the impact of workload
balance at node level on tpc-h query performance.
1. https://gist.github.com/jinfengni/5ff4c5f25dca5f12805bac4853778789
> Drill should ensure balanced workload assignment at node level in order to
> get better query performance
> -------------------------------------------------------------------------------------------------------
>
> Key: DRILL-5223
> URL: https://issues.apache.org/jira/browse/DRILL-5223
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Reporter: Jinfeng Ni
>
> Drill's work assignment logic currently aims to achieve workload balance
> across different minor fragment (or slices) and honor data affinity in order
> to get as much local read as possible.
> However, when the # of work units could not be evenly divided by # of minor
> fragments, the remaining work units would tender to go to the first subset of
> drill endpoints. This means the drill endpoints assigned with the remaining
> work units could have larger workload than the rest of them. When
> MuxExchange is enabled (by default), all the minor fragments on the same node
> have to send data to a single Muxer per node, and unbalanced workload
> assignment at node level could impact query elapse time. which is
> essentially decided by the slowest drill endpoint.
> Some prototype experimental run shows that with more balanced workload
> assignment, Drill shows quite significant improvement for most of TPC-H
> queries.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
