[
https://issues.apache.org/jira/browse/DRILL-855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015637#comment-14015637
]
Aditya Kishore commented on DRILL-855:
--------------------------------------
Merged as
https://github.com/apache/incubator-drill/commit/d929faace270302c5e00e272775be98d5a8f83e1
> Improve work assignment parallelization
> ---------------------------------------
>
> Key: DRILL-855
> URL: https://issues.apache.org/jira/browse/DRILL-855
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.0.0-rc1
> Reporter: Aditya Kishore
> Assignee: Aditya Kishore
> Fix For: 1.0.0-rc1
>
> Attachments: DRILL-855-Improve-work-assignment-parallelization.patch
>
>
> Currently if only a small number of endpoints have data affinity, all the
> work units get assigned to this small subset leaving other endpoints in the
> cluster underutilized.
> It will lead to poor query performance since the level of parallelization is
> reduced below what is otherwise possible.
> This JIRA intend to modify the assignment to do a slightly more equitable
> distribution of work fragments among all available nodes while slightly
> favoring the nodes with data affinity.
> This \*favor\* is controlled by the setting
> {{"drill.exec.work.affinity.factor"}} (defaults to 1.2f).
> As an example, let's say that there are 2 endpoints of which only one is with
> affinity and 10 work fragments. In the current scheme of things, all of these
> work fragments will get assigned to the single node with affinity. With the
> modified scheme and with default affinity factor of 1.2, it will get 20% more
> assignment that it would get under equal distribution (i.e. 6:4 instead of
> 5:5).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
