[ https://issues.apache.org/jira/browse/HIVE-22966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17052118#comment-17052118 ]
Panagiotis Garefalakis commented on HIVE-22966: ----------------------------------------------- Just to be clear here: waiting time has nothing to do with starvation as all tasks eventually complete within a vertex no matter how long they wait. Across vertices, a vertex can not really affect the completion of another as their priorities are different (so no starvation there either). In this patch we are using wait time as a proxy of a long-waiting task which is a really weak assumption – it might work for a query but it might not work for another based on the task runtime distributions – a better approach would be to use input/split size stats to make more involved decisions when prioritising tasks which requires significantly more work. Other than that, with this patch we are assigning resources to tasks in a more Fair manner so it does not hurt – but to avoid tasks with long-tails executed at the end there is more work involved. +1 from me well > LLAP: Consider including waitTime for comparing attempts in same vertex > ----------------------------------------------------------------------- > > Key: HIVE-22966 > URL: https://issues.apache.org/jira/browse/HIVE-22966 > Project: Hive > Issue Type: Improvement > Components: llap > Reporter: Rajesh Balamohan > Assignee: Rajesh Balamohan > Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22966.3.patch, HIVE-22966.4.patch > > > When attempts are compared within same vertex, it should pick up the attempt > with longest wait time to avoid starvation. -- This message was sent by Atlassian Jira (v8.3.4#803005)