[jira] [Updated] (HIVE-24914) Improve LLAP scheduling by only traversing hosts with capacity
[ https://issues.apache.org/jira/browse/HIVE-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-24914: -- Fix Version/s: 4.0.0 > Improve LLAP scheduling by only traversing hosts with capacity > -- > > Key: HIVE-24914 > URL: https://issues.apache.org/jira/browse/HIVE-24914 > Project: Hive > Issue Type: Sub-task >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > *schedulePendingTasks* on the LlapTaskScheduler currently goes through all > the pending tasks and tries to allocate them based on their Priority -- if a > priority can not be scheduled completely, we bail out as lower priorities > would not be able to get allocations either. > An optimization here could be to only walk through the nodes with capacity > (if any) ,and not all available hosts, for scheduling these tasks based on > their priority and locality preferences. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24914) Improve LLAP scheduling by only traversing hosts with capacity
[ https://issues.apache.org/jira/browse/HIVE-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-24914: -- Component/s: llap > Improve LLAP scheduling by only traversing hosts with capacity > -- > > Key: HIVE-24914 > URL: https://issues.apache.org/jira/browse/HIVE-24914 > Project: Hive > Issue Type: Sub-task > Components: llap >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > *schedulePendingTasks* on the LlapTaskScheduler currently goes through all > the pending tasks and tries to allocate them based on their Priority -- if a > priority can not be scheduled completely, we bail out as lower priorities > would not be able to get allocations either. > An optimization here could be to only walk through the nodes with capacity > (if any) ,and not all available hosts, for scheduling these tasks based on > their priority and locality preferences. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24914) Improve LLAP scheduling by only traversing hosts with capacity
[ https://issues.apache.org/jira/browse/HIVE-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-24914: -- Summary: Improve LLAP scheduling by only traversing hosts with capacity (was: Improve scheduling by only traversing hosts with capacity) > Improve LLAP scheduling by only traversing hosts with capacity > -- > > Key: HIVE-24914 > URL: https://issues.apache.org/jira/browse/HIVE-24914 > Project: Hive > Issue Type: Sub-task >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > *schedulePendingTasks* on the LlapTaskScheduler currently goes through all > the pending tasks and tries to allocate them based on their Priority -- if a > priority can not be scheduled completely, we bail out as lower priorities > would not be able to get allocations either. > An optimization here could be to only walk through the nodes with capacity > (if any) ,and not all available hosts, for scheduling these tasks based on > their priority and locality preferences. -- This message was sent by Atlassian Jira (v8.3.4#803005)
