Yida Wu created IMPALA-14549:
--------------------------------
Summary: Admission control to use smarter queue scheduling to
mitigate large query blocking
Key: IMPALA-14549
URL: https://issues.apache.org/jira/browse/IMPALA-14549
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Yida Wu
Currently, Impala’s admission control processes queued requests in FIFO order.
When a large query at the head of the queue can't be admitted due to
insufficient resources, smaller queries behind it can be blocked for a long
time, even if they could fit in the available capacity.
One idea is to enhance the admission scheduler to be smarter by implementing
backfill scheduling. When the query at the head of the queue is blocked due to
insufficient resources, the scheduler should scan the queue and allow smaller,
runnable queries (that fit within currently available resources) to execute out
of order.
To prevent starvation of the large head query, this backfill process should be
constrained. For example, the scheduler could admit one small query, wait for
it to complete, and then reevaluate the queue. This process would repeat until
the large query has sufficient resources to be scheduled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
