[
https://issues.apache.org/jira/browse/IMPALA-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872584#comment-16872584
]
Tim Armstrong commented on IMPALA-2968:
---------------------------------------
[~bikramjeet.vig] and I had discussed a different variant of this in the past,
where it was time-based. I.e. it would by default operate in a greedy mode
where the first query that could run got admitted, but then flip to a "fair"
mode when the head of the queue had been blocked for more than some amount of
time. The time would have to have some relationship to the queue timeout, since
part of the goal would be to prevent queries timing out in the queue.
I think the argument for doing something like this is either:
* Compared to FIFO, it's better in the common case for small queries
* Compared to a purely greedy approach, it avoids starvation in the worst cases.
We'd also talked about something more sophisticated where you determine if two
queries are actually contending on the same resource, and let queries skip
ahead if they're not. This is partially subsumed by the executor group concept,
and I think otherwise is not that useful - it does nothing for queries that are
actually blocked on the contended resources, which seems like the common case.
Unless you can reschedule the query to avoid the contended resources.
> Improve admission control dequeuing policy
> ------------------------------------------
>
> Key: IMPALA-2968
> URL: https://issues.apache.org/jira/browse/IMPALA-2968
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.3.0
> Reporter: Matthew Jacobs
> Priority: Minor
> Labels: admission-control, resource-management
>
> The current behavior only attempts to admit the head of the queue but the
> head might need resources which are contended (e.g. on a hot node) while a
> queued request behind the head might be admitted. We should consider a policy
> which would not block the entire queue but yet is unlikely to starve the head
> if other requests are continuously admitted from behind it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
