Andrew Sherman created IMPALA-10244:
---------------------------------------
Summary: Make non-scalable failures to dequeue observable
Key: IMPALA-10244
URL: https://issues.apache.org/jira/browse/IMPALA-10244
Project: IMPALA
Issue Type: Improvement
Reporter: Andrew Sherman
Assignee: Andrew Sherman
One of the important ways to observe Impala throughput is by looking at when
queries are queued. This can be an indication that more resources should be
added to the cluster by adding more executor groups. This is only a good
strategy if adding more resources will help with the current workload. In some
situations the head of the query queue cannot be executed because of resource
constraints on the coordinator.
{code}
Could not dequeue query id=89440761baaaa4c2:261de88600000000 reason: Not enough
memory available on host coordinator-0:22050. Needed 104.02 MB but only 103.16
MB out of 5.18 GB was available.
{code}
or
{code}
Could not dequeue query id=6d48e94edd686f8f:a114d87300000000 reason: Not enough
admission control slots available on host coordinator-0:22050. Needed 1 slots
but 30/30 are already in use.
{code}
In these cases the coordinator is the bottleneck so adding more executor groups
will not help. This jira is to make these cases observable by adding a new
counter which is incremented when a dequeue fails because of resource
constraints that would not be resolved by adding more executor groups.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
