[jira] [Commented] (IMPALA-5216) Admission control queuing should be asynchronous
[
https://issues.apache.org/jira/browse/IMPALA-5216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16609850#comment-16609850
]
ASF subversion and git services commented on IMPALA-5216:
-
Commit ab6bd74ff73f52a3bd0247027bd82d977189ded6 in impala's branch
refs/heads/master from [~bikram.sngh91]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=ab6bd74 ]
IMPALA-7516: Fix query location accounting
As a side-effect of IMPALA-5216, queries that were scheduled to be
executed but eventually got rejected or canceled before starting
execution (initializing the coordinator object) got added to the
'query_locations' map but never got removed. As a result, the queries
debug page would sometimes show queries running under the
"Query Locations" section even if there weren't. This patch fixes that
behavior.
Testing:
Added a custom cluster test.
Change-Id: I66c8e59299747df57c9f39db1cb1f46ff6bbab7e
Reviewed-on: http://gerrit.cloudera.org:8080/11372
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Admission control queuing should be asynchronous
>
>
> Key: IMPALA-5216
> URL: https://issues.apache.org/jira/browse/IMPALA-5216
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
>Reporter: Dan Hecht
>Assignee: Bikramjeet Vig
>Priority: Major
> Labels: admission-control, resource-management
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> Currently, admission control queuing occurs synchronously w.r.t. the
> {{ExecuteStatement}} client RPC. That is, a query handle is not returned
> until the query is admitted.
> Instead, the queuing should occur on the asynchronous path. That way, the
> client gets a query handle back immediately and can e.g. cancel a query that
> is in the admission control queue.
> We'll also need a way to better expose the progress of a query handle
> (related to IMPALA-124). E.g. that the query is queued for admission and what
> resource(s) it's waiting on.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-5216) Admission control queuing should be asynchronous
[
https://issues.apache.org/jira/browse/IMPALA-5216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511338#comment-16511338
]
ASF subversion and git services commented on IMPALA-5216:
-
Commit 2de9db8fc64dc1054ac3ca41ab0f0047db670da6 in impala's branch
refs/heads/master from [~bikram.sngh91]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=2de9db8 ]
IMPALA-5216: Make admission control queuing async
Implement asynchronous admission control queuing. This is achieved by
running the admission control code-path in a separate thread. Major
changes include: propagating cancellation to the admission control
thread and dequeuing thread, and adding a new Query Operation State
called "PENDING" that represents the state between completion of
planning and starting of query execution.
Testing:
- Added a deterministic end to end test and a session expiry test.
- Ran multiple stress tests successfully with a cancellation probability
of 60% and with different values for the following parameters:
max_requests, queue_wait_timeout_ms. Ensured that the impalad was in a
valid state afterwards (no orphan fragments or wrong metrics).
- Ran all exhaustive tests and ASAN core tests successfully.
- Ran data load successfully.
Change-Id: I989cf5b259afb8f5bc5c35590c94961c81ce88bf
Reviewed-on: http://gerrit.cloudera.org:8080/10060
Reviewed-by: Tim Armstrong
Tested-by: Impala Public Jenkins
> Admission control queuing should be asynchronous
>
>
> Key: IMPALA-5216
> URL: https://issues.apache.org/jira/browse/IMPALA-5216
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
>Reporter: Dan Hecht
>Assignee: Bikramjeet Vig
>Priority: Major
> Labels: admission-control, resource-management
>
> Currently, admission control queuing occurs synchronously w.r.t. the
> {{ExecuteStatement}} client RPC. That is, a query handle is not returned
> until the query is admitted.
> Instead, the queuing should occur on the asynchronous path. That way, the
> client gets a query handle back immediately and can e.g. cancel a query that
> is in the admission control queue.
> We'll also need a way to better expose the progress of a query handle
> (related to IMPALA-124). E.g. that the query is queued for admission and what
> resource(s) it's waiting on.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
