[
https://issues.apache.org/jira/browse/IMPALA-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927844#comment-17927844
]
ASF subversion and git services commented on IMPALA-13201:
----------------------------------------------------------
Commit aac67a077eb80e23f433eef5fcbf9edda75deb75 in impala's branch
refs/heads/master from jasonmfehr
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=aac67a077 ]
IMPALA-13201: System Table Queries Execute When Admission Queues are Full
Queries that run only against in-memory system tables are currently
subject to the same admission control process as all other queries.
Since these queries do not use any resources on executors, admission
control does not need to consider the state of executors when
deciding to admit these queries.
This change adds a boolean configuration option 'onlyCoordinators'
to the fair-scheduler.xml file for specifying a request pool only
applies to the coordinators. When a query is submitted to a
coordinator only request pool, then no executors are required to be
running. Instead, all fragment instances are executed exclusively on
the coordinators.
A new member was added to the ClusterMembershipMgr::Snapshot struct
to hold the ExecutorGroup of all coordinators. This object is kept up
to date by processing statestore messages and is used when executing
queries that either require the coordinators (such as queries against
sys.impala_query_live) or that use an only coordinators request pool.
Testing was accomplished by:
1. Adding cluster membership manager ctests to assert cluster
membership manager correctly builds the list of non-quiescing
coordinators.
2. RequestPoolService JUnit tests to assert the new optional
<onlyCoords> config in the fair scheduler xml file is correctly
parsed.
3. ExecutorGroup ctests modified to assert the new function.
4. Custom cluster admission controller tests to assert queries with a
coordinator only request pool only run on the active coordinators.
Change-Id: I5e0e64db92bdbf80f8b5bd85d001ffe4c8c9ffda
Reviewed-on: http://gerrit.cloudera.org:8080/22249
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Queries Against System Tables Must Not Be Subject to Admission Control
> ----------------------------------------------------------------------
>
> Key: IMPALA-13201
> URL: https://issues.apache.org/jira/browse/IMPALA-13201
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Jason Fehr
> Assignee: Jason Fehr
> Priority: Critical
> Labels: workload-management
>
> Impala system tables are in-memory coordinator-only virtual tables without
> any backing of data on disk. Currently, the only system table is
> sys.impala_query_live. Since these tables only exist in coordinator memory,
> queries that only select from these tables do not need to by subject to
> admission control.
> Modify Impala so that queries that only reference system tables are not
> subject to admission control and can run if there is enough available memory
> on the coordinator.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
