Sahil Takiar created IMPALA-8803:
------------------------------------
Summary: Coordinator should release admitted memory per-backend
rather than per-query
Key: IMPALA-8803
URL: https://issues.apache.org/jira/browse/IMPALA-8803
Project: IMPALA
Issue Type: Sub-task
Reporter: Sahil Takiar
Assignee: Sahil Takiar
When {{SPOOL_QUERY_RESULTS}} is true, the coordinator backend may be long
lived, even though all other backends for the query have completed. Currently,
the Coordinator only releases admitted memory when the entire query has
completed (include the coordinator fragment) -
https://github.com/apache/impala/blob/72c9370856d7436885adbee3e8da7e7d9336df15/be/src/runtime/coordinator.cc#L562
In order to more aggressively return admitted memory, the coordinator should
release memory when each backend for a query completes, rather than waiting for
the entire query to complete.
Releasing memory per backend should be batched because releasing admitted
memory in the admission controller requires obtaining a global lock and
refreshing the internal stats of the admission controller. Batching will help
mitigate any additional overhead from releasing admitted memory per backend.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
