[
https://issues.apache.org/jira/browse/IMPALA-11786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655458#comment-17655458
]
ASF subversion and git services commented on IMPALA-11786:
----------------------------------------------------------
Commit 839a25c89b5620e8bdc37383c8512d912fcb5d0f in impala's branch
refs/heads/master from Yida Wu
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=839a25c89 ]
IMPALA-11786: Preserve memory for codegen cache
IMPALA-11470 adds support for codegen cache, however the admission
controller is not aware of the memory usage of the codegen cache,
while the codegen cache is actually using the memory quota from
the query memory. It could result in query failures when running
heavy workloads and admission controller has fully admitted queries.
This patch subtracts the codegen cache capacity from the admission
memory limit during initialization, therefore preserving the memory
consumption of codegen cache from the beginning, and treating it as
a separate memory independent to the query memory reservation.
Also reduces the max codegen cache memory from 20 percent to 10
percent, and changes some failed testcases due to the reduction of
the admit memory limit.
Tests:
Passed exhaustive tests.
Change-Id: Iebdc04ba1b91578d74684209a11c815225b8505a
Reviewed-on: http://gerrit.cloudera.org:8080/19377
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Take codegen cache memory into account for admission controller
> ---------------------------------------------------------------
>
> Key: IMPALA-11786
> URL: https://issues.apache.org/jira/browse/IMPALA-11786
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Yida Wu
> Assignee: Yida Wu
> Priority: Major
>
> Currently admission controller is not aware of the memory usage of the
> codegen cache, and the codegen cache is long-running and damon-level, this
> may lead to more allocation failures during heavy workload scenario when the
> available memory is running out. One solution is to subtract the codegen
> cache capacity from the available memory that admission controller can use
> when initialization, so that the admission controller would take account of
> the usage of codegen cache since the beginning.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
