Michael Smith created IMPALA-12641:
--------------------------------------
Summary: Impala underestimates LLVM memory use
Key: IMPALA-12641
URL: https://issues.apache.org/jira/browse/IMPALA-12641
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 4.3.0
Reporter: Michael Smith
As I attempted to address in [https://gerrit.cloudera.org/c/20697/] - and
reverted in [https://gerrit.cloudera.org/c/20796] - Impala currently
under-estimates memory used by LLVM's Memory Manager. It allocates memory
aligned to pages (usually 4KB) but currently only reports the actual memory
that was requested from those pages. Trying to account for that starts to
introduce significant unplanned memory and causes memory reservation failures
in tests (see IMPALA-12635).
We should fix Impala's tracking of LLVM memory use along the lines of the patch
20697. It's not clear to me that the planner accounts for memory use by code
generation; it looks like it needs to be updated to better plan for it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
