[
https://issues.apache.org/jira/browse/IMPALA-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-691:
------------------------------------
Assignee: Tim Armstrong
> Process mem limit does not account for the JVM's memory usage
> -------------------------------------------------------------
>
> Key: IMPALA-691
> URL: https://issues.apache.org/jira/browse/IMPALA-691
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 1.2.1, Impala 2.0, Impala 2.1, Impala 2.2, Impala
> 2.3.0
> Reporter: Skye Wanderman-Milne
> Assignee: Tim Armstrong
> Priority: Major
> Labels: incompatibility, resource-management
>
> The JVM doesn't appear to use malloc, so it's memory usage is not reported by
> tcmalloc and we do not count it in the process mem limit. I verified this by
> adding a large allocation in the FE, and noting that the total memory usage
> (virtual or resident) reported in /memz is not affected, but the virtual and
> resident memory usage reported by top is.
> This is problematic especially because Impala caches table metadata in the FE
> (JVM) which can become quite big (few GBs) in extreme cases.
> *Workaround*
> As a workaround, we recommend reducing the process memory limit by 1-2GB to
> "reserve" memory for the JVM. How much memory you should reserve typically
> depends on the size of your catalog ( number of
> tables/partitions/columns/blocks etc.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
