[
https://issues.apache.org/jira/browse/IMPALA-10193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fifteen updated IMPALA-10193:
-----------------------------
Description:
The mini-cluster contains 3 virtual nodes, and all of them runs in a single
'Machine'. By quoting, it implies the machine can be a docker container. If the
container is started with `-priviledged` and the actual memory is limited by
CGROUPS, then the total memory in `top` and the actual available memory can be
different!
For example, in the container below, `top` tells us the total memory is 128GB,
while the total memory set in CGROUPS is actually 32GB. If the acutal mem usage
exceeds 32GB, process (such as impalad, hivemaster2 etc.) get killed.
!image-2020-09-28-17-18-15-358.png!
So we may need a way to limit the whole mini-cluster's memory usage.
was:
The mini-cluster contains 3 virtual nodes, and all of them runs in a single
'Machine'. By quoting, it implies the machine can be a docker container. If the
container is started with `-priviledged` and the actual memory is limited by
CGROUPS, then the total memory in `top` and the actual available memory can be
different!
For example, in the container below, `top` tells us the total memory is 128GB,
while the total memory set in CGROUPS is actually 32GB. If the acutal mem usage
exceeds 32GB, process (such as impalad, hivemaster2 etc.) get killed.
!image-2020-09-28-17-08-02-604.png!
So we may need a way to limit the whole mini-cluster's memory usage.
> Limit the memory usage of the whole mini-cluster
> ------------------------------------------------
>
> Key: IMPALA-10193
> URL: https://issues.apache.org/jira/browse/IMPALA-10193
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.4.0
> Reporter: Fifteen
> Priority: Minor
> Attachments: image-2020-09-28-17-18-15-358.png
>
>
> The mini-cluster contains 3 virtual nodes, and all of them runs in a single
> 'Machine'. By quoting, it implies the machine can be a docker container. If
> the container is started with `-priviledged` and the actual memory is limited
> by CGROUPS, then the total memory in `top` and the actual available memory
> can be different!
>
> For example, in the container below, `top` tells us the total memory is
> 128GB, while the total memory set in CGROUPS is actually 32GB. If the acutal
> mem usage exceeds 32GB, process (such as impalad, hivemaster2 etc.) get
> killed.
> !image-2020-09-28-17-18-15-358.png!
>
> So we may need a way to limit the whole mini-cluster's memory usage.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
