Joe McDonnell created IMPALA-15729:
--------------------------------------
Summary: Migrate non-impalad daemon containers from gperftools to
Google TCMalloc
Key: IMPALA-15729
URL: https://issues.apache.org/jira/browse/IMPALA-15729
Project: IMPALA
Issue Type: Task
Components: Backend
Affects Versions: Impala 5.1.0
Reporter: Joe McDonnell
In IMPALA-15602, we are switching from using gperftools TCMalloc to Google
TCMalloc. The one exception is the non-impalad daemon Docker containers. They
rely on operating within strict memory bounds, and the aggressive decommit mode
from gperftools suits that well. However, it would be useful to consolidate on
Google TCMalloc.
One option is to investigate Google TCMalloc's SetMemoryLimit():
{noformat}
// Make a best effort attempt to prevent more than limit bytes of memory
// from being allocated by the system. In particular, if satisfying a given
// malloc call would require passing this limit, release as much memory to
// the OS as needed to stay under it if possible.
//
// If limit_kind == kHard, crash if returning memory is unable to get below
// the limit.
[[nodiscard]] static size_t GetMemoryLimit(LimitKind limit_kind);
static void SetMemoryLimit(size_t limit, LimitKind limit_kind);{noformat}
Another approach is to more carefully track memory in non-impalad daemons and
call ReleaseMemoryToSystem() as needed to stay under the limit.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
