Zhiyuan Yang created TEZ-3647:
---------------------------------
Summary: Lower default Xmx for small AM container
Key: TEZ-3647
URL: https://issues.apache.org/jira/browse/TEZ-3647
Project: Apache Tez
Issue Type: Bug
Reporter: Zhiyuan Yang
Assignee: Zhiyuan Yang
The default Xmx is 80% of container size. For a 2G container, non-heap memory
can cost more than 20%. Total off-heap memory is around 450MB. In case of
nearly full heap memory usage, we will be out of physical memory and killed by
YARN.
Here is native memory summary for a AM running tpcds query 13.
{code}
Native Memory Tracking:
Total: reserved=3424708KB, committed=2155712KB
- Java Heap (reserved=1677312KB, committed=1677312KB)
(mmap: reserved=1677312KB, committed=1677312KB)
- Class (reserved=1096701KB, committed=54061KB)
(classes #8465)
(malloc=1021KB #11433)
(mmap: reserved=1095680KB, committed=53040KB)
- Thread (reserved=195103KB, committed=195103KB)
(thread #191)
(stack: reserved=193948KB, committed=193948KB)
(malloc=611KB #986)
(arena=543KB #382)
- Code (reserved=254302KB, committed=31222KB)
(malloc=4702KB #7076)
(mmap: reserved=249600KB, committed=26520KB)
- GC (reserved=148963KB, committed=148963KB)
(malloc=53951KB #17562)
(mmap: reserved=95012KB, committed=95012KB)
- Compiler (reserved=306KB, committed=306KB)
(malloc=176KB #311)
(arena=131KB #3)
- Internal (reserved=17872KB, committed=17872KB)
(malloc=17840KB #15275)
(mmap: reserved=32KB, committed=32KB)
- Symbol (reserved=12659KB, committed=12659KB)
(malloc=10829KB #106221)
(arena=1830KB #1)
- Native Memory Tracking (reserved=2735KB, committed=2735KB)
(malloc=192KB #2968)
(tracking overhead=2543KB)
- Arena Chunk (reserved=5034KB, committed=5034KB)
(malloc=5034KB)
- Tracing (reserved=10444KB, committed=10444KB)
(malloc=10444KB #274)
- Unknown (reserved=3276KB, committed=0KB)
(mmap: reserved=3276KB, committed=0KB)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
