[
https://issues.apache.org/jira/browse/YARN-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15812167#comment-15812167
]
Bibin A Chundatt edited comment on YARN-6017 at 1/9/17 4:36 PM:
----------------------------------------------------------------
[~chenrongwei]
This could be because of JDK bug. Found the following link
[JDK-8054841|http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8054841]
during reading.
As per the defect description {{ProcessBuilder leaks native memory}} .
{{Shell.java}} do use {{ProcessBuilder}} for all child process launch
{quote}
For the Oracle JDK, it does not appear to be present in 7u45, but does appear
to be present in 7u55.
{quote}
Looks like the issue is available in 7u65 . If possible could you try changing
JRE version.
[GITHUB|https://github.com/cprice404/jvm-processbuilder-leak/blob/master/README.md]
{quote}
For Oracle JDK, the leak does not appear to be present in 7u45, but does appear
to be present in 7u55. (I believe, though I have less data, that the leak is
not present in Oracle 7u51. The leak definitely appears to be present in Oracle
7u65 and 7u67 as well.) For OpenJDK, the leak does not appear to be present in
7u55, but does appear to be present in 7u65.
{quote}
was (Author: bibinchundatt):
[~chenrongwei]
This could be because of JDK bug. Found the following link
[JDK-8054841|http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8054841]
during reading.
As per the defect description {{ProcessBuilder leaks native memory}} .
{{Shell.java}} do use {{ProcessBuilder}} for all child process launch
{quote}
For the Oracle JDK, it does not appear to be present in 7u45, but does appear
to be present in 7u55.
{quote}
Looks like the issue is available in 7u65 . If possible could you try changing
JRE version.
> node manager physical memory leak
> ---------------------------------
>
> Key: YARN-6017
> URL: https://issues.apache.org/jira/browse/YARN-6017
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Affects Versions: 2.7.1
> Environment: OS:
> Linux guomai124041 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013
> x86_64 x86_64 x86_64 GNU/Linux
> jvm:
> java version "1.7.0_65"
> Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
> Reporter: chenrongwei
> Attachments: 31169_smaps.txt, 31169_smaps.txt
>
>
> In our produce environment, node manager's jvm memory has been set to
> '-Xmx2048m',but we notice that after a long time running the process' actual
> physical memory size had been reached to 12g (we got this value by top
> command as follow).
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 31169 data 20 0 13.2g 12g 6092 S 16.9 13.0 49183:13 java
> 31169: /usr/local/jdk/bin/java -Dproc_nodemanager -Xmx2048m
> -Dhadoop.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs
> -Dyarn.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs
> -Dhadoop.log.file=yarn-data-nodemanager.log
> -Dyarn.log.file=yarn-data-nodemanager.log -Dyarn.home.dir= -Dyarn.id.str=data
> -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA
> -Djava.library.path=/home/data/programs/apache-hadoop-2.7.1/lib/native
> -Dyarn.policy.file=hadoop-policy.xml -XX:PermSize=128M -XX:MaxPermSize=256M
> -XX:+UseC
> Address Kbytes Mode Offset Device Mapping
> 0000000000400000 4 r-x-- 0000000000000000 008:00001 java
> 0000000000600000 4 rw--- 0000000000000000 008:00001 java
> 0000000000601000 10094936 rw--- 0000000000000000 000:00000 [ anon ]
> 0000000770000000 2228224 rw--- 0000000000000000 000:00000 [ anon ]
> 00000007f8000000 131072 rw--- 0000000000000000 000:00000 [ anon ]
> 000000325ee00000 128 r-x-- 0000000000000000 008:00001 ld-2.12.so
> 000000325f01f000 4 r---- 000000000001f000 008:00001 ld-2.12.so
> 000000325f020000 4 rw--- 0000000000020000 008:00001 ld-2.12.so
> 000000325f021000 4 rw--- 0000000000000000 000:00000 [ anon ]
> 000000325f200000 1576 r-x-- 0000000000000000 008:00001 libc-2.12.so
> 000000325f38a000 2048 ----- 000000000018a000 008:00001 libc-2.12.so
> 000000325f58a000 16 r---- 000000000018a000 008:00001 libc-2.12.so
> 000000325f58e000 4 rw--- 000000000018e000 008:00001 libc-2.12.so
> 000000325f58f000 20 rw--- 0000000000000000 000:00000 [ anon ]
> 000000325f600000 92 r-x-- 0000000000000000 008:00001 libpthread-2.12.so
> 000000325f617000 2048 ----- 0000000000017000 008:00001 libpthread-2.12.so
> 000000325f817000 4 r---- 0000000000017000 008:00001 libpthread-2.12.so
> 000000325f818000 4 rw--- 0000000000018000 008:00001 libpthread-2.12.so
> 000000325f819000 16 rw--- 0000000000000000 000:00000 [ anon ]
> 000000325fa00000 8 r-x-- 0000000000000000 008:00001 libdl-2.12.so
> 000000325fa02000 2048 ----- 0000000000002000 008:00001 libdl-2.12.so
> 000000325fc02000 4 r---- 0000000000002000 008:00001 libdl-2.12.so
> 000000325fc03000 4 rw--- 0000000000003000 008:00001 libdl-2.12.so
> 000000325fe00000 28 r-x-- 0000000000000000 008:00001 librt-2.12.so
> 000000325fe07000 2044 ----- 0000000000007000 008:00001 librt-2.12.so
> 0000003260006000 4 r---- 0000000000006000 008:00001 librt-2.12.so
> 0000003260007000 4 rw--- 0000000000007000 008:00001 librt-2.12.so
> 0000003260200000 524 r-x-- 0000000000000000 008:00001 libm-2.12.so
> 0000003260283000 2044 ----- 0000000000083000 008:00001 libm-2.12.so
> 0000003260482000 4 r---- 0000000000082000 008:00001 libm-2.12.so
> 0000003260483000 4 rw--- 0000000000083000 008:00001 libm-2.12.so
> 0000003261200000 88 r-x-- 0000000000000000 008:00001 libresolv-2.12.so
> 0000003261216000 2048 ----- 0000000000016000 008:00001 libresolv-2.12.so
> 0000003261416000 4 r---- 0000000000016000 008:00001 libresolv-2.12.so
> 0000003261417000 4 rw--- 0000000000017000 008:00001 libresolv-2.12.so
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
