[jira] [Updated] (YARN-6017) node manager physical memory leak
[ https://issues.apache.org/jira/browse/YARN-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenrongwei updated YARN-6017: -- Attachment: 31169_smaps.txt > node manager physical memory leak > - > > Key: YARN-6017 > URL: https://issues.apache.org/jira/browse/YARN-6017 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 2.7.1 > Environment: OS: > Linux guomai124041 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 > x86_64 x86_64 x86_64 GNU/Linux > jvm: > java version "1.7.0_65" > Java(TM) SE Runtime Environment (build 1.7.0_65-b17) > Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) >Reporter: chenrongwei > Attachments: 31169_smaps.txt, 31169_smaps.txt > > > In our produce environment, node manager's jvm memory has been set to > '-Xmx2048m',but we notice that after a long time running the process' actual > physical memory size had been reached to 12g (we got this value by top > command as follow). > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND > 31169 data 20 0 13.2g 12g 6092 S 16.9 13.0 49183:13 java > 31169: /usr/local/jdk/bin/java -Dproc_nodemanager -Xmx2048m > -Dhadoop.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs > -Dyarn.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs > -Dhadoop.log.file=yarn-data-nodemanager.log > -Dyarn.log.file=yarn-data-nodemanager.log -Dyarn.home.dir= -Dyarn.id.str=data > -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA > -Djava.library.path=/home/data/programs/apache-hadoop-2.7.1/lib/native > -Dyarn.policy.file=hadoop-policy.xml -XX:PermSize=128M -XX:MaxPermSize=256M > -XX:+UseC > Address Kbytes Mode Offset DeviceMapping > 0040 4 r-x-- 008:1 java > 0060 4 rw--- 008:1 java > 00601000 10094936 rw--- 000:0 [ anon ] > 00077000 2228224 rw--- 000:0 [ anon ] > 0007f800 131072 rw--- 000:0 [ anon ] > 00325ee0 128 r-x-- 008:1 ld-2.12.so > 00325f01f000 4 r 0001f000 008:1 ld-2.12.so > 00325f02 4 rw--- 0002 008:1 ld-2.12.so > 00325f021000 4 rw--- 000:0 [ anon ] > 00325f201576 r-x-- 008:1 libc-2.12.so > 00325f38a0002048 - 0018a000 008:1 libc-2.12.so > 00325f58a000 16 r 0018a000 008:1 libc-2.12.so > 00325f58e000 4 rw--- 0018e000 008:1 libc-2.12.so > 00325f58f000 20 rw--- 000:0 [ anon ] > 00325f60 92 r-x-- 008:1 libpthread-2.12.so > 00325f6170002048 - 00017000 008:1 libpthread-2.12.so > 00325f817000 4 r 00017000 008:1 libpthread-2.12.so > 00325f818000 4 rw--- 00018000 008:1 libpthread-2.12.so > 00325f819000 16 rw--- 000:0 [ anon ] > 00325fa0 8 r-x-- 008:1 libdl-2.12.so > 00325fa020002048 - 2000 008:1 libdl-2.12.so > 00325fc02000 4 r 2000 008:1 libdl-2.12.so > 00325fc03000 4 rw--- 3000 008:1 libdl-2.12.so > 00325fe0 28 r-x-- 008:1 librt-2.12.so > 00325fe070002044 - 7000 008:1 librt-2.12.so > 003260006000 4 r 6000 008:1 librt-2.12.so > 003260007000 4 rw--- 7000 008:1 librt-2.12.so > 00326020 524 r-x-- 008:1 libm-2.12.so > 0032602830002044 - 00083000 008:1 libm-2.12.so > 003260482000 4 r 00082000 008:1 libm-2.12.so > 003260483000 4 rw--- 00083000 008:1 libm-2.12.so > 00326120 88 r-x-- 008:1 libresolv-2.12.so > 0032612160002048 - 00016000 008:1 libresolv-2.12.so > 003261416000 4 r 00016000 008:1 libresolv-2.12.so > 003261417000 4 rw--- 00017000 008:1 libresolv-2.12.so -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6017) node manager physical memory leak
[ https://issues.apache.org/jira/browse/YARN-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenrongwei updated YARN-6017: -- Attachment: 31169_smaps.txt smaps of process 31169 > node manager physical memory leak > - > > Key: YARN-6017 > URL: https://issues.apache.org/jira/browse/YARN-6017 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 2.7.1 > Environment: OS: > Linux guomai124041 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 > x86_64 x86_64 x86_64 GNU/Linux > jvm: > java version "1.7.0_65" > Java(TM) SE Runtime Environment (build 1.7.0_65-b17) > Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) >Reporter: chenrongwei > Attachments: 31169_smaps.txt > > > In our produce environment, node manager's jvm memory has been set to > '-Xmx2048m',but we notice that after a long time running the process' actual > physical memory size had been reached to 12g (we got this value by top > command as follow). > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND > 31169 data 20 0 13.2g 12g 6092 S 16.9 13.0 49183:13 java > 31169: /usr/local/jdk/bin/java -Dproc_nodemanager -Xmx2048m > -Dhadoop.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs > -Dyarn.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs > -Dhadoop.log.file=yarn-data-nodemanager.log > -Dyarn.log.file=yarn-data-nodemanager.log -Dyarn.home.dir= -Dyarn.id.str=data > -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA > -Djava.library.path=/home/data/programs/apache-hadoop-2.7.1/lib/native > -Dyarn.policy.file=hadoop-policy.xml -XX:PermSize=128M -XX:MaxPermSize=256M > -XX:+UseC > Address Kbytes Mode Offset DeviceMapping > 0040 4 r-x-- 008:1 java > 0060 4 rw--- 008:1 java > 00601000 10094936 rw--- 000:0 [ anon ] > 00077000 2228224 rw--- 000:0 [ anon ] > 0007f800 131072 rw--- 000:0 [ anon ] > 00325ee0 128 r-x-- 008:1 ld-2.12.so > 00325f01f000 4 r 0001f000 008:1 ld-2.12.so > 00325f02 4 rw--- 0002 008:1 ld-2.12.so > 00325f021000 4 rw--- 000:0 [ anon ] > 00325f201576 r-x-- 008:1 libc-2.12.so > 00325f38a0002048 - 0018a000 008:1 libc-2.12.so > 00325f58a000 16 r 0018a000 008:1 libc-2.12.so > 00325f58e000 4 rw--- 0018e000 008:1 libc-2.12.so > 00325f58f000 20 rw--- 000:0 [ anon ] > 00325f60 92 r-x-- 008:1 libpthread-2.12.so > 00325f6170002048 - 00017000 008:1 libpthread-2.12.so > 00325f817000 4 r 00017000 008:1 libpthread-2.12.so > 00325f818000 4 rw--- 00018000 008:1 libpthread-2.12.so > 00325f819000 16 rw--- 000:0 [ anon ] > 00325fa0 8 r-x-- 008:1 libdl-2.12.so > 00325fa020002048 - 2000 008:1 libdl-2.12.so > 00325fc02000 4 r 2000 008:1 libdl-2.12.so > 00325fc03000 4 rw--- 3000 008:1 libdl-2.12.so > 00325fe0 28 r-x-- 008:1 librt-2.12.so > 00325fe070002044 - 7000 008:1 librt-2.12.so > 003260006000 4 r 6000 008:1 librt-2.12.so > 003260007000 4 rw--- 7000 008:1 librt-2.12.so > 00326020 524 r-x-- 008:1 libm-2.12.so > 0032602830002044 - 00083000 008:1 libm-2.12.so > 003260482000 4 r 00082000 008:1 libm-2.12.so > 003260483000 4 rw--- 00083000 008:1 libm-2.12.so > 00326120 88 r-x-- 008:1 libresolv-2.12.so > 0032612160002048 - 00016000 008:1 libresolv-2.12.so > 003261416000 4 r 00016000 008:1 libresolv-2.12.so > 003261417000 4 rw--- 00017000 008:1 libresolv-2.12.so -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6017) node manager physical memory leak
[ https://issues.apache.org/jira/browse/YARN-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenrongwei updated YARN-6017: -- Description: In our produce environment, node manager's jvm memory has been set to '-Xmx2048m',but we notice that after a long time running the process' actual physical memory size had been reached to 12g (we got this value by top command as follow). PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31169 data 20 0 13.2g 12g 6092 S 16.9 13.0 49183:13 java 31169: /usr/local/jdk/bin/java -Dproc_nodemanager -Xmx2048m -Dhadoop.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs -Dyarn.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs -Dhadoop.log.file=yarn-data-nodemanager.log -Dyarn.log.file=yarn-data-nodemanager.log -Dyarn.home.dir= -Dyarn.id.str=data -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA -Djava.library.path=/home/data/programs/apache-hadoop-2.7.1/lib/native -Dyarn.policy.file=hadoop-policy.xml -XX:PermSize=128M -XX:MaxPermSize=256M -XX:+UseC Address Kbytes Mode Offset DeviceMapping 0040 4 r-x-- 008:1 java 0060 4 rw--- 008:1 java 00601000 10094936 rw--- 000:0 [ anon ] 00077000 2228224 rw--- 000:0 [ anon ] 0007f800 131072 rw--- 000:0 [ anon ] 00325ee0 128 r-x-- 008:1 ld-2.12.so 00325f01f000 4 r 0001f000 008:1 ld-2.12.so 00325f02 4 rw--- 0002 008:1 ld-2.12.so 00325f021000 4 rw--- 000:0 [ anon ] 00325f201576 r-x-- 008:1 libc-2.12.so 00325f38a0002048 - 0018a000 008:1 libc-2.12.so 00325f58a000 16 r 0018a000 008:1 libc-2.12.so 00325f58e000 4 rw--- 0018e000 008:1 libc-2.12.so 00325f58f000 20 rw--- 000:0 [ anon ] 00325f60 92 r-x-- 008:1 libpthread-2.12.so 00325f6170002048 - 00017000 008:1 libpthread-2.12.so 00325f817000 4 r 00017000 008:1 libpthread-2.12.so 00325f818000 4 rw--- 00018000 008:1 libpthread-2.12.so 00325f819000 16 rw--- 000:0 [ anon ] 00325fa0 8 r-x-- 008:1 libdl-2.12.so 00325fa020002048 - 2000 008:1 libdl-2.12.so 00325fc02000 4 r 2000 008:1 libdl-2.12.so 00325fc03000 4 rw--- 3000 008:1 libdl-2.12.so 00325fe0 28 r-x-- 008:1 librt-2.12.so 00325fe070002044 - 7000 008:1 librt-2.12.so 003260006000 4 r 6000 008:1 librt-2.12.so 003260007000 4 rw--- 7000 008:1 librt-2.12.so 00326020 524 r-x-- 008:1 libm-2.12.so 0032602830002044 - 00083000 008:1 libm-2.12.so 003260482000 4 r 00082000 008:1 libm-2.12.so 003260483000 4 rw--- 00083000 008:1 libm-2.12.so 00326120 88 r-x-- 008:1 libresolv-2.12.so 0032612160002048 - 00016000 008:1 libresolv-2.12.so 003261416000 4 r 00016000 008:1 libresolv-2.12.so 003261417000 4 rw--- 00017000 008:1 libresolv-2.12.so was: In our produce environment, node manager's jvm memory has been set to '-Xmx2048m',but we notice that after a long time running the process' actual physical memory size had been reached to 12g (we got this value by top command as follow). PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31169 data 20 0 13.2g 12g 6092 S 16.9 13.0 49183:13 java > node manager physical memory leak > - > > Key: YARN-6017 > URL: https://issues.apache.org/jira/browse/YARN-6017 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 2.7.1 > Environment: OS: > Linux guomai124041 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 > x86_64 x86_64 x86_64 GNU/Linux > jvm: > java version "1.7.0_65" > Java(TM) SE Runtime Environment (build 1.7.0_65-b17) > Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) >Reporter: chenrongwei > > In our produce environment, node manager's jvm memory has been set to > '-Xmx2048m',but we notice that after a long time running the process' actual > physical memory size had been reached to 12g (we got this value by top > command as follow). > PID USER PR NI VIRT RES SHR S %CPU
