You can create several heap dumps of JVM process in question and compare heap allocations To create heap dump:
jmap pid To analize: 1. jhat 2. visualvm 3. any commercial profiler One note: -Xmn12G ??? How long is your minor collections GC pauses? Best regards, Vladimir Rodionov Principal Platform Engineer Carrier IQ, www.carrieriq.com e-mail: vrodio...@carrieriq.com ________________________________________ From: Ramkrishna S Vasudevan [ramkrishna.vasude...@huawei.com] Sent: Wednesday, November 30, 2011 6:51 PM To: u...@hbase.apache.org; dev@hbase.apache.org Subject: RE: Suspected memory leak Adding dev list to get some suggestions. Regards Ram -----Original Message----- From: Shrijeet Paliwal [mailto:shrij...@rocketfuel.com] Sent: Thursday, December 01, 2011 8:08 AM To: u...@hbase.apache.org Cc: Gaojinchao; Chenjian Subject: Re: Suspected memory leak Jieshan, We backported https://issues.apache.org/jira/browse/HBASE-2937 to 0.90.3 -Shrijeet 2011/11/30 bijieshan <bijies...@huawei.com> > Hi Shrijeet, > > I think that's jira relevant to trunk, but not for 90.X. For there's no > timeout mechanism in 90.X. Right? > We found this problem in 90.x. > > Thanks, > > Jieshan. > > -----邮件原件----- > 发件人: Shrijeet Paliwal [mailto:shrij...@rocketfuel.com] > 发送时间: 2011年12月1日 10:26 > 收件人: u...@hbase.apache.org > 抄送: Gaojinchao; Chenjian > 主题: Re: Suspected memory leak > > Gaojinchao, > > I had filed this some time ago, > https://issues.apache.org/jira/browse/HBASE-4633 > But after some recent insights on our application code, I am inclined to > think leak (or memory 'hold') is in our application. But it will be good to > check out either way. > I need to update the jira with my saga. See if the description of issue I > posted there, matches yours. If not, may be you can update with your story > in detail. > > -Shrijeet > > 2011/11/30 Gaojinchao <gaojinc...@huawei.com> > > > I have noticed some memory leak problems in my HBase client. > > RES has increased to 27g > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > 12676 root 20 0 30.8g 27g 5092 S 2 57.5 587:57.76 > > /opt/java/jre/bin/java -Djava.library.path=lib/. > > > > But I am not sure the leak comes from HBase Client jar itself or just our > > client code. > > > > This is some parameters of jvm. > > :-Xms15g -Xmn12g -Xmx15g -XX:PermSize=64m -XX:+UseParNewGC > > -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=65 > > -XX:+UseCMSCompactAtFullCollection -XX:CMSFullGCsBeforeCompaction=1 > > -XX:+CMSParallelRemarkEnabled > > > > Who has experience in this case? , I need continue to dig :) > > > > > > > > 发件人: Gaojinchao > > 发送时间: 2011年11月30日 11:02 > > 收件人: u...@hbase.apache.org > > 主题: Suspected memory leak > > > > In HBaseClient proceess, I found heap has been increased. > > I used command ’cat smaps’ to get the heap size. > > It seems in case when the threads pool in HTable has released the no > using > > thread, if you use putlist api to put data again, the memory is > increased. > > > > Who has experience in this case? > > > > Below is the heap of Hbase client: > > C3S31:/proc/18769 # cat smaps > > 4010a000-4709d000 rwxp 00000000 00:00 0 > > [heap] > > Size: 114252 kB > > Rss: 114044 kB > > Pss: 114044 kB > > > > 4010a000-4709d000 rwxp 00000000 00:00 0 > > [heap] > > Size: 114252 kB > > Rss: 114044 kB > > Pss: 114044 kB > > > > 4010a000-48374000 rwxp 00000000 00:00 0 > > [heap] > > Size: 133544 kB > > Rss: 133336 kB > > Pss: 133336 kB > > > > 4010a000-49f20000 rwxp 00000000 00:00 0 > > [heap] > > Size: 161880 kB > > Rss: 161672 kB > > Pss: 161672 kB > > > > 4010a000-4c5de000 rwxp 00000000 00:00 0 > > [heap] > > Size: 201552 kB > > Rss: 201344 kB > > Pss: 201344 kB > > > Confidentiality Notice: The information contained in this message, including any attachments hereto, may be confidential and is intended to be read only by the individual or entity to whom this message is addressed. If the reader of this message is not the intended recipient or an agent or designee of the intended recipient, please note that any review, use, disclosure or distribution of this message or its attachments, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or notificati...@carrieriq.com and delete or destroy any copy of this message and its attachments.
