No, I'm not using LZO on this host. Only cloudera hadoop 0.20.2+320 + hbase 0.89.20100830
Digging google gives only hints, that jit or something in jvm can eat memory, but nothing concrete. pmap shows that some memory blocks are grow in size... but what are they, i can't imagine. 000000004010a000 29564K 28996K 28996K 28996K 0K rwxp [heap] 00007f0d88000000 65492K 132K 132K 132K 0K rwxp [anon] 00007f0d8bff5000 44K 0K 0K 0K 0K ---p [anon] 00007f0d8c000000 51760K 132K 132K 132K 0K rwxp [anon] 00007f0d8f28c000 13776K 0K 0K 0K 0K ---p [anon] 00007f0d90000000 65536K 65536K 65536K 65536K 0K rwxp [anon] 00007f0d9c000000 65488K 65488K 65488K 65488K 0K rwxp [anon] 00007f0dc14af000 170372K 170372K 170372K 170372K 0K rwxp [anon] 00007f0dcbb10000 1877632K 1874624K 1874624K 1874624K 0K rwxp [anon] 00007f0e3e4b0000 33496K 20028K 20028K 20028K 0K rwxp [anon] 00007f0e40566000 52520K 0K 0K 0K 0K rwxp [anon] ... many rows skipped with small numbers 2010/12/30 Todd Lipcon <[email protected]> > Hi Andrey, > > Any chance you're using hadoop-lzo with CDH3b3? There was a leak in earlier > versions of hadoop-lzo that showed up under CDH3b3. You should upgrade to > the newest. > > If that's not it, let me know, will keep thinking. > > -Todd > > On Thu, Dec 30, 2010 at 12:13 AM, Andrey Stepachev <[email protected]> > wrote: > > > Addition information: > > > > ps shows, that my HBase process eats up to 4GB of RSS. > > > > $ ps --sort=-rss -eopid,rss | head | grep HMaster > > PID RSS > > 23476 3824892 > > > > > > 2010/12/30 Andrey Stepachev <[email protected]> > > > > > Hi All. > > > > > > After heavy load into hbase (single node, nondistributed test system) I > > got > > > 4Gb process size of my HBase java process. > > > On 6GB machine there was no room for anything else (disk cache and so > > on). > > > > > > Does anybody knows, what is going on, and how you solve this. What heap > > > memory is set on you hosts > > > and how much of RSS hbase process actually use. > > > > > > I don't see such things before, all tomcat and other java apps don't > eats > > > significally more memory then -Xmx. > > > > > > Connection name: pid: 23476 org.apache.hadoop.hbase.master.HMaster > > > start Virtual Machine: Java HotSpot(TM) 64-Bit Server VM version > > > 17.1-b03 Vendor: Sun Microsystems Inc. Name: 23...@mars > > Uptime: 12 > > > hours 4 minutes Process CPU time: 5 hours 45 minutes JIT > compiler: > > HotSpot > > > 64-Bit Server Compiler Total compile time: 19,223 seconds > > > ------------------------------ > > > Current heap size: 703 903 kbytes Maximum heap size: 2 030 > > 976kbytes Committed memory: > > > 2 030 976 kbytes Pending finalization: 0 objects Garbage > > > collector: Name = 'ParNew', Collections = 9 990, Total time spent = 5 > > > minutes Garbage collector: Name = 'ConcurrentMarkSweep', > Collections > > = > > > 20, Total time spent = 35,754 seconds > > > ------------------------------ > > > Operating System: Linux 2.6.34.7-0.5-xen Architecture: amd64 > > Number of processors: > > > 8 Committed virtual memory: 4 403 512 kbytes Total physical > > > memory: 6 815 744 kbytes Free physical memory: 82 720 kbytes > > Total swap space: > > > 8 393 924 kbytes Free swap space: 8 050 880 kbytes > > > > > > > > > > > > > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
