I think I figure it out. The memory is cost by Ranges which load by replay_update() in RangeServer.cc those Ranges will not be compact because their needs_compaction=false;
So I add one line to RangeServer.cc:3616, RangeServer::replay_load_range 3612 schema = table_info->get_schema(); 3613 3614 range = new Range(m_master_client, schema, range_entity, live_table_info.get()); 3615 //## 3616 range->set_needs_compaction(true); //## Qiao Fix Memory Bug 2012-06-20 3617 //## 3618 range->recovery_initialize(); 3619 3620 table_info->add_range(range); After replay finish, it will start to compact cellcaches. We have set Hypertable.RangeServer.MemoryLimit=4GB, but rangeserver still > run out of memory(almost cost 10GB), and 99% cost by CellCache, > > I wait it for almost 1 hour, but nothing changed. I belive the > compactions stopped. > > > Hi, >> >> "Application queue PAUSED due to low memory condition" is actually quite >> common and usually not a problem, unless you have MANY of them. These are >> usually caused either by low memory (because load is too high) or because >> hypertable fails to run compactions. We had a bug in 0.9.5.4 which did not >> reset some internal flags correctly, thus stopping compactions, and the >> cellcaches were not flushed to disk. but this was fixed in 0.9.5.6 (or >> 0.9.5.5, if i remember correctly). >> >> There are a couple of options you can use to control memory behavior: >> >> >> --Hypertable.RangeServer.MemoryLimit arg >> RangeServer memory limit >> --Hypertable.RangeServer.MemoryLimit.Percentage arg (=60) >> RangeServer memory limit >> specified as >> percentage of physical RAM >> --Hypertable.RangeServer.LowMemoryLimit.Percentage arg (=10) >> Amount of memory to free in low >> memory >> condition as percentage of >> RangeServer >> memory limit >> --Hypertable.RangeServer.MemoryLimit.EnsureUnused arg >> Amount of unused physical memory >> --Hypertable.RangeServer.MemoryLimit.EnsureUnused.Percentage arg >> Amount of unused physical memory >> specified as percentage of >> physical RAM >> >> bye >> Christoph >> 2012/6/19 gcc.lua <[email protected]> >> >>> Hi,all >>> I have use c++ stream pipe to implement map/reduce logic handle >>> big data(hypertable/hadoop) >>> I had search the old issues like my problem, it's like CellCache >>> memmory fragment, >>> and fixed use CellCachePool import in 0.9.1.0/0.9.1.1, It be removed >>> in latest version 0.9.5.6(or sth. old version), >>> where can I commit the Hypertable.RangeServer.log, thanks >>> >>> server:(16GB Memory) >>> >>> 20674 line: >>> 1340084923 INFO Hypertable.RangeServer : (/home/hadoop/temp/ >>> hypertable-0615/src/cc/Hypertable/RangeServe r/ >>> MaintenanceScheduler.cc:255) Memory Allocation: BlockCache=0.00% >>> BlockIndex=0.00% BloomFilter=0.00% CellCache=99.51% ShadowCache=0.00% >>> QueryCache=0.49% >>> >>> >>> 221688 line: >>> 1340093490 INFO Hypertable.RangeServer : (/home/hadoop/temp/ >>> hypertable-0615/src/cc/Hypertable/RangeServe r/RangeServer.cc: >>> 3710) Application queue PAUSED due to low memory condition >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Hypertable Development" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/hypertable-dev?hl=en. >>> >>> >> -- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To view this discussion on the web visit https://groups.google.com/d/msg/hypertable-dev/-/oNBjwrL8XIYJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.
