Hi, This is a known problem and we're working on a fix for it now. Your fix should work for now, but is not an ideal solution and could potentially cause worse problems. You should be ok for now, but as soon as we come out with the fixed version, I advise that you upgrade to it immediately.
- Doug On Wed, Jun 20, 2012 at 9:46 AM, BigQiao <[email protected]> wrote: > I think I figure it out. > The memory is cost by Ranges which load by replay_update() in > RangeServer.cc > those Ranges will not be compact because their needs_compaction=false; > > So I add one line to RangeServer.cc:3616, RangeServer::replay_load_range > > 3612 schema = table_info->get_schema(); > 3613 > 3614 range = new Range(m_master_client, schema, range_entity, > live_table_info.get()); > 3615 //## > 3616 range->set_needs_compaction(true); //## Qiao Fix Memory Bug > 2012-06-20 > 3617 //## > 3618 range->recovery_initialize(); > 3619 > 3620 table_info->add_range(range); > > After replay finish, it will start to compact cellcaches. > > > > We have set Hypertable.RangeServer.**MemoryLimit=4GB, but rangeserver >> still run out of memory(almost cost 10GB), and 99% cost by CellCache, >> >> I wait it for almost 1 hour, but nothing changed. I belive the >> compactions stopped. >> >> >> Hi, >>> >>> "Application queue PAUSED due to low memory condition" is actually quite >>> common and usually not a problem, unless you have MANY of them. These are >>> usually caused either by low memory (because load is too high) or because >>> hypertable fails to run compactions. We had a bug in 0.9.5.4 which did not >>> reset some internal flags correctly, thus stopping compactions, and the >>> cellcaches were not flushed to disk. but this was fixed in 0.9.5.6 (or >>> 0.9.5.5, if i remember correctly). >>> >>> There are a couple of options you can use to control memory behavior: >>> >>> >>> --Hypertable.RangeServer.**MemoryLimit arg >>> ** RangeServer memory limit >>> --Hypertable.RangeServer.**MemoryLimit.Percentage arg (=60) >>> ** RangeServer memory limit >>> specified as >>> ** percentage of physical RAM >>> --Hypertable.RangeServer.**LowMemoryLimit.Percentage arg (=10) >>> ** Amount of memory to free in >>> low memory >>> ** condition as percentage of >>> RangeServer >>> ** memory limit >>> --Hypertable.RangeServer.**MemoryLimit.EnsureUnused arg >>> ** Amount of unused physical >>> memory >>> --Hypertable.RangeServer.**MemoryLimit.EnsureUnused.**Percentage arg >>> ** Amount of unused physical >>> memory >>> ** specified as percentage of >>> physical RAM >>> >>> bye >>> Christoph >>> 2012/6/19 gcc.lua <[email protected]> >>> >>>> Hi,all >>>> I have use c++ stream pipe to implement map/reduce logic handle >>>> big data(hypertable/hadoop) >>>> I had search the old issues like my problem, it's like CellCache >>>> memmory fragment, >>>> and fixed use CellCachePool import in 0.9.1.0/0.9.1.1, It be removed >>>> in latest version 0.9.5.6(or sth. old version), >>>> where can I commit the Hypertable.RangeServer.log, thanks >>>> >>>> server:(16GB Memory) >>>> >>>> 20674 line: >>>> 1340084923 INFO Hypertable.RangeServer : (/home/hadoop/temp/ >>>> hypertable-0615/src/cc/**Hypertable/RangeServe r/ >>>> MaintenanceScheduler.cc:255) Memory Allocation: BlockCache=0.00% >>>> BlockIndex=0.00% BloomFilter=0.00% CellCache=99.51% ShadowCache=0.00% >>>> QueryCache=0.49% >>>> >>>> >>>> 221688 line: >>>> 1340093490 INFO Hypertable.RangeServer : (/home/hadoop/temp/ >>>> hypertable-0615/src/cc/**Hypertable/RangeServe r/RangeServer.cc: >>>> 3710) Application queue PAUSED due to low memory condition >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Hypertable Development" group. >>>> To post to this group, send email to >>>> hypertable-dev@googlegroups.**com<[email protected]> >>>> . >>>> To unsubscribe from this group, send email to >>>> hypertable-dev+unsubscribe@**googlegroups.com<hypertable-dev%[email protected]> >>>> . >>>> For more options, visit this group at http://groups.google.com/** >>>> group/hypertable-dev?hl=en<http://groups.google.com/group/hypertable-dev?hl=en> >>>> . >>>> >>>> >>> -- > You received this message because you are subscribed to the Google Groups > "Hypertable Development" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/hypertable-dev/-/oNBjwrL8XIYJ. > > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/hypertable-dev?hl=en. > -- Doug Judd CEO, Hypertable Inc. -- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.
