This is most likely due to heap fragmentation. It's not uncommon to see the RSS reach twice the configured memory limit. Does the RSS eventually level off? Also, if you post your test code (insert client code), I might be able to give you some ideas for how to mitigate the problem. I just need to know exactly the API methods you're using.
Also, it appears that most of the memory is taken up by the block cache. Does your test involve reads as well? - Doug On Thu, Apr 14, 2011 at 2:25 AM, pan yue <[email protected]> wrote: > I deployed Hypertable in two machines, Hyperspace and Master was deployed > on machine1, and RangeServer deployed on machine2. > when I start 4 clients to insert k-v pairs into this cluster (1000000 cells > per client, every cell was 64KB), RangerServer consumes 9G memory: > (machine2 have 8 cores, 16GB memory) > > here is the memory staticstic from log file: > > Memory Statistics (MB): VM=10594.29, RSS=9019.38, tracked=3889.75, > computed=3888.70 limit=4096.00 > Memory Allocation: BlockCache=97.74% BlockIndex=0.79% BloomFilter=0.09% > CellCache=0.09% ShadowCache=0.00% QueryCache=1.29% > Memory Usage: 3889751572 bytes > > The resident memory about 9GB(RSS=9019.38), but rangeserver declared it > only use 4GB ( computed=3888.70), where is remain memory going ? > does this memory leak? > > The config file fo RangerServer like this: > # Hypertable.RangeServer > Hypertable.RangeServer.Port=38060 > Hypertable.RangeServer.Workers=100 > Hypertable.RangeServer.Reactors=20 > > Hypertable.RangeServer.MemoryLimit=4096000000 > Hypertable.RangeServer.MemoryLimit.Percentage=25 > > Thank you! > > -- > You received this message because you are subscribed to the Google Groups > "Hypertable User" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/hypertable-user?hl=en. > -- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.
