[
https://issues.apache.org/jira/browse/HBASE-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12964499#action_12964499
]
Ted Yu commented on HBASE-2506:
-------------------------------
Currently load balancer defines server load by adding number of regions.
Another definition of server load can be the summation of number of (write)
accesses in the last N minutes for each region hosted by the server.
HRegionInfo can use a rolling buffer of length N to record the number of
accesses in each minute.
Under this definition, load balancer gives low weight to regions that are
rarely accessed in the near past and tries to balance 'hot' regions.
> Too easy to OOME a RS
> ---------------------
>
> Key: HBASE-2506
> URL: https://issues.apache.org/jira/browse/HBASE-2506
> Project: HBase
> Issue Type: Bug
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.92.0
>
>
> Testing a cluster with 1GB heap, I found that we are letting the region
> servers kill themselves too easily when scanning using pre-fetching. To
> reproduce, get 10-20M rows using PE and run a count in the shell using CACHE
> => 30000 or any other very high number. For good measure, here's the stack
> trace:
> {code}
> 2010-04-30 13:20:23,241 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: OutOfMemoryError,
> aborting.
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:2786)
> at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
> at java.io.DataOutputStream.write(DataOutputStream.java:90)
> at org.apache.hadoop.hbase.client.Result.writeArray(Result.java:478)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.writeObject(HbaseObjectWritable.java:312)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.write(HbaseObjectWritable.java:229)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:941)
> 2010-04-30 13:20:23,241 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> request=0.0, regions=29, stores=29, storefiles=44, storefileIndexSize=6,
> memstoreSize=255,
> compactionQueueSize=0, usedHeap=926, maxHeap=987, blockCacheSize=1700064,
> blockCacheFree=205393696, blockCacheCount=0, blockCacheHitRatio=0
> {code}
> I guess the same could happen with largish write buffers. We need something
> better than OOME.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
