Hi there- Here are the recommendations from the HBase RefGuide:
http://hbase.apache.org/book.html#perf.os ... and they are consistently with what the book says (recommends 64-bit OS and more memory). Also, keep this in mind... http://hbase.apache.org/book.html#arch.overview ... the standalone deployment should be considered a development config only. On 3/9/12 10:20 PM, "Bing Li" <lbl...@gmail.com> wrote: >Dear all, > >Today I ran HBase in the pseudo-distributed mode. Since I need to save the >meta information about Web pages crawled from more than 10,000 Web sites >simultaneously and periodically. I believe the data size is large. After >running for about 30 minutes, it got the following exceptions. > > [java] ...............2012-3-10 10:09:04 >org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementatio >n >processBatchCallback > [java] Warning: Failed all from >region=PlayTable,,1331343367325.2bf6bf580051da504c347c19eb08f9d5., >hostname=greatfreeinpain, port=60020 > [java] java.util.concurrent.ExecutionException: >java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space > [java] at >java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) > [java] at java.util.concurrent.FutureTask.get(FutureTask.java:83) > [java] at >org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementatio >n.processBatchCallback(HConnectionManager.java:1557) > [java] at >org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementatio >n.processBatch(HConnectionManager.java:1409) > [java] at >org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:900) > [java] at >org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:777) > [java] at org.apache.hadoop.hbase.client.HTable.put(HTable.java:760) > [java] at >org.apache.hadoop.hbase.client.HTablePool$PooledHTable.put(HTablePool.java >:402) > [java] at >com.greatfree.hbase.AuthorityPersister.InsertInitialPlayRecord(AuthorityPe >rsister.java:128) > [java] at >com.greatfree.hbase.RandomPersistDispatcher.run(RandomPersistDispatcher.ja >va:92) > [java] at >java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor. >java:886) > [java] at >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java >:908) > [java] at java.lang.Thread.run(Thread.java:662) > [java] Caused by: java.lang.RuntimeException: >java.lang.OutOfMemoryError: Java heap space > [java] at >org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementatio >n.getRegionServerWithoutRetries(HConnectionManager.java:1371) > [java] at >org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementatio >n$3.call(HConnectionManager.java:1383) > [java] at >org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementatio >n$3.call(HConnectionManager.java:1381) > [java] at >java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > [java] at java.util.concurrent.FutureTask.run(FutureTask.java:138) > [java] ... 3 more > [java] Caused by: java.lang.OutOfMemoryError: Java heap space > [java] at java.util.Arrays.copyOf(Arrays.java:2786) > [java] at >java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94) > [java] at java.io.DataOutputStream.write(DataOutputStream.java:90) > [java] at org.apache.hadoop.hbase.client.Put.write(Put.java:422) > [java] at >org.apache.hadoop.hbase.io.HbaseObjectWritable.writeObject(HbaseObjectWrit >able.java:464) > [java] at org.apache.hadoop.hbase.client.Action.write(Action.java:94) > [java] at >org.apache.hadoop.hbase.io.HbaseObjectWritable.writeObject(HbaseObjectWrit >able.java:464) > [java] at >org.apache.hadoop.hbase.client.MultiAction.write(MultiAction.java:102) > [java] at >org.apache.hadoop.hbase.io.HbaseObjectWritable.writeObject(HbaseObjectWrit >able.java:464) > [java] at >org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:138) > [java] at >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.j >ava:537) > [java] at >org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:879) > [java] at >org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEn >gine.java:150) > [java] at $Proxy6.multi(Unknown Source) > [java] at >org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementatio >n$3$1.call(HConnectionManager.java:1386) > [java] at >org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementatio >n$3$1.call(HConnectionManager.java:1384) > [java] at >org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementatio >n.getRegionServerWithoutRetries(HConnectionManager.java:1365) > [java] ... 7 more > >The machine running the HBase has 1G memory only over Ubuntu 10. According >to the book, HBase the Definitive Guide, 64-bit OS and 4GB are required. I >need to upgrade my system? > >I change the parameter, HBASE_HEAPSIZE, from 1000 to 2048. The exceptions >still occurred. When the value was changed to 3072, it got the following >errors when I ran start-hbase.sh. > > Could not create the Java virtual machine. > Could not create the Java virtual machine. > starting master, logging to >/opt/hbase-0.92.0/bin/../logs/hbase-libing-master-greatfreeinpain.out > Error occurred during initialization of VM > Could not reserve enough space for object heap > Could not create the Java virtual machine. > localhost: starting regionserver, logging to >/opt/hbase-0.92.0/bin/../logs/hbase-libing-regionserver-greatfreeinpain.ou >t > localhost: Error occurred during initialization of VM > localhost: Could not reserve enough space for object heap > localhost: Could not create the Java virtual machine. > >It seems that I need to increase the memory size? Do I need to install a >64-bit Ubuntu Linux? > >Thanks so much for your help! > >Best regards, >Bing
