The HBase process just died? The logs end suddenly with nothing about shutting down, no exceptions, etc? Did you check the .out files as well?
> -----Original Message----- > From: Jorome m [mailto:jorom...@gmail.com] > Sent: Tuesday, May 11, 2010 5:58 PM > To: hbase-user@hadoop.apache.org > Subject: Running HBase in Standalone Limitation > > I wonder if there is known performance limitation running HBase in the > standalone-mode? In particular, is there an upper bound to the resource > usage (e.g., total number of HTables, column families, number row > records > per HTable) in a single region server that is not backed by an HDFS? > > I'm experiencing a strange HBase server behavior. After running a Java > HBase > client program for 6+ hours, the HBase server randomly dies. The > machine is > a Linux box, dual core with 6G of physical memory and 4G swap. > > I'm running a standalone HBase server and a single Java client. I start > the > HBase server using the default "start_hbase.sh" BASH script. The Java > client > runs on the same host as the HBase server. Because the Java client is > called > by a separate BASH program that feeds the client with different data > files, > the Java program is periodically started and stopped. > > I have been very careful about the closing and releasing HTable > connections > in the client code. In the program, there is a singleton HBaseAdmin, > which > is responsible for creating different HTable objects. Whenever I'm done > using an HTable object, I explicitly call HTable.close, and whenever > I'm > done using ResutlScanner, I explicit call ResultScanner.close. > > In additional, because I expect the program to creates a lot of HTable > (300+), I also carefully manage the number of active HTable instances. > The > client program explicitly calls HBaseAdmin to disable HTables when its > done > using them. > > I used the HBase SHELL to monitor the HBase server status (status > 'detailed'). I saw that at all time, there is no more than 5 active > HTable > instances. > > When using the "status 'detailed'" command to monitor my system health, > I > saw the number of "heapUsed" keeps increasing. I have set the > HBASE_HEAPSIZE > BASH environment variable to 4G. Because the Java client program is a > long > running program, eventually the HBase server dies. I wonder if it's > caused > by an internal OOM exception. > > I look around the HBase log files and didn't find any obvious Java > exceptions. > > System Information: > > - HBase version: 0.20.4 > > - Java version "1.6.0_20" > Java(TM) SE Runtime Environment (build 1.6.0_20-b02) > Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode) > > - HBase runs with the following JVM args > -Xmx4000m > -XX:+HeapDumpOnOutOfMemoryError > -XX:+UseConcMarkSweepGC > -XX:+CMSIncrementalMode > > > Thanks > J.
