If you can use 0.92 or later or backport a patch check out HBASE-4805. It gives you an option to manage your HConnection directly and create very lightweight HTable objects when you need them.
-- Lars T Vinod Gupta <[email protected]> schrieb: >yeah your usage looks inefficient. what you need to do is keep a copy of >HTable object around and reuse it. but that is not thread safe. so as an >alternative, you need to create a HTablePool and get a HTable from it each >time you need to connect to hbase. after you are done, release the HTable >back to the pool. This is much more efficient. > >thanks > >On Wed, Jan 11, 2012 at 7:39 AM, Shawn Quinn <[email protected]> wrote: > >> Hello, >> >> I'm a relatively new HBase user, and have recently upgraded to the CDH3u2 >> (HBase 0.90.4-cdh3u2) version and after the upgrade I'm running into a new >> problem that appears to be a code problem in HBase - but wanted to post it >> here for validation. >> >> I'm running a client web application (within an OSGi container just for >> extra fun) that does a lot of queries against HBase using the Java client >> via HBaseAdmin to connect. My symptom is that after the upgrade I'm running >> into these errors consistently: >> >> org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to >> connect to ZooKeeper but the connection closes immediately. This could be a >> sign that the server has too many connections (30 is the default). Consider >> inspecting your ZK server logs for that error and then make sure you are >> reusing HBaseConfiguration as often as you can. See HTable's javadoc for >> more information. >> >> And I confirmed in my zookeeper log that I am indeed running out of >> connections. >> >> Digging into my code I validated that I am reusing the same HBase >> Configuration instance (via a static singleton) everytime I construct a new >> HBaseAdmin instance, so that led me to dig into the HBase code a bit and I >> discovered the new HBaseAdmin constructor looks like so: >> >> public HBaseAdmin(Configuration c) >> throws MasterNotRunningException, ZooKeeperConnectionException { >> this.conf = HBaseConfiguration.create(c); >> this.connection = HConnectionManager.getConnection(this.conf); >> this.pause = this.conf.getLong("hbase.client.pause", 1000); >> this.numRetries = this.conf.getInt("hbase.client.retries.number", 10); >> this.retryLongerMultiplier = >> this.conf.getInt("hbase.client.retries.longer.multiplier", 10); >> this.connection.getMaster(); >> } >> >> Where as in prior versions it looked like so: >> >> public HBaseAdmin(Configuration conf) >> throws MasterNotRunningException, ZooKeeperConnectionException { >> this.connection = HConnectionManager.getConnection(conf); >> this.conf = conf; >> this.pause = conf.getLong("hbase.client.pause", 1000); >> this.numRetries = conf.getInt("hbase.client.retries.number", 10); >> this.retryLongerMultiplier = >> conf.getInt("hbase.client.retries.longer.multiplier", 10); >> this.connection.getMaster(); >> } >> >> So, I put a break point in the new version and confirmed that the call to >> "HBaseConfiguration.create(c)" is creating a new Configuration instance >> each time the constructor is invoked, which then leads the >> "HConnectionManager.getConnection(this.conf)" to create a new connection >> instance each time. I'm guessing that's the reason why I'm running out of >> ZooKeeper connections, but it seems like others would have run into a >> similar problem so I'm doubting my conclusion a bit! I did confirm that if >> I change the HBaseAdmin constructor code back to how it looks in previous >> versions that I no longer have the issue. >> >> Can anyone confirm or deny if this updated constructor is problematic? >> >> Thanks, >> >> -Shawn >>
