Created HBASE-5058
----- Original Message ----- From: lars hofhansl <[email protected]> To: "[email protected]" <[email protected]> Cc: Sent: Friday, December 16, 2011 9:17 AM Subject: Re: HBaseAdmin short leaved and ZK connections You could try to backport HBASE-4283 to 0.90,that should allow HBaseAdmin to survive a master restart. And I think I should extend HBASE-4805 to HBaseAdmin (which would allow you to maintain an HConnection and create an HBaseAdmin when needed, just like HBASE-4805 let's you do for HTables). -- Lars ________________________________ From: Frédéric Fondement <[email protected]> To: [email protected] Sent: Friday, December 16, 2011 3:20 AM Subject: HBaseAdmin short leaved and ZK connections Dear all, I'm using HBase 0.90.3. In the Javadoc for HBaseAdmin, it's written: "Currently HBaseAdmin instances are not expected to be long-lived. For example, an HBaseAdmin instance will not ride over a Master restart." As such, I re-create an HBaseAdmin object as soon as necessary (e.g. to check that a table exists, get a table descriptor, alter tables, get the ZK connection to synchronize my processes, ...) at different points on my program (actually a library, which runs concurrently on different programs on different JVMs possibly on a same node). Moreover, I have to say that I cache HTable (and HTableDesciptor) objects ; as such, those are long-lived. A single Configuration object is used for all that stuff. However, in the (unique) HBaseAdmin constructor, the code is the following: public HBaseAdmin(Configuration c) throws MasterNotRunningException, ZooKeeperConnectionException { this.conf = HBaseConfiguration.create(c); this.connection = HConnectionManager.getConnection(this.conf); ... } This makes the HConnectionManager create a new connection to Zookeeper each time I create an HBaseAdmin. Finally, I end up with numeous 2011-12-16 11:43:22,913 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.59.14.112 - max is 60 in my zookeeper logs, which is not the case when I reuse a single (long-lived) HBaseAdmin object. To overcome this problem, I encapsulate any use of my admin objet with a try/finally block to call a HConnectionManager.deleteConnection(admin.getConfiguration(), true); so that connections are closed... I have to say I prefer using a long-lived HBaseAdmin and treat errors in case of problem by recreating another one (which is far less frequent). Is there any mean to force the HBaseAdmin reuse the Zookeeper client connection ? Best regards, Frédéric
