Hmm... Seems like we need to add a "removeListener(ZooKeeperListener)" method to ZooKeeperWatcher and call it from ZooKeeperNodeTracker.stop()... Messy.
________________________________ From: Ramkrishna S Vasudevan <[email protected]> To: 'lars hofhansl' <[email protected]>; [email protected] Sent: Monday, December 19, 2011 5:32 PM Subject: RE: Registered listeners not getting removed - Leading to memory leak Hi All In trunk the problem doesnot exist. Every time when we do getCatalogTracker as part of the constructor atalogTracker(final ZooKeeperWatcher zk, final Configuration conf, HConnection connection, Abortable abortable, final int defaultTimeout) throws IOException { this.connection = connection; if (abortable == null) { // A connection is abortable. this.abortable = this.connection; } Abortable throwableAborter = new Abortable() { @Override public void abort(String why, Throwable e) { throw new RuntimeException(why, e); } @Override public boolean isAborted() { return true; } }; if (zk == null) { // Create our own. Set flag so we tear it down on stop. this.zookeeper = new ZooKeeperWatcher(conf, "catalogtracker-on-" + connection.toString(), abortable); instantiatedzkw = true; } else { this.zookeeper = zk; Everytime as zk is null we create new zookeeper watcher.. But in the 0.90.x version we don't do that.. we get the zkwatcher from the connection object. That is why the listener is not getting cleared. Any suggestions on this? Regards Ram ________________________________ From:lars hofhansl [mailto:[email protected]] Sent: Tuesday, December 20, 2011 12:17 AM To: [email protected]; [email protected] Subject: Re: Registered listeners not getting removed - Leading to memory leak A friend of mine who is evaluating HBase just told that calling HBaseAdmin.tableExists frequently slows down the client. This is probably the same issue. He was using stock CHD3 HBase. -- Lars ________________________________ From:Ramkrishna S Vasudevan <[email protected]> To: [email protected] Sent: Monday, December 19, 2011 3:59 AM Subject: Registered listeners not getting removed - Leading to memory leak Hi From the HBaseAdmin if i try to use apis like tableExists, compact, flush, closeRegion all creates new CataLogTracker. Everytime we register a listener but we dont remove the listeners. Note that, every catalog tracker creation calls HConnection.getConnection(). On continuous run of admin apis this is leading to memory leak. Correct me if am wrong. Any suggestions on this on how to clear the listeners? Regards Ram
