[ https://issues.apache.org/jira/browse/HBASE-14370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735527#comment-14735527 ]
Ted Yu commented on HBASE-14370: -------------------------------- w.r.t. thread leak, have you seen the following code ? {code} + public void close() { + executor.shutdown(); {code} w.r.t. AtomicReference, the goal is for refresher thread to be interruptible. w.r.t. race condition between nodeChildrenChanged and nodeDeleted, if a table (namespace) is deleted, client would get TableNotFoundException (NamespaceNotFoundException) for future access - before ACL is checked. Do you think tighter coordination is needed between the zk thread and the refresher thread ? > Use separate thread for calling ZKPermissionWatcher#refreshNodes() > ------------------------------------------------------------------ > > Key: HBASE-14370 > URL: https://issues.apache.org/jira/browse/HBASE-14370 > Project: HBase > Issue Type: Bug > Affects Versions: 0.98.0 > Reporter: Ted Yu > Assignee: Ted Yu > Attachments: 14370-v1.txt, 14370-v3.txt > > > I came off a support case (0.98.0) where main zk thread was seen doing the > following: > {code} > at > org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.refreshAuthManager(ZKPermissionWatcher.java:152) > at > org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.refreshNodes(ZKPermissionWatcher.java:135) > at > org.apache.hadoop.hbase.security.access.ZKPermissionWatcher.nodeChildrenChanged(ZKPermissionWatcher.java:121) > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:348) > at > org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519) > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495) > {code} > There were 62000 nodes under /acl due to lack of fix from HBASE-12635, > leading to slowness in table creation because zk notification for region > offline was blocked by the above. > The attached patch separates refreshNodes() call into its own thread. > Thanks to Enis and Devaraj for offline discussion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)