[
https://issues.apache.org/jira/browse/PHOENIX-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16253337#comment-16253337
]
Prashant Agrawal commented on PHOENIX-4041:
-------------------------------------------
Hello [~samarthjain] , We are still getting an issue where zk connection is
still not getting closed with 4.12 too, do you have any idea if that bug is
still there. (Not sure if this is related to this task though).
More over the details for that is captured at
https://issues.apache.org/jira/browse/PHOENIX-4319 , would be good if you can
share some thought for same as if this is a bug or not.
> CoprocessorHConnectionTableFactory should not open a new HConnection when
> shutting down
> ---------------------------------------------------------------------------------------
>
> Key: PHOENIX-4041
> URL: https://issues.apache.org/jira/browse/PHOENIX-4041
> Project: Phoenix
> Issue Type: Bug
> Reporter: Samarth Jain
> Assignee: Samarth Jain
> Labels: secondary_index
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4041.patch
>
>
> It is wasteful to possibly establish a new HConnection when the
> CoprocessorHConnectionTableFactory is shutting down.
> {code}
> @Override
> public void shutdown() {
> try {
> getConnection(conf).close();
> } catch (IOException e) {
> LOG.error("Exception caught while trying to close the
> HConnection used by CoprocessorHConnectionTableFactory");
> }
> }
> {code}
> In fact, in one of the test runs I saw that the region server aborted when
> getConnection() call in shutDown() ran into an OOM.
> {code}
> org.apache.hadoop.hbase.regionserver.HRegionServer(1950): ABORTING region
> server asf921.gq1.ygridcore.net,43200,1500441052416: Caught throwable while
> processing event M_RS_CLOSE_REGION
> java.lang.RuntimeException: java.lang.OutOfMemoryError: unable to create new
> native thread
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:165)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:714)
> at org.apache.zookeeper.ClientCnxn.start(ClientCnxn.java:406)
> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:450)
> at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.checkZk(RecoverableZooKeeper.java:141)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.<init>(RecoverableZooKeeper.java:128)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:135)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:171)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:145)
> at
> org.apache.hadoop.hbase.client.ZooKeeperKeepAliveConnection.<init>(ZooKeeperKeepAliveConnection.java:43)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher(HConnectionManager.java:1872)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:82)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:926)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:711)
> at
> org.apache.hadoop.hbase.client.CoprocessorHConnection.<init>(CoprocessorHConnection.java:113)
> at
> org.apache.phoenix.hbase.index.write.IndexWriterUtils$CoprocessorHConnectionTableFactory.getConnection(IndexWriterUtils.java:124)
> at
> org.apache.phoenix.hbase.index.write.IndexWriterUtils$CoprocessorHConnectionTableFactory.shutdown(IndexWriterUtils.java:137)
> at
> org.apache.phoenix.hbase.index.write.ParallelWriterIndexCommitter.stop(ParallelWriterIndexCommitter.java:228)
> at
> org.apache.phoenix.hbase.index.write.IndexWriter.stop(IndexWriter.java:225)
> at org.apache.phoenix.hbase.index.Indexer.stop(Indexer.java:222)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:755)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:148)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:318)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:518)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:511)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1280)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1141)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:151)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
