[
https://issues.apache.org/jira/browse/HBASE-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905232#action_12905232
]
Prakash Khemani commented on HBASE-2952:
----------------------------------------
I don’t think that it can be a shutdownHook thread that HCM can accept. The JVM
doesn't allow you to order in any way how the shutdownThreads will be run. It
will have to be a user 'callback' that HCM's shutdownHook() thread will invoke.
Also, I think to get to the HCM instance we have to go through the HTable.
There can be multiple instances of HCM in the same process.
How about HTable::disableShutdownHook(). And then it becomes the caller's
responsibility to make sure HTable::close() is called for every instance of
HTable.
> HConnectionManager's shutdown hook interferes with client's operations
> ----------------------------------------------------------------------
>
> Key: HBASE-2952
> URL: https://issues.apache.org/jira/browse/HBASE-2952
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.20.0
> Reporter: Prakash Khemani
>
> My HBase client calls incrementColValue() in pairs. If someone kills the
> client (SIGINT or SIGTERM) I want my client's increment threads to gracefully
> exit. If a thread has already done one of the incrementColValue() then I want
> that thread to complete the other incrementColValue() and then exit.
> For this purpose I installed my own shutdownHook(). My shitdownHook() thread
> 'sugnals' all the threads in my process that it is time to exit and then
> waits for them to complete.
> The problem is that HConnectionManager's shutdownHook thread also runs and
> shuts down all connections and IPC threads.
> My increment thread keeps waiting to increment and then times out after 240s.
> Two problems with this - the incrementColValiue() didn't go through which
> will increase the chances of inconsistency in my HBase data. And it too 240s
> to exit. I am pasting some of the messages that the client thread outputs
> while it tries contact the HBase server.
> Signalled. Exiting ...
> 2010-09-01 12:11:14,769 DEBUG [HCM.shutdownHook]
> zookeeper.ZooKeeperWrapper(787):
> <localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Closed
> connection with ZooKeeper; /hbase/root-region-server
> flushing after 7899
> 2010-09-01 12:11:19,669 DEBUG [Line Processing Thread 0]
> client.HConnectionManager$TableServers(903): Cache hit for row <> in
> tableName .META.: location server hadoop2205.snc3.facebook.com:60020,
> location region name .META.,,1.1028785192
> 2010-09-01 12:11:19,671 INFO [Line Processing Thread 0]
> zookeeper.ZooKeeperWrapper(206): Reconnecting to zookeeper
> 2010-09-01 12:11:19,671 DEBUG [Line Processing Thread 0]
> zookeeper.ZooKeeperWrapper(212):
> <localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Connected
> to zookeeper again
> 2010-09-01 12:11:24,679 DEBUG [Line Processing Thread 0]
> client.HConnectionManager$TableServers(964): Removed .META.,,1.1028785192 for
> tableName=.META. from cache because of content_action_url_metrics,\x080r&
> B\xF7\x81_T\x07\x08\x16uOrcom.gigya 429934274290948,99999999999999
> 2010-09-01 12:11:24,680 DEBUG [Line Processing Thread 0]
> client.HConnectionManager$TableServers(857): locateRegionInMeta attempt 0 of
> 4 failed; retrying after sleep of 5000 because: The client is stopped
> 2010-09-01 12:11:24,680 DEBUG [Line Processing Thread 0]
> zookeeper.ZooKeeperWrapper(470):
> <localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Trying to
> read /hbase/root-region-server
> 2010-09-01 12:11:24,681 DEBUG [Line Processing Thread 0]
> zookeeper.ZooKeeperWrapper(489):
> <localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Read
> ZNode /hbase/root-region-server got 10.26.119.190:60020
> 2010-09-01 12:11:24,681 DEBUG [Line Processing Thread 0]
> client.HConnectionManager$TableServers(1116): Root region location changed.
> Sleeping.
> ===
> It might be a good idea to only run the HCM shutdown code when all the
> HTables referring to it have been closed. That way the client can control
> when the shutdown actually happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
