[
https://issues.apache.org/jira/browse/HBASE-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-2669:
-------------------------
Attachment: 2669-v2.txt
Version 2. In this version, we add more explicit closeups of HConnection and
then add a bunch of javadoc explaining how HConnection works and how its
cleanup is done.
{code}
A set of changes that allow doing away with shutdown hook in client.
M src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
Removed unused import and changed message from info to debug
-- when info it shows in shell whenever we run a command.
M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
Changed message from info to debug -- when info it shows in shell
whenever we run a command.
M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Changed order; online region before we tell everyone where the
region is (I changed this order recently but reviewing comments
and issues I can't figure why I did it -- I think there was a
reason but can't recall so just put this back until we trip
over the issue again. My change made it so that we had
strange issue where we'd get a NSRE though the region was coming
up here on this server... rather than do retries of NSREs,
put it into online servers before updating zk... again).
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
Removed noisy message
M src/main/java/org/apache/hadoop/hbase/master/LogCleanerDelegate.java
Have this interface implement Stoppable... Some implemenations
need their Stop called.
M src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
Startup wont work w/o this change.
M src/main/java/org/apache/hadoop/hbase/master/LogCleaner.java
Wrap the chore run so we can call the stop on all log cleaners.
M src/main/java/org/apache/hadoop/hbase/master/TimeToLiveLogCleaner.java
M
src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
Implement Stoppable
M src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java
Cleanup all connections on way out.
M src/main/java/org/apache/hadoop/hbase/util/HMerge.java
Cleanup proxies.... was false.
M src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
M src/main/java/org/apache/hadoop/hbase/client/HConnection.java
Javadoc explaining how HConnections work.
M src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
Make it so we make a new Configuration and that we then
do our own cleanup when pool is shutdown.
M src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
Use alternate method now the one that takes a Connection removed.
M src/main/java/org/apache/hadoop/hbase/client/HTable.java
More javadoc on how HConnection works.
{code}
> HCM.shutdownHook causes data loss with hbase.client.write.buffer != 0
> ---------------------------------------------------------------------
>
> Key: HBASE-2669
> URL: https://issues.apache.org/jira/browse/HBASE-2669
> Project: HBase
> Issue Type: Bug
> Components: client
> Reporter: Benoit Sigoure
> Assignee: stack
> Priority: Critical
> Fix For: 0.90.0
>
> Attachments: 2669-v2.txt, 2669.txt
>
>
> In my application I set {{hbase.client.write.buffer}} to a reasonably small
> value (roughly 64 edits) in order to try to batch a few {{Put}} together
> before talking to HBase. When my application does a graceful shutdown, I
> call {{HTable#flushCommits}} in order to flush any pending change to HBase.
> I want to do the same thing when I get a {{SIGTERM}} by using
> {{Runtime#addShutdownHook}} but this is impossible since
> {{HConnectionManager}} already registers a shutdown hook that invokes
> {{HConnectionManager#deleteAllConnections}}. This static method closes all
> the connections to HBase and then all connections to ZooKeeper. Because all
> shutdown hooks run in parallel, my hook will attempt to flush edits while
> connections are getting closed.
> There is no way to guarantee the order in which the hooks will execute, so I
> propose that we remove the hook in the HCM altogether and provide some
> user-visible API they call in their own hook after they're done flushing
> their stuff, if they really want to do a graceful shutdown. I expect that a
> lot of users won't use a hook though, otherwise this issue would have cropped
> up already. For those users, connections won't get "gracefully" terminated,
> but I don't think that would be a problem since the underlying TCP socket
> will get closed by the OS anyway, so things like ZooKeeper and such should
> realize that the connection has been terminated and assume the client is
> gone, and do the necessary clean-up on their side.
> An alternate fix would be to leave the hook in place by default but keep a
> reference to it and add a user-visible API to be able to un-register the
> hook. I find this ugly.
> Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
