[
https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256823#comment-13256823
]
Jesse Yates commented on HBASE-5547:
------------------------------------
Talked offline with Lars about the design of this patch. Currently, it is
updating ZK with the table to archive and then essentially waiting on all the
RS to respond that they recieved the update (simple barrier) before considering
backups enabled (in the syncrhonous case). This means we get high overhead to
zk when turning on/off backups, but not a lot when hfiles are deleted. However,
this adds a lot of complexity around watching for nodes, making sure you get
updates, etc. which has been the source of a bit of pain in the development of
this patch.
The alternative design is one where the client just writes the 'enable
archiving on this table' node to ZK and then when any of the RS want to delete
an hfile they go an read ZK. The thought here is that we always have a
consistent view of which tables need to be archived without the complexity of
keeping up-to-date with the latest state. This means a much simpler design for
checking if we should archive a table and very fast enable/disable for backups.
Clearly, there is a tradeoff in that we have to make a network roundtrip to
read if we should delete or archive a table, but there is little else that
needs to happen on the server or wire (zk holds the info in memory and all
thats happening is a checkExists). However, compared to the time cost in
deleting a multi-MB to GB file, this roundtrip seems negligable.
The last alternative would be the dynamic configuration that nicholas recently
rolled into trunk. From what I gather from stack's comments, this would take a
disable/enable on a table for the change to propagate to the HTD (please
correct me if I'm wrong here). This just seems like a lot of flux on a table
for a backup situation that I was thinking could occur as often as a nightly
basis. Making the database unavailable just for backup is unreasonable and
should be done dynamically, with minimal impact to the cluster.
What does everyone think? I'm happy to do the rewrite on the patch to make
either of the alternative designs work, as the current on is not quite ready
(just needs to have the barrier stuff reworked slightly to have new/split
regions register for archiving).
> Don't delete HFiles when in "backup mode"
> -----------------------------------------
>
> Key: HBASE-5547
> URL: https://issues.apache.org/jira/browse/HBASE-5547
> Project: HBase
> Issue Type: New Feature
> Reporter: Lars Hofhansl
> Assignee: Jesse Yates
>
> This came up in a discussion I had with Stack.
> It would be nice if HBase could be notified that a backup is in progress (via
> a znode for example) and in that case either:
> 1. rename HFiles to be delete to <file>.bck
> 2. rename the HFiles into a special directory
> 3. rename them to a general trash directory (which would not need to be tied
> to backup mode).
> That way it should be able to get a consistent backup based on HFiles (HDFS
> snapshots or hard links would be better options here, but we do not have
> those).
> #1 makes cleanup a bit harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
