[
https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249319#comment-13249319
]
[email protected] commented on HBASE-5547:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4633/
-----------------------------------------------------------
(Updated 2012-04-07 19:51:11.515830)
Review request for hbase, Michael Stack and Lars Hofhansl.
Changes
-------
A bit rearchitecting in the new patch:
1) Separated the client and server archiving into distinct classes (and saved
changes to HMasterInferface)
2) A lot of the functions in the ArchiveUtil were moved to their respective
classes and move a couple smaller methods in
3) Added a synchronous enable as the default for the client
4) Merged the tracker and the manger into the tracker and pulled out the
important methods into a monitor interface.
5) Addressed Ted's comments from the last patch (and hopefully didn't add them
back in the change).
6) Added a bunch of testing
Summary
-------
Essentially, whenever an hfile would be deleted, it is instead moved to the
archive directory. In this impl, the archive directory is on a per table basis,
but defaults to '.archive'. Removing hfiles occurs in three places -
compaction, merge and catalog janitor. The former and two latter are distinctly
different code paths, but but did pull out some similarities. The latter two
end up calling the same method, so there should be a reasonable amount of
overlap.
Implementation wise:
Updated the HMasterInterface to pass the calls onto the zookeeper.
Added a zk listener to handle updates from the master to the RS to backup.
Added a utility for removing files and finding archive directories
Added tests for the regionserver and catalogjanitor approaches.
Added creation of manager in regionserver.
This addresses bug HBASE-5547.
https://issues.apache.org/jira/browse/HBASE-5547
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/HConstants.java a4b989e
src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017
src/main/java/org/apache/hadoop/hbase/client/HFileArchiveManager.java
PRE-CREATION
src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java 79d5fdd
src/main/java/org/apache/hadoop/hbase/master/HMaster.java fb21bdd
src/main/java/org/apache/hadoop/hbase/regionserver/HFileArchiveMonitor.java
PRE-CREATION
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c3df319
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d
src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
6884d53
src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 509a467
src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/zookeeper/HFileArchiveTracker.java
PRE-CREATION
src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerHFileTracker.java
PRE-CREATION
src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 4fc105f
src/main/resources/hbase-default.xml 44ee689
src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8
src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83
src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionHFileArchiving.java
PRE-CREATION
src/test/java/org/apache/hadoop/hbase/util/HFileArchiveTestingUtil.java
PRE-CREATION
src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
7d02759
Diff: https://reviews.apache.org/r/4633/diff
Testing
-------
Added two tests for the separate cases - archiving via the regionserver and for
the catalog tracker. Former runs in a mini cluster and also touches the changes
to HMasterInterface and zookeeper.
Thanks,
Jesse
> Don't delete HFiles when in "backup mode"
> -----------------------------------------
>
> Key: HBASE-5547
> URL: https://issues.apache.org/jira/browse/HBASE-5547
> Project: HBase
> Issue Type: New Feature
> Reporter: Lars Hofhansl
> Assignee: Jesse Yates
>
> This came up in a discussion I had with Stack.
> It would be nice if HBase could be notified that a backup is in progress (via
> a znode for example) and in that case either:
> 1. rename HFiles to be delete to <file>.bck
> 2. rename the HFiles into a special directory
> 3. rename them to a general trash directory (which would not need to be tied
> to backup mode).
> That way it should be able to get a consistent backup based on HFiles (HDFS
> snapshots or hard links would be better options here, but we do not have
> those).
> #1 makes cleanup a bit harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
