[ https://issues.apache.org/jira/browse/HBASE-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906486#comment-12906486 ]
Zhihong Yu edited comment on HBASE-50 at 3/29/12 7:02 PM: ---------------------------------------------------------- Message from: "Chongxin Li" <lichong...@zju.edu.cn> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/467/ ----------------------------------------------------------- (Updated 2010-09-06 04:34:53.459404) Review request for hbase. Changes ------- Add Mapreduce based export (ExportSnapshot) and import (ImportSnapshot) for snapshot, so that snapshot of an hbase table could be exported and imported to other data centers. Unit test (TestSnapshotExport) has passed. Summary ------- This patch includes the first three sub-tasks of HBASE-50: 1. Start and monitor the creation of snapshot via ZooKeeper 2. Create snapshot of an HBase table 3. Some existing functions of HBase are modified to support snapshot Currently snapshots can be created as expected, but can not be restored or deleted yet This addresses bug HBASE-50. http://issues.apache.org/jira/browse/HBASE-50 Diff: http://review.cloudera.org/r/467/diff Testing ------- Unit tests and integration tests with mini cluster passed. Thanks, Chongxin was (Author: hbasereviewboard): Message from: "Chongxin Li" <lichong...@zju.edu.cn> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/467/ ----------------------------------------------------------- (Updated 2010-09-06 04:34:53.459404) Review request for hbase. Changes ------- Add Mapreduce based export (ExportSnapshot) and import (ImportSnapshot) for snapshot, so that snapshot of an hbase table could be exported and imported to other data centers. Unit test (TestSnapshotExport) has passed. Summary ------- This patch includes the first three sub-tasks of HBASE-50: 1. Start and monitor the creation of snapshot via ZooKeeper 2. Create snapshot of an HBase table 3. Some existing functions of HBase are modified to support snapshot Currently snapshots can be created as expected, but can not be restored or deleted yet This addresses bug HBASE-50. http://issues.apache.org/jira/browse/HBASE-50 Diffs (updated) ----- bin/add_snapshot_family.rb PRE-CREATION src/main/java/org/apache/hadoop/hbase/HConstants.java bfaa4a1 src/main/java/org/apache/hadoop/hbase/HRegionInfo.java ee94690 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 0d57270 src/main/java/org/apache/hadoop/hbase/SnapshotDescriptor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/SnapshotExistsException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/TablePartiallyOpenException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 8b01aa0 src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java d35a28a src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 20860d6 src/main/java/org/apache/hadoop/hbase/io/Reference.java 219203c src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPCProtocolVersion.java d4bcbed src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java bd48a4b src/main/java/org/apache/hadoop/hbase/mapreduce/ExportSnapshot.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/mapreduce/ImportSnapshot.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 1183584 src/main/java/org/apache/hadoop/hbase/master/BaseScanner.java 2deea4a src/main/java/org/apache/hadoop/hbase/master/DeleteSnapshot.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/master/HMaster.java 4735304 src/main/java/org/apache/hadoop/hbase/master/LogsCleaner.java 9d1a8b8 src/main/java/org/apache/hadoop/hbase/master/RestoreSnapshot.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/master/SnapshotLogCleaner.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/master/SnapshotMonitor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/master/SnapshotOperation.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/master/SnapshotSentinel.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/master/TableDelete.java 1153e62 src/main/java/org/apache/hadoop/hbase/master/TableSnapshot.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 9fdd86d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8356d64 src/main/java/org/apache/hadoop/hbase/regionserver/Snapshotter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java f1d52b7 src/main/java/org/apache/hadoop/hbase/regionserver/Store.java ae9e190 src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 107d641 src/main/java/org/apache/hadoop/hbase/regionserver/ZKSnapshotWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 20a535c src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 4d4b00a src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 5cf3481 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWrapper.java 3256ac9 src/main/resources/hbase-default.xml 419bc6d src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java fadee21 src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java c9b78b9 src/test/java/org/apache/hadoop/hbase/mapreduce/TestSnapshotExport.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 8b7f60f src/test/java/org/apache/hadoop/hbase/master/TestSnapshot.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/master/TestSnapshotFailure.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java 34b8044 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java c425953 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionSnapshot.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 38ef520 src/test/java/org/apache/hadoop/hbase/regionserver/TestZKSnapshotWatcher.java PRE-CREATION Diff: http://review.cloudera.org/r/467/diff Testing ------- Unit tests and integration tests with mini cluster passed. Thanks, Chongxin > Snapshot of table > ----------------- > > Key: HBASE-50 > URL: https://issues.apache.org/jira/browse/HBASE-50 > Project: HBase > Issue Type: New Feature > Affects Versions: 0.96.0 > Reporter: Billy Pearson > Assignee: Li Chongxin > Priority: Minor > Labels: gsoc > Fix For: 0.96.0 > > Attachments: HBase Snapshot Design Report V2.pdf, HBase Snapshot > Design Report V3.pdf, HBase Snapshot Implementation Plan.pdf, Snapshot Class > Diagram.png > > > Havening an option to take a snapshot of a table would be vary useful in > production. > What I would like to see this option do is do a merge of all the data into > one or more files stored in the same folder on the dfs. This way we could > save data in case of a software bug in hadoop or user code. > The other advantage would be to be able to export a table to multi locations. > Say I had a read_only table that must be online. I could take a snapshot of > it when needed and export it to a separate data center and have it loaded > there and then i would have it online at multi data centers for load > balancing and failover. > I understand that hadoop takes the need out of havening backup to protect > from failed servers, but this does not protect use from software bugs that > might delete or alter data in ways we did not plan. We should have a way we > can roll back a dataset. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira