Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The "Hbase/MigrationToNewCluster" page has been changed by NatHarward. The comment on this change is: Adding the rest of the story.. http://wiki.apache.org/hadoop/Hbase/MigrationToNewCluster?action=diff&rev1=1&rev2=2 -------------------------------------------------- = Migrating an Hbase instance to a new cluster = - ''Note: these steps were performed on a trivially small Hbase instance to start with, results on a larger migration forthcoming'' + ''Note: these steps were performed on a trivially small Hbase instance to start with, results on the larger "real" migration forthcoming'' - The steps below were taken to successfully migrate a complete and working Hbase instance to a brand new, clean cluster in a different location (no shared hardware). The initial setup was as follows: + This is a Hbase users account of migrating data from one data center to another. The steps below were taken to successfully (and non-destructively) migrate a complete and working Hbase instance to a brand new, clean cluster in a different location (no shared hardware). The initial setup was as follows: + * Assumptions + * You have network connectivity between the two clusters + * The new environment is a working environment but pristine, at least for the HDFS and Zookeeper roots you use for Hbase + * Familiarity with Hbase and HDFS administration + * At least one working Map/Reduce cluster available * Existing cluster (data center 1) * HDFS (v. 0.20.2) * Zookeeper (v. 3.2.2) * Hbase (v. 0.20.3) - * New cluster (data center 2) + * New cluster (same software versions, data center 2) + * HDFS + * '''Map/Reduce''' (this was not part of the original cluster, but is necessary to have at least one Map/Reduce cluster available) + * Zookeeper + * Hbase - (to be continued, small interruption...) + The steps: + # Get existing Hbase into a stable state + # Disable access to the existing Hbase, so that no data is changing (optionally disabling your tables just to be sure) + # Major compact all tables + # Flush all tables + # Shut down Hbase + # Get existing HDFS into a stable state + # Disable any other HDFS access besides Hbase which should now be shut down + # Enter HDFS safe-mode + # Run ''fsck'' and verify that everything is ok + # Push Hbase data from the old cluster to the new one + # Use the ''distcp'' command to copy your Hbase root directory to the new cluster (this command uses Map/Reduce, so it should be available where you run this command) + # ''Note: when I tried copying the root of the entire HDFS tree I got a NPE, but pushing a top-level directory worked fine'' + # Verify data in the new HDFS cluster + # I did a spot-check of file sizes and directories to my satisfaction -- depending on how important your data is you may want some kind of checksumming crawler for verification + # Run ''fsck'' to make sure HDFS is happy and does not think anything is wrong + # Fire up Hbase in the new cluster + # Make sure your configuration points to the ''new'' Hbase root directory in the new cluster + # Verify Hbase connectivity, and again to your satisfaction verify the data accessible through your new Hbase instance + # Finito! + + I specifically did not attempt to migrate any Zookeeper data, and found that I did not need to -- I did not encounter any problems in skipping this but your mileage may vary. + + == If something goes wrong == + + Your old cluster should still be intact, though in read-only mode. To bring it back online, make sure HDFS is out of safe mode (if you put it there), then ensure Hbase is running and your tables are enabled. +
