[ 
https://issues.apache.org/jira/browse/HBASE-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569958#comment-15569958
 ] 

stack commented on HBASE-7912:
------------------------------

I wanted to try removing the hbase:backup table since my install already had it 
so I do a listing in the shell and don't see the backup table but it is in the 
list of regions. I suppose we don't show hbase:backup in our  shell listing (or 
any of our system tables).  Over in review on the mega patch, I ask about the 
hbase:backup table. I get this even though I don't do any backups it seems. We 
don't want that, right?

I was able to disable it and drop it (that ok?) even though it not listed as a 
table.

I restart and get the complaint about TableNotFoundException again. The table 
is created though I've not asked for it and have done no backups.

Up in the review of the  monster patch  note that command descriptions are 
capitalized for all commands but the two newly added backup and restore.

{code}
Some commands take arguments. Pass no args or -h for usage.
  shell           Run the HBase shell
  hbck            Run the hbase 'fsck' tool
  snapshot        Create a new snapshot of a table
  snapshotinfo    Tool for dumping snapshot information
  wal             Write-ahead-log analyzer
  hfile           Store file analyzer
  zkcli           Run the ZooKeeper shell
  master          Run an HBase HMaster node
  regionserver    Run an HBase HRegionServer node
  zookeeper       Run a Zookeeper server
  rest            Run an HBase REST server
  thrift          Run the HBase Thrift server
  thrift2         Run the HBase Thrift2 server
  clean           Run the HBase clean up script
  classpath       Dump hbase CLASSPATH
  mapredcp        Dump CLASSPATH entries required by mapreduce
  pe              Run PerformanceEvaluation
  ltt             Run LoadTestTool
  canary          Run the Canary tool
  version         Print the version
  backup          backup tables for recovery
  restore         restore tables from existing backup image
  CLASSNAME       Run the class named CLASSNAME
{code}

 I notice here that the inconsistency is carried over into the restore and 
backup subcommands. Would suggest that the format of the below align w/ the 
above. Why do something 'new'.

{code}
Usage: hbase backup COMMAND [command-specific arguments]
where COMMAND is one of:
  create     create a new backup image
  delete     delete an existing backup image
  describe   show the detailed information of a backup image
  history    show history of all successful backups
  progress   show the progress of the latest backup request
  set        backup set management
Run 'hbase backup COMMAND -h' to see help message for each command
{code}

When I ran backup command I got the above. When I ran the restore to see what 
it can do I got an error:
{code}
stack@ve0524:~$ ./hbase/bin/hbase --config ~/conf_hbase restore
2016-10-12 14:27:54,702 DEBUG [main] backup.RestoreDriver: Will automatically 
restore all the dependencies
ERROR: remain args length=0
Usage: hbase restore [-set set_name] <backup_root_path> <backup_id> <tables> 
[tableMapping]
       [-overwrite] [-check]
 backup_root_path  The parent location where the backup images are stored
 backup_id         The id identifying the backup image
 table(s)          Table(s) from the backup image to be restored.
                   Tables are separated by comma.
 Options:
   tableMapping    A comma separated list of target tables.
                   If specified, each table in <tables> must have a mapping.
   -overwrite      With this option, restore overwrites to the existing table 
if there's any in
                   restore target. The existing table must be online before 
restore.
   -check          With this option, sequence and dependencies are checked
                   and verified without executing the restore command
   -set set_name   Backup set to restore, mutually exclusive with table list 
<tables>.
{code}

I tried 'backup create' and I got an error for seemingly the same reason as I 
got an error above -- missing an argument (which is fine) but the error message 
is different this time. Above it is: 'ERROR: remain args length=0'. Here it is 
'ERROR: wrong number of arguments: 1'.

Looking in my shell at the output for backup, restore, and 'backup create', 
none of the columns align... sometimes Options: is up against the left edge, 
other times it is offset. Ditto for options. Needs to be consistent else gives 
the impression that the tool can't be trusted to do backup and restore (see 
monster patch review for some detail on why we might have the inconsistency 
here).

In 'restore', the '-overwrite' is mentioned in the options list bu tnot in the 
usage top-line.

In monster patch review it notes that sometimes options are single letter but 
mostly they are words.. set, table.

If I do 'backup set', I get an error because I am missing args but as opposed 
to the above complaints that I'm missing an arg., I get a new error type again:

{code}
ERROR: Command line format
{code}

I'd think all complaints about missing args would error the same.

In this command, the options come before the commands listing (I'd think 
options would come after as is usual and as per their listing on the usage 
line).

Trying to make a backup, I am looking at BACKUP_ROOT which is capitalized for 
no good reason. I see in the description it says: " BACKUP_ROOT     The full 
root path to store the backup image, the prefix can be hdfs, webhdfs or gpfs" 
What is 'prefix'. Do we mean 'scheme'? We support gpfs now?

I tried to create a backup:
{code}
$ ./hbase/bin/hbase --config ~/conf_hbase backup create full 
hdfs://ve0524.halxg.cloudera.com:8020/backu
{code}

It failed with java.lang.IllegalArgumentException Wrong FS: 
hdfs://ve0524.halxg.cloudera.com:8020/hbase/WALs, expected: file:/// and then 
later I got an NPE: 

2016-10-12 14:45:39,064 ERROR [main] backup.BackupDriver: Error running 
command-line tool
java.lang.NullPointerException
        at 
org.apache.hadoop.hbase.backup.impl.FullTableBackupClient.cleanupTargetDir(FullTableBackupClient.java:178)
        at 
org.apache.hadoop.hbase.backup.impl.FullTableBackupClient.failBackup(FullTableBackupClient.java:251)
        at 
org.apache.hadoop.hbase.backup.impl.FullTableBackupClient.execute(FullTableBackupClient.java:533)
        at 
org.apache.hadoop.hbase.backup.impl.HBaseBackupAdmin.backupTables(HBaseBackupAdmin.java:532)
        at 
org.apache.hadoop.hbase.backup.impl.BackupCommands$CreateCommand.execute(BackupCommands.java:225)
        at 
org.apache.hadoop.hbase.backup.BackupDriver.parseAndRun(BackupDriver.java:114)
        at 
org.apache.hadoop.hbase.backup.BackupDriver.doWork(BackupDriver.java:135)
        at 
org.apache.hadoop.hbase.backup.BackupDriver.run(BackupDriver.java:171)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at 
org.apache.hadoop.hbase.backup.BackupDriver.main(BackupDriver.java:140)

Not sure why it says 'Wrong FS'.

I'll leave it at that for now.











> HBase Backup/Restore Based on HBase Snapshot
> --------------------------------------------
>
>                 Key: HBASE-7912
>                 URL: https://issues.apache.org/jira/browse/HBASE-7912
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Richard Ding
>            Assignee: Vladimir Rodionov
>              Labels: backup
>             Fix For: 2.0.0
>
>         Attachments: Backup-and-Restore-Apache_19Sep2016.pdf, 
> Backup-and-Restore-Apache_9Sep2016.pdf, HBaseBackupAndRestore - v0.8.pdf, 
> HBaseBackupAndRestore -0.91.pdf, HBaseBackupAndRestore-v0.9.pdf, 
> HBaseBackupAndRestore.pdf, HBaseBackupRestore-Jira-7912-DesignDoc-v1.pdf, 
> HBaseBackupRestore-Jira-7912-DesignDoc-v2.pdf, 
> HBaseBackupRestore-Jira-7912-v4.pdf, HBaseBackupRestore-Jira-7912-v5 .pdf, 
> HBaseBackupRestore-Jira-7912-v6.pdf, HBase_BackupRestore-Jira-7912-CLI-v1.pdf
>
>
> Finally, we completed the implementation of our backup/restore solution, and 
> would like to share with community through this jira. 
> We are leveraging existing hbase snapshot feature, and provide a general 
> solution to common users. Our full backup is using snapshot to capture 
> metadata locally and using exportsnapshot to move data to another cluster; 
> the incremental backup is using offline-WALplayer to backup HLogs; we also 
> leverage global distribution rolllog and flush to improve performance; other 
> added-on values such as convert, merge, progress report, and CLI commands. So 
> that a common user can backup hbase data without in-depth knowledge of hbase. 
>  Our solution also contains some usability features for enterprise users. 
> The detail design document and CLI command will be attached in this jira. We 
> plan to use 10~12 subtasks to share each of the following features, and 
> document the detail implement in the subtasks: 
> * *Full Backup* : provide local and remote back/restore for a list of tables
> * *offline-WALPlayer* to convert HLog to HFiles offline (for incremental 
> backup)
> * *distributed* Logroll and distributed flush 
> * Backup *Manifest* and history
> * *Incremental* backup: to build on top of full backup as daily/weekly backup 
> * *Convert*  incremental backup WAL files into hfiles
> * *Merge* several backup images into one(like merge weekly into monthly)
> * *add and remove* table to and from Backup image
> * *Cancel* a backup process
> * backup progress *status*
> * full backup based on *existing snapshot*
> *-------------------------------------------------------------------------------------------------------------*
> *Below is the original description, to keep here as the history for the 
> design and discussion back in 2013*
> There have been attempts in the past to come up with a viable HBase 
> backup/restore solution (e.g., HBASE-4618).  Recently, there are many 
> advancements and new features in HBase, for example, FileLink, Snapshot, and 
> Distributed Barrier Procedure. This is a proposal for a backup/restore 
> solution that utilizes these new features to achieve better performance and 
> consistency. 
>  
> A common practice of backup and restore in database is to first take full 
> baseline backup, and then periodically take incremental backup that capture 
> the changes since the full baseline backup. HBase cluster can store massive 
> amount data.  Combination of full backups with incremental backups has 
> tremendous benefit for HBase as well.  The following is a typical scenario 
> for full and incremental backup.
> # The user takes a full backup of a table or a set of tables in HBase. 
> # The user schedules periodical incremental backups to capture the changes 
> from the full backup, or from last incremental backup.
> # The user needs to restore table data to a past point of time.
> # The full backup is restored to the table(s) or to different table name(s).  
> Then the incremental backups that are up to the desired point in time are 
> applied on top of the full backup. 
> We would support the following key features and capabilities.
> * Full backup uses HBase snapshot to capture HFiles.
> * Use HBase WALs to capture incremental changes, but we use bulk load of 
> HFiles for fast incremental restore.
> * Support single table or a set of tables, and column family level backup and 
> restore.
> * Restore to different table names.
> * Support adding additional tables or CF to backup set without interruption 
> of incremental backup schedule.
> * Support rollup/combining of incremental backups into longer period and 
> bigger incremental backups.
> * Unified command line interface for all the above.
> The solution will support HBase backup to FileSystem, either on the same 
> cluster or across clusters.  It has the flexibility to support backup to 
> other devices and servers in the future.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to