[
https://issues.apache.org/jira/browse/HBASE-14417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15452824#comment-15452824
]
Ted Yu commented on HBASE-14417:
--------------------------------
bq. Delete file ref from hbase:backup after backup completes
Did you mean after restore completes ?
What if the user wants to restore to a different destination afterward ?
The removal of hfile ref from hbase:backup can be coupled with the deletion of
backup(s).
Restoring incremental backup would ship hfiles along with WAL files to
destination.
Suppose given this sequence of events where f means full backup, b means bulk
load and i means incremental backup:
* f
* b1 (with hfile1)
* b2 (with hfile2)
* i1
* b3 (with hfile3)
* i2
The design of hfile ref in hbase:backup should make BackupHFileCleaner
operation efficient.
If we consolidate hfile ref from b1 and b2 into i1, b3 into i2,
BackupHFileCleaner needs to search backward (across all outstanding incremental
backups): i2 -> i1 -> f.
If we consolidate hfile ref from b1 and b2 by merging them and storing b1',
there is no need to search incremental backup(s).
> Incremental backup and bulk loading
> -----------------------------------
>
> Key: HBASE-14417
> URL: https://issues.apache.org/jira/browse/HBASE-14417
> Project: HBase
> Issue Type: New Feature
> Affects Versions: 2.0.0
> Reporter: Vladimir Rodionov
> Assignee: Vladimir Rodionov
> Priority: Critical
> Labels: backup
> Fix For: 2.0.0
>
>
> Currently, incremental backup is based on WAL files. Bulk data loading
> bypasses WALs for obvious reasons, breaking incremental backups. The only way
> to continue backups after bulk loading is to create new full backup of a
> table. This may not be feasible for customers who do bulk loading regularly
> (say, every day).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
