[jira] [Comment Edited] (HBASE-14135) HBase Backup/Restore Phase 3: Merge backup images
[
https://issues.apache.org/jira/browse/HBASE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122777#comment-16122777
]
Josh Elser edited comment on HBASE-14135 at 8/11/17 3:33 AM:
-
Re-attaching v10 since qa -since- seems to have not been accurate.
was (Author: elserj):
Re-attaching v10 since qa since to have not been accurate.
> HBase Backup/Restore Phase 3: Merge backup images
> -
>
> Key: HBASE-14135
> URL: https://issues.apache.org/jira/browse/HBASE-14135
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
> Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-14135-v10.patch, HBASE-14135-v3.patch,
> HBASE-14135-v5.patch, HBASE-14135-v6.patch, HBASE-14135-v7.patch,
> HBASE-14135-v8.patch, HBASE-14135-v9.patch
>
>
> User can merge incremental backup images into single incremental backup image.
> # Merge supports only incremental images
> # Merge supports only images for the same backup destinations
> Command:
> {code}
> hbase backup merge image1,image2,..imageK
> {code}
> Example:
> {code}
> hbase backup merge backup_143126764557,backup_143126764456
> {code}
> When operation is complete, only the most recent backup image will be kept
> (in above example - backup_143126764557) as a merged backup image, all other
> images will be deleted from both: file system and backup system tables,
> corresponding backup manifest for the merged backup image will be updated to
> remove dependencies from deleted images. Merged backup image will contains
> all the data from original image and from deleted images.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-14135) HBase Backup/Restore Phase 3: Merge backup images
[
https://issues.apache.org/jira/browse/HBASE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091989#comment-16091989
]
Vladimir Rodionov edited comment on HBASE-14135 at 7/18/17 6:52 PM:
Patch v7 add some UT with merge failures
cc: [~te...@apache.org]
was (Author: vrodionov):
Patch v7 add some UT with merge failures
> HBase Backup/Restore Phase 3: Merge backup images
> -
>
> Key: HBASE-14135
> URL: https://issues.apache.org/jira/browse/HBASE-14135
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-14135-v3.patch, HBASE-14135-v5.patch,
> HBASE-14135-v6.patch, HBASE-14135-v7.patch
>
>
> User can merge incremental backup images into single incremental backup image.
> # Merge supports only incremental images
> # Merge supports only images for the same backup destinations
> Command:
> {code}
> hbase backup merge image1,image2,..imageK
> {code}
> Example:
> {code}
> hbase backup merge backup_143126764557,backup_143126764456
> {code}
> When operation is complete, only the most recent backup image will be kept
> (in above example - backup_143126764557) as a merged backup image, all other
> images will be deleted from both: file system and backup system tables,
> corresponding backup manifest for the merged backup image will be updated to
> remove dependencies from deleted images. Merged backup image will contains
> all the data from original image and from deleted images.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-14135) HBase Backup/Restore Phase 3: Merge backup images
[ https://issues.apache.org/jira/browse/HBASE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510810#comment-15510810 ] Vladimir Rodionov edited comment on HBASE-14135 at 9/21/16 6:47 PM: Two issues with incremental backup implementation: # Storage usage. WAL files are significantly larger than hfiles, especially when compression is enabled in hfiles # Restore operation time. The most significant is the first one. We can that storage reduction factor will be something between 3-5 once convert (merge) is implemented. Running conversion (merge) on a remote destination (cluster) is not feasible in many cases, so merge will require reading/writing data back and forth between source and destination. I will postpone this feature until filtering at the source during incremental backup is done (HBASE-14141). was (Author: vrodionov): Two issues with incremental backup implementation: # Storage usage. WAL files are significantly larger than hfiles, especially when compression is enabled in hfiles # Restore operation time. The most significant is the first one. We can that storage reduction factor will be something between 3-5 once convert (merge) is implemented. Running conversion (merge) on a remote destination (cluster) is not feasible in many cases, so merge will require reading/writing data back and forth between source and destination. I will postpone this feature until filtering at the source during incremental backup is done. > HBase Backup/Restore Phase 3: Merge backup images > - > > Key: HBASE-14135 > URL: https://issues.apache.org/jira/browse/HBASE-14135 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
