[jira] [Commented] (HBASE-26034) Add support to take parallel backups
[ https://issues.apache.org/jira/browse/HBASE-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797410#comment-17797410 ] Duo Zhang commented on HBASE-26034: --- [~bbeaudreault] Mind taking a look at this backup/restore improvement? Thanks. > Add support to take parallel backups > > > Key: HBASE-26034 > URL: https://issues.apache.org/jira/browse/HBASE-26034 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Major > Fix For: 3.0.0-beta-1 > > Attachments: existing_design.png, proposed_design.png > > > *Existing Design:* > !existing_design.png|width=632,height=1238! > *Proposed Changes:* > *!proposed_design.png|width=637,height=1300!* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-26034) Add support to take parallel backups
[ https://issues.apache.org/jira/browse/HBASE-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17558900#comment-17558900 ] Mallikarjun commented on HBASE-26034: - That part I could not solve for. Each backup would take the all WAL files of those regionservers resulting into more data than what is necessary. This problem exists only for Incremental backups since they depend on WAL files. > Add support to take parallel backups > > > Key: HBASE-26034 > URL: https://issues.apache.org/jira/browse/HBASE-26034 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Major > Fix For: 3.0.0-alpha-4 > > Attachments: existing_design.png, proposed_design.png > > > *Existing Design:* > !existing_design.png|width=632,height=1238! > *Proposed Changes:* > *!proposed_design.png|width=637,height=1300!* -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-26034) Add support to take parallel backups
[ https://issues.apache.org/jira/browse/HBASE-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17558776#comment-17558776 ] Duo Zhang commented on HBASE-26034: --- I wonder how do you deal with the WAL files? Different tables may have data in the same WAL file... Anyway, let me take a look at the code first. Thanks. > Add support to take parallel backups > > > Key: HBASE-26034 > URL: https://issues.apache.org/jira/browse/HBASE-26034 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Major > Fix For: 3.0.0-alpha-4 > > Attachments: existing_design.png, proposed_design.png > > > *Existing Design:* > !existing_design.png|width=632,height=1238! > *Proposed Changes:* > *!proposed_design.png|width=637,height=1300!* -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-26034) Add support to take parallel backups
[ https://issues.apache.org/jira/browse/HBASE-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17555619#comment-17555619 ] Mallikarjun commented on HBASE-26034: - One can take single backup at a time. Which takes exclusive system wide lock and resulting into following problems, esp if you have rsgroup enabled with multiple tenants wanting to take backups with different intervals. Following are the list of changes in the PR. # Remove exclusive system wide lock and replace with table level locks with checkAndPut. Repairs in case of abruptly dead jobs. This helps in taking parallel backup and configuring independent RPO # Take snapshot of backup table at the begining of backup and restore the snapshot at the end was unnecessary. This is removed as it serves no purpose. These are the 2 changes. Because of posibility of multiple backups happening at any point of time. Had to change BackupId to List while handling sessions. These are the changes in this PR. [~zhangduo] I have listed the changes above. If want any other information, please ask. P.S: Thank you very much for taking time to look into this. > Add support to take parallel backups > > > Key: HBASE-26034 > URL: https://issues.apache.org/jira/browse/HBASE-26034 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Major > Fix For: 3.0.0-alpha-4 > > Attachments: existing_design.png, proposed_design.png > > > *Existing Design:* > !existing_design.png|width=632,height=1238! > *Proposed Changes:* > *!proposed_design.png|width=637,height=1300!* -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-26034) Add support to take parallel backups
[ https://issues.apache.org/jira/browse/HBASE-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1782#comment-1782 ] Duo Zhang commented on HBASE-26034: --- Do we have a simple design doc to describe what are the key changes here? Looking at the two pictures, I do not know why and how to remove some steps... > Add support to take parallel backups > > > Key: HBASE-26034 > URL: https://issues.apache.org/jira/browse/HBASE-26034 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Major > Fix For: 3.0.0-alpha-4 > > Attachments: existing_design.png, proposed_design.png > > > *Existing Design:* > !existing_design.png|width=632,height=1238! > *Proposed Changes:* > *!proposed_design.png|width=637,height=1300!* -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-26034) Add support to take parallel backups
[ https://issues.apache.org/jira/browse/HBASE-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452404#comment-17452404 ] Mallikarjun commented on HBASE-26034: - [~zhangduo] [~stack] [~anoop.hbase] Kindly help me on this review when you can find some free time. > Add support to take parallel backups > > > Key: HBASE-26034 > URL: https://issues.apache.org/jira/browse/HBASE-26034 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Major > Fix For: 3.0.0-alpha-2 > > Attachments: existing_design.png, proposed_design.png > > > *Existing Design:* > !existing_design.png|width=632,height=1238! > *Proposed Changes:* > *!proposed_design.png|width=637,height=1300!* -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26034) Add support to take parallel backups
[ https://issues.apache.org/jira/browse/HBASE-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430482#comment-17430482 ] Mallikarjun commented on HBASE-26034: - [~zhangduo] [~stack] [~anoop.hbase] Patch for this is ready for review. Please someone have a look when you have some time. [|https://issues.apache.org/jira/secure/AddComment!default.jspa?id=13404591] > Add support to take parallel backups > > > Key: HBASE-26034 > URL: https://issues.apache.org/jira/browse/HBASE-26034 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Major > Fix For: 3.0.0-alpha-2 > > Attachments: existing_design.png, proposed_design.png > > > *Existing Design:* > !existing_design.png|width=632,height=1238! > *Proposed Changes:* > *!proposed_design.png|width=637,height=1300!* -- This message was sent by Atlassian Jira (v8.3.4#803005)
