[jira] [Comment Edited] (HBASE-17825) Backup: further optimizations
[
https://issues.apache.org/jira/browse/HBASE-17825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374850#comment-16374850
]
Vladimir Rodionov edited comment on HBASE-17825 at 2/23/18 7:04 PM:
{quote}
How does this change output
{quote}
It does not change output, but it changes output directory structure. It used
to be flat, now it adds subdirectories for every table:
If we convert WALs for tables T1, T2, T3 the tool will create separate
directories for each table:
T1/
T2/
T3/
and files for each table go to this table's subdirectory
Forgot to mention:
This change allows to run WALPlayer for multiple tables simultaneously.
was (Author: vrodionov):
{quote}
How does this change output
{quote}
It does not change output, but it changes output directory structure. It used
to be flat, now it adds subdirectories for every table:
If we convert WALs for tables T1, T2, T3 the tool will create separate
directories for each table:
T1/
T2/
T3/
and files for each table go to this table's subdirectory
> Backup: further optimizations
> -
>
> Key: HBASE-17825
> URL: https://issues.apache.org/jira/browse/HBASE-17825
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
> Labels: backup
> Fix For: 3.0.0
>
> Attachments: HBASE-17825-v1.patch, HBASE-17825-v2.patch,
> HBASE-17825-v3.patch, HBASE-17825-v4.patch
>
>
> Some phases of backup and restore can be optimized:
> # WALPlayer support for multiple tables
> # Run DistCp once per all tables during backup/restore
> The eventual goal:
> # 2 M/R jobs per backup/restore
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-17825) Backup: further optimizations
[
https://issues.apache.org/jira/browse/HBASE-17825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374850#comment-16374850
]
Vladimir Rodionov edited comment on HBASE-17825 at 2/23/18 7:03 PM:
{quote}
How does this change output
{quote}
It does not change output, but it changes output directory structure. It used
to be flat, now it adds subdirectories for every table:
If we convert WALs for tables T1, T2, T3 the tool will create separate
directories for each table:
T1/
T2/
T3/
and files for each table go to this table's subdirectory
was (Author: vrodionov):
{quote}
How does this change output
{quote}
It does not change output, but it changes output directory structure. It used
to be flat, now it adds subdirectories for every table:
If we convert WALs for tables T1, T2, T3 the tool will create separate
directories for each table:
T1/
T2/
T3/
> Backup: further optimizations
> -
>
> Key: HBASE-17825
> URL: https://issues.apache.org/jira/browse/HBASE-17825
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
> Labels: backup
> Fix For: 3.0.0
>
> Attachments: HBASE-17825-v1.patch, HBASE-17825-v2.patch,
> HBASE-17825-v3.patch, HBASE-17825-v4.patch
>
>
> Some phases of backup and restore can be optimized:
> # WALPlayer support for multiple tables
> # Run DistCp once per all tables during backup/restore
> The eventual goal:
> # 2 M/R jobs per backup/restore
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-17825) Backup: further optimizations
[
https://issues.apache.org/jira/browse/HBASE-17825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373909#comment-16373909
]
stack edited comment on HBASE-17825 at 2/23/18 4:24 AM:
bq. What are our compatibility guarantees on WALPlayer?
@InterfaceAudience.Public
public class WALPlayer extends Configured implements Tool {
A recent fix ensured hbase-1 can play hbase-2 WALs and vice-versa.
How does this change output?
was (Author: stack):
bq. What are our compatibility guarantees on WALPlayer?
@InterfaceAudience.Public
public class WALPlayer extends Configured implements Tool {
A recent fix ensured hbase-1 can play hbase-2 WALs and vice-versa.
How does this change
> Backup: further optimizations
> -
>
> Key: HBASE-17825
> URL: https://issues.apache.org/jira/browse/HBASE-17825
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
> Labels: backup
> Fix For: 3.0.0
>
> Attachments: HBASE-17825-v1.patch, HBASE-17825-v2.patch,
> HBASE-17825-v3.patch, HBASE-17825-v4.patch
>
>
> Some phases of backup and restore can be optimized:
> # WALPlayer support for multiple tables
> # Run DistCp once per all tables during backup/restore
> The eventual goal:
> # 2 M/R jobs per backup/restore
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-17825) Backup: further optimizations
[
https://issues.apache.org/jira/browse/HBASE-17825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359031#comment-16359031
]
Vladimir Rodionov edited comment on HBASE-17825 at 2/9/18 10:59 PM:
{quote}
When false is returned, some files would be deleted while some are left. Is
this desirable?
{quote}
This needs to be added to FT Jira. We need to restore not only meta in case of
failure but physical data as well.
That is the separate issue: HBASE-19969
was (Author: vrodionov):
{quote}
When false is returned, some files would be deleted while some are left. Is
this desirable?
{quote}
This needs to be added to FT Jira. We need to restore not only meta in case of
failure but physical data as well.
> Backup: further optimizations
> -
>
> Key: HBASE-17825
> URL: https://issues.apache.org/jira/browse/HBASE-17825
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
> Labels: backup
> Fix For: 3.0.0
>
> Attachments: HBASE-17825-v1.patch
>
>
> Some phases of backup and restore can be optimized:
> # WALPlayer support for multiple tables
> # Run DistCp once per all tables during backup/restore
> The eventual goal:
> # 2 M/R jobs per backup/restore
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-17825) Backup: further optimizations
[ https://issues.apache.org/jira/browse/HBASE-17825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252073#comment-16252073 ] Vladimir Rodionov edited comment on HBASE-17825 at 11/14/17 8:00 PM: - Why, it is almost done, [~elserj]. Taking into account that 2.0 GA is at least 9 months away, I am pretty sure, that this will make it into 2.0. Phase 4 is post 2.0 anyway. was (Author: vrodionov): Why, it is almost done, [~elserj]. Phase 4 is post 2.0 anyway. > Backup: further optimizations > - > > Key: HBASE-17825 > URL: https://issues.apache.org/jira/browse/HBASE-17825 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: backup > Fix For: 2.1.0 > > > Some phases of backup and restore can be optimized: > # WALPlayer support for multiple tables > # Run DistCp once per all tables during backup/restore > The eventual goal: > # 2 M/R jobs per backup/restore -- This message was sent by Atlassian JIRA (v6.4.14#64029)
