[jira] [Commented] (HBASE-16727) Backup refactoring: move MR dependencies from HMaster
[ https://issues.apache.org/jira/browse/HBASE-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546661#comment-15546661 ] Ted Yu commented on HBASE-16727: [~mbertozzi]: Are you good with the latest patch ? > Backup refactoring: move MR dependencies from HMaster > - > > Key: HBASE-16727 > URL: https://issues.apache.org/jira/browse/HBASE-16727 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-16727-v1.patch, HBASE-16727-v2.patch > > > * No MR jobs in HMaster > * No proc2 implementation > * client-driven backup-restore > * basic security: only super user is allowed to run backup/restore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16727) Backup refactoring: move MR dependencies from HMaster
[ https://issues.apache.org/jira/browse/HBASE-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545526#comment-15545526 ] Ted Yu commented on HBASE-16727: Mind updating review board ? > Backup refactoring: move MR dependencies from HMaster > - > > Key: HBASE-16727 > URL: https://issues.apache.org/jira/browse/HBASE-16727 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-16727-v1.patch, HBASE-16727-v2.patch > > > * No MR jobs in HMaster > * No proc2 implementation > * client-driven backup-restore > * basic security: only super user is allowed to run backup/restore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16727) Backup refactoring: move MR dependencies from HMaster
[ https://issues.apache.org/jira/browse/HBASE-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543561#comment-15543561 ] Vladimir Rodionov commented on HBASE-16727: --- I will double check, but in my environment it seems true. W.o patch no WARN in mapred. Digging into this. > Backup refactoring: move MR dependencies from HMaster > - > > Key: HBASE-16727 > URL: https://issues.apache.org/jira/browse/HBASE-16727 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-16727-v1.patch > > > * No MR jobs in HMaster > * No proc2 implementation > * client-driven backup-restore > * basic security: only super user is allowed to run backup/restore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16727) Backup refactoring: move MR dependencies from HMaster
[
https://issues.apache.org/jira/browse/HBASE-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543499#comment-15543499
]
Ted Yu commented on HBASE-16727:
Looking back at a TestIncrementalBackup output from two weeks ago:
{code}
2016-08-16 10:14:09,805 WARN [ResourceManager Event Processor]
capacity.LeafQueue(632): maximum-am-resource-percent is insufficient to start a
single application in queue, it is likely set too low. skipping enforcement to
allow at least one application to start
2016-08-16 10:14:09,805 WARN [ResourceManager Event Processor]
capacity.LeafQueue(653): maximum-am-resource-percent is insufficient to start a
single application in queue foruser, it is likely set too low. skipping
enforcement to allow at least one application to start
{code}
> Backup refactoring: move MR dependencies from HMaster
> -
>
> Key: HBASE-16727
> URL: https://issues.apache.org/jira/browse/HBASE-16727
> Project: HBase
> Issue Type: Task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-16727-v1.patch
>
>
> * No MR jobs in HMaster
> * No proc2 implementation
> * client-driven backup-restore
> * basic security: only super user is allowed to run backup/restore
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16727) Backup refactoring: move MR dependencies from HMaster
[
https://issues.apache.org/jira/browse/HBASE-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543492#comment-15543492
]
Vladimir Rodionov commented on HBASE-16727:
---
With the patch during ExportSnapshot phase the following WARNs messages appear
in a log file:
{code}
2016-10-03 14:15:08,801 WARN [ResourceManager Event Processor]
capacity.LeafQueue(610): maximum-am-resource-percent is insufficient to start a
single application in queue, it is likely set too low. skipping enforcement to
allow at least one application to start
2016-10-03 14:15:08,801 WARN [ResourceManager Event Processor]
capacity.LeafQueue(631): maximum-am-resource-percent is insufficient to start a
single application in queue for user, it is likely set too low. skipping
enforcement to allow at least one application to start
2016-10-03 14:15:18,095 WARN [ContainersLauncher #1]
nodemanager.DefaultContainerExecutor(223): Exit code from container
container_1475529282221_0001_01_02 is : 143
{code}
No messages when we run ExportSnapshot phase w/o patch.
> Backup refactoring: move MR dependencies from HMaster
> -
>
> Key: HBASE-16727
> URL: https://issues.apache.org/jira/browse/HBASE-16727
> Project: HBase
> Issue Type: Task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-16727-v1.patch
>
>
> * No MR jobs in HMaster
> * No proc2 implementation
> * client-driven backup-restore
> * basic security: only super user is allowed to run backup/restore
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16727) Backup refactoring: move MR dependencies from HMaster
[ https://issues.apache.org/jira/browse/HBASE-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534771#comment-15534771 ] Vladimir Rodionov commented on HBASE-16727: --- Review board https://reviews.apache.org/r/52413/ Unit test now run 70% slower (47 vs 28 min). No explanation yet. > Backup refactoring: move MR dependencies from HMaster > - > > Key: HBASE-16727 > URL: https://issues.apache.org/jira/browse/HBASE-16727 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-16727-v1.patch > > > * No MR jobs in HMaster > * No proc2 implementation > * client-driven backup-restore > * basic security: only super user is allowed to run backup/restore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16727) Backup refactoring: move MR dependencies from HMaster
[ https://issues.apache.org/jira/browse/HBASE-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534763#comment-15534763 ] Vladimir Rodionov commented on HBASE-16727: --- We have separate JIRA for that, [~tedyu]. For fault tolerance. > Backup refactoring: move MR dependencies from HMaster > - > > Key: HBASE-16727 > URL: https://issues.apache.org/jira/browse/HBASE-16727 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-16727-v1.patch > > > * No MR jobs in HMaster > * No proc2 implementation > * client-driven backup-restore > * basic security: only super user is allowed to run backup/restore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16727) Backup refactoring: move MR dependencies from HMaster
[ https://issues.apache.org/jira/browse/HBASE-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534757#comment-15534757 ] Ted Yu commented on HBASE-16727: I looked at FullTableBackupClient.java Suppose there was some interruption in the middle of execution, how does the next invocation recover ? Or, user always starts with new backup request ? Thanks > Backup refactoring: move MR dependencies from HMaster > - > > Key: HBASE-16727 > URL: https://issues.apache.org/jira/browse/HBASE-16727 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-16727-v1.patch > > > * No MR jobs in HMaster > * No proc2 implementation > * client-driven backup-restore > * basic security: only super user is allowed to run backup/restore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16727) Backup refactoring: move MR dependencies from HMaster
[ https://issues.apache.org/jira/browse/HBASE-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534676#comment-15534676 ] Ted Yu commented on HBASE-16727: Any chance of putting it on review board ? > Backup refactoring: move MR dependencies from HMaster > - > > Key: HBASE-16727 > URL: https://issues.apache.org/jira/browse/HBASE-16727 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-16727-v1.patch > > > * No MR jobs in HMaster > * No proc2 implementation > * client-driven backup-restore > * basic security: only super user is allowed to run backup/restore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16727) Backup refactoring: move MR dependencies from HMaster
[ https://issues.apache.org/jira/browse/HBASE-16727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531237#comment-15531237 ] stack commented on HBASE-16727: --- +1 from me. Thanks Vladimir. > Backup refactoring: move MR dependencies from HMaster > - > > Key: HBASE-16727 > URL: https://issues.apache.org/jira/browse/HBASE-16727 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > > * No MR jobs in HMaster > * No proc2 implementation > * client-driven backup-restore > * basic security: only super user is allowed to run backup/restore -- This message was sent by Atlassian JIRA (v6.3.4#6332)
