subject:"\[jira\] \[Commented\] \(HBASE\-15433\) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception"

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205607#comment-15205607
 ] 

Jianwei Cui commented on HBASE-15433:
-

Thanks for the review, Ted and Ashish.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205441#comment-15205441
 ] 

Hudson commented on HBASE-15433:


FAILURE: Integrated in HBase-1.1-JDK8 #1772 (See 
[https://builds.apache.org/job/HBase-1.1-JDK8/1772/])
HBASE-15433 SnapshotManager#restoreSnapshot not update table and region (tedyu: 
rev 40e19e80b2363c899f9b8cba98f3c2326d6329d1)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceStateManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java


> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204933#comment-15204933
 ] 

Hudson commented on HBASE-15433:


FAILURE: Integrated in HBase-1.2 #583 (See 
[https://builds.apache.org/job/HBase-1.2/583/])
HBASE-15433 SnapshotManager#restoreSnapshot not update table and region (tedyu: 
rev 7c8d72b0414f0eabd431b5873e5eefe5910085c2)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceStateManager.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceAuditor.java


> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204920#comment-15204920
 ] 

Hudson commented on HBASE-15433:


SUCCESS: Integrated in HBase-1.3 #614 (See 
[https://builds.apache.org/job/HBase-1.3/614/])
HBASE-15433 SnapshotManager#restoreSnapshot not update table and region (tedyu: 
rev 7affa5abce2a26db207fb8723f34c89c54799ea1)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceStateManager.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java


> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204561#comment-15204561
 ] 

Hudson commented on HBASE-15433:


SUCCESS: Integrated in HBase-1.1-JDK7 #1686 (See 
[https://builds.apache.org/job/HBase-1.1-JDK7/1686/])
HBASE-15433 SnapshotManager#restoreSnapshot not update table and region (tedyu: 
rev 40e19e80b2363c899f9b8cba98f3c2326d6329d1)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceStateManager.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java


> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204502#comment-15204502
 ] 

Hudson commented on HBASE-15433:


FAILURE: Integrated in HBase-Trunk_matrix #794 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/794/])
HBASE-15433 SnapshotManager#restoreSnapshot not update table and region (tedyu: 
rev f1d453599a90b8a1075082e2dc7b7676416f)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceStateManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceAuditor.java


> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204483#comment-15204483
 ] 

Hudson commented on HBASE-15433:


SUCCESS: Integrated in HBase-1.2-IT #468 (See 
[https://builds.apache.org/job/HBase-1.2-IT/468/])
HBASE-15433 SnapshotManager#restoreSnapshot not update table and region (tedyu: 
rev 7c8d72b0414f0eabd431b5873e5eefe5910085c2)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceStateManager.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java


> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204420#comment-15204420
 ] 

Hudson commented on HBASE-15433:


SUCCESS: Integrated in HBase-1.3-IT #571 (See 
[https://builds.apache.org/job/HBase-1.3-IT/571/])
HBASE-15433 SnapshotManager#restoreSnapshot not update table and region (tedyu: 
rev ee4bf43577e8ca3f1554dedebc1815a29b49cca2)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceStateManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java


> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204214#comment-15204214
 ] 

Ted Yu commented on HBASE-15433:


Thanks for the patch, Jianwei.

Thanks for the review, Ashish.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204198#comment-15204198
 ] 

Hudson commented on HBASE-15433:


FAILURE: Integrated in HBase-1.4 #39 (See 
[https://builds.apache.org/job/HBase-1.4/39/])
HBASE-15433 SnapshotManager#restoreSnapshot not update table and region (tedyu: 
rev ee4bf43577e8ca3f1554dedebc1815a29b49cca2)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceAuditor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceStateManager.java


> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204012#comment-15204012
 ] 

Ted Yu commented on HBASE-15433:


Integrated to master and branch-1.

Once Jenkins build comes back clean, will integrate to other branches.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203928#comment-15203928
 ] 

Hadoop QA commented on HBASE-15433:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
52s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
19s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
15s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
54s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 29s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 29s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
11m 39s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 3s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 84m 29s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
16s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 108m 53s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12794476/HBASE-15433-branch-1-v1.patch
 |
| JIRA Issue | HBASE-15433 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | branch-1 / a307132 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions |  /home/jenkins/tools/java/jdk1.8.0:1.8.0 
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/1

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203848#comment-15203848
 ] 

Jianwei Cui commented on HBASE-15433:
-

HBASE-15433-branch-1-v1.patch could also be applied to branch-1.1/1.2/1.3.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-21 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203833#comment-15203833
 ] 

Jianwei Cui commented on HBASE-15433:
-

Sorry to reply late and thanks for your review [~yuzhih...@gmail.com], have 
attached patch for branch-1 and will wait for the tests result. Thanks.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15433-branch-1-v1.patch, 
> HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-19 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201414#comment-15201414
 ] 

Ted Yu commented on HBASE-15433:


Jianwei:
I looked at the scenario (region counts of 8 and 10 in snapshots) above where 
retrieving from region locator may produce inaccurate end result.

We can go with path v4.

Since branch-1 is among the Fix versions, can you attach patch for branch-1 ? 
branch-1 tests are more stable, hopefully we can get higher confidence.

Thanks

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.4.0, 1.1.5
>
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-19 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201155#comment-15201155
 ] 

Jianwei Cui commented on HBASE-15433:
-

Run the failed tests locally and passed, seems the test fail is unrelated to 
this patch. Could you please take a look at the patch v4? 
[~yuzhih...@gmail.com] [~mbertozzi]. Thanks.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Assignee: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.4.0, 1.1.5
>
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch, HBASE-15433-v4.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-16 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15197071#comment-15197071
 ] 

Hadoop QA commented on HBASE-15433:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s 
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m 
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 7s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s 
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 5s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m 
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
30m 42s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 8s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 32s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 102m 28s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_79. 
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
40s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 253m 23s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0 Timed out junit tests | 
org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure |
|   | org.apache.hadoop.hbase.master.TestGetLastFlushedSequenceId |
|   | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
|   | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
|   | org.apache.hadoop.hbase.master.TestMasterFailover |
|   | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
|   | org.apache.hadoop.hbase.wal.TestWALFiltering |
|   | org.apache.hadoop.hbase.coprocessor.TestRegionServerObserver |
|   | org.apache.hadoop.hbase.regionserver.TestScannerHeartbeatMessages |
|   | org.apache.hadoop.hbase.ma

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195474#comment-15195474
 ] 

Hadoop QA commented on HBASE-15433:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s 
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m 
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 3s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s 
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 56s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 56s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 5m 21s 
{color} | {color:red} hbase-server: patch generated 1 new + 24 unchanged - 0 
fixed = 25 total (was 24) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
19s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
35m 35s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 132m 21s 
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} 
|
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 37m 28s {color} 
| {color:red} hbase-server in the patch failed with JDK v1.7.0_79. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
12s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 234m 37s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0 Timed out junit tests | 
org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
|   | org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot |
|   | org.apache.hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient |
|   | org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient |
| JDK v1.7.0_79 Failed junit tests | hadoop.hbase.ipc.TestSimpleRpcScheduler |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12793507/HBASE-15433-v3.patch |
| JIRA Issue | HBASE-15433 |
| Optional Tests |  asflicense

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-15 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195033#comment-15195033
 ] 

Jianwei Cui commented on HBASE-15433:
-

{quote}
Instead of getting table region count from quota cache we can get it from 
RegionLocator which will solve the corner case you described.
{quote}
This may make other corner cases fail if I am not wrong. For example, the table 
has 5 regions, clientA is trying to restore the table to snapshot with 8 
regions, while clientB is trying to restore the snapshot with 10 regions, then:
1. clientA firstly invokes {{checkAndUpdateNamespaceRegionQuota}} before 
{{restoreSnapshot}}, the {{tableRegionCount}} is 5 for clientA and it updates 
the region count of the table to 8
2. Before clientA invokes {{restoreSnapshot}}, clientB invokes 
{{checkAndUpdateNamespaceRegionQuota}} before {{restoreSnapshot}}, the 
{{tableRegionCount}} is also 5(when using RegionLocator) for clientB and it 
updates the region count of the table to 10
3. clientA successfully restored its snapshot, so that the actual region count 
is 8
4. clientB encountered IOE in {{restoreSnapshot}} and will reset the region 
count to 5 in IOE catch clause. However, the region count should be 8 because 
clientA succeeded.
I think it is not easy to resolve the concurrent issues in {{SnapshotManager}} 
without lock, we may wait for RestoreSnapshotHandler rewritten by procedure v2 
and move quota updating in RestoreSnapshotHandler?

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.4.0, 1.1.5
>
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-15 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194977#comment-15194977
 ] 

Ashish Singhi commented on HBASE-15433:
---

Instead of getting table region count from quota cache we can get it from 
RegionLocator which will solve the corner case you described. Otherwise LGTM.

[~mbertozzi] [~tedyu], can you also take a look.
Thanks.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-15 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194966#comment-15194966
 ] 

Jianwei Cui commented on HBASE-15433:
-

Upload patch v3 according to comments. There has been a unit test named as 
TestNamespaceAuditor#testRestoreSnapshotQuotaExceed, the new patch checks the 
exception cause type and region count in this unit test. 
{{TestNamespaceAuditor}} and {{TestRestoreFlushSnapshotFromClient}} passed 
locally. [~ashish singhi], could you please have a look at v3? Thanks.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch, HBASE-15433-v3.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-15 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194849#comment-15194849
 ] 

Ashish Singhi commented on HBASE-15433:
---

Sure. I am ok with it.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-15 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194845#comment-15194845
 ] 

Jianwei Cui commented on HBASE-15433:
-

Reasonable for this case IMO. It seems there are other issues when concurrently 
restoring snapshots for the same table. We need to keep the 
{{checkAndUpdateNamespaceRegionQuota}} the same order before and after 
{{restoreSnapshot}} among concurrent restore requests. For example, beofre 
{{restoreSnapshot}}, if clientA invoked {{checkAndUpdateNamespaceRegionQuota}} 
ahead of clientB, then after {{restoreSnapshot}}, we need to make sure clientA 
also invoked {{checkAndUpdateNamespaceRegionQuota}} ahead of clientB?
In the document of 
[HBASE-12439|https://issues.apache.org/jira/browse/HBASE-12439], it seems the 
CloneSnapshotHandler/RestoreSnapshotHandler will be rewritten by procedure v2? 
After that, we can keep the quota updating sync with 
CloneSnapshot/RestoreSnapshot steps and rollbacks. Currently, without steps and 
rollbacks, RestoreSnapshotHandler may not update the quota information 
correctly. Therefore, I think we can still keep the quota updating in 
{{SnapshotManager}} before procedure v2 rewritten? For concurrent request 
issues, we can add some comments in the code to explain the problem?

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-14 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194785#comment-15194785
 ] 

Ashish Singhi commented on HBASE-15433:
---

Your idea sounds good to me.

{quote}
4. clientA firstly reset the region count to 5, and then clientB reset the 
region count to 8, so the final region count for the table is 8 in such case, 
but it should be 5 because both operations failed.
{quote}
Just a another thought in my mind, instead of getting {{tableRegionCount}} 
value from quota cache can we get it from RegionLocator by this we can solve 
these problem. WDYT ?

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-14 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194691#comment-15194691
 ] 

Jianwei Cui commented on HBASE-15433:
-

Thanks for your comment.
{quote}
Not required I think, because we are having enough quota for this table in the 
cache before restoring the snapshot and after restoring snapshot we are only 
decrementing it, so it will work.
{quote}
There may be corner case if I am not wrong. For example, if the table has 5 
regions, clientA is trying to restore the table to snapshot with 8 regions, 
while clientB is trying to restore the snapshot with 10 regions, then:
1. clientA firstly invokes {{checkAndUpdateNamespaceRegionQuota}} before 
{{restoreSnapshot}}, the {{tableRegionCount}} is 5 for clientA and it updates 
the region count of the table to 8
2. Before clientA invokes {{restoreSnapshot}}, clientB invokes 
{{checkAndUpdateNamespaceRegionQuota}} before {{restoreSnapshot}}, the 
{{tableRegionCount}} is 8 for clientB and it updates the region count of the 
table to 10
3. Both clientA and clientB encountered IOE in {{restoreSnapshot}}, and the two 
clients are trying to reset the region count in IOE catch clause
4. clientA firstly reset the region count to 5, and then clientB reset the 
region count to 8, so the final region count for the table is 8 in such case, 
but it should be 5 because both operations failed.
It seems not easy to update the quota information correctly without lock if 
there are concurrent restoreSnapshot requests IMO. Maybe, it is more easy to do 
such work in {{RestoreSnapshotHandler}} with table lock held(like 
{{CreateTableProcedure}})?
1. In {{RestoreSnapshotHandler}}, overwrite {{prepareWithTableLock}} method 
with {{checkAndUpdateNamespaceRegionQuota}} if {{snapshotRegionCount}} is 
larger than {{tableRegionCount}}. If {{checkAndUpdateNamespaceRegionQuota}} 
fails here, we do not need to reset the region count and {{SnapshotManager}} 
will throw exception directly.
2. In {{RestoreSnapshotHandler#completed}}, if exception received and 
{{tableRegionCount < SnapshotRegionCount}}, we reset region count to 
{{tableRegionCount}}, if no exception received and {{tableRegionCount > 
snapshotRegionCount}}, we set region count to {{snapshotRegionCount}}. 
What's your opinion about this issue? [~ashish singhi]

{quote}
We can also include quota has exceeded in the error message
{quote}
Yes, will polish the message and update the patch.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-14 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193177#comment-15193177
 ] 

Ashish Singhi commented on HBASE-15433:
---

I was not saying about concurrent restoreSnapshot requests, any other operation 
on the namespace also can update the quota of the namespace.

bq. checkAndUpdateNamespaceRegionQuota after restoreSnapshot will be executed 
only when tableRegionCount > snapshotRegionCount satisfied
OK. That means we have reserved enough quota before hand.

bq. if there are concurrent restoreSnapshot requests for the same table, it 
will cause problem, and we may need lock to make sure the quota information is 
updated correctly, or we can move the quota check and update logic in the 
RestoreSnapshotHandler after table lock is held?
Not required I think, because we are having enough quota for this table in the 
cache before restoring the snapshot and after restoring snapshot we are only 
decrementing it, so it will work.

{code}
 } catch (QuotaExceededException e) {
LOG.error("Exception occurred while restoring the snapshot " + 
snapshot.getName()
  + " as table " + tableName.getNameAsString(), e);
// If QEE is thrown before restoreSnapshot, quota information is not 
updated, and we
// should throw the exception directly. If QEE is thrown after 
restoreSnapshot, there
// must be unexpected reasons, we also throw the exception directly
throw e;
{code}
We can also include quota has exceeded in the error message.

Thanks.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-14 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193148#comment-15193148
 ] 

Jianwei Cui commented on HBASE-15433:
-

The table must be disabled during restoreSnapshot, so that the 
{{tableRegionCount}} won't change. Assume there won't be concurrent 
restoreSnapshot requests for the same table, the 
{{checkAndUpdateNamespaceRegionQuota}} after {{restoreSnapshot}} will be 
executed only when {{tableRegionCount > snapshotRegionCount}} satisfied, this 
means we have preserved enough region count for the 
{{checkAndUpdateNamespaceRegionQuota}} from the namespace quota. Therefore, 
other thread operations won't make the {{checkAndUpdateNamespaceRegionQuota}} 
fail if they operating on different tables? However, if there are concurrent 
restoreSnapshot requests for the same table, it will cause problem, and we may 
need lock to make sure the quota information is updated correctly, or we can 
move the quota check and update logic in the {{RestoreSnapshotHandler}} after 
table lock is held?

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-14 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193091#comment-15193091
 ] 

Ashish Singhi commented on HBASE-15433:
---

bq. Here, the checkAndUpdateNamespaceRegionQuota should succeed because it will 
reduce the region count for the table?
Not necessary, suppose through some other thread operations there is a increase 
in the namespace regions count immediately after the snapshot is restored and 
there is not enough region quota available now.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-14 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193038#comment-15193038
 ] 

Jianwei Cui commented on HBASE-15433:
-

{quote}
When QEE is thrown we will still end up in updating the region quota which is 
not really required, may be we can avoid that.
{quote}
Yes, we should catch QEE firstly and not update the quota information in such 
situation as you suggested above.
{quote}
Also suggest to rename currentRegionCount to tableRegionCount and 
updatedRegionCount to snapshotRegionCount for better understanding. Please add 
more comments like why are we doing this way.
{quote}
Good suggestions, will update the patch.

{quote}
If this throws exception then there will be another issue, because now the 
snapshot has been successfully restored but in the catch clause we are updating 
the table region count in namespace quota.
{quote}
Good find. Here, the {{checkAndUpdateNamespaceRegionQuota}} should succeed 
because it will reduce the region count for the table? However, if the 
{{checkAndUpdateNamespaceRegionQuota}} throws exception, there must be some 
unexpected reasons, and call {{checkAndUpdateNamespaceRegionQuota}} in catch 
clause may also fail. We can log an error message in QEE catch clause and throw 
it directly? And the code here can be updated as:
{code}
  int tableRegionCount = -1;
  try {
// Table already exist. Check and update the region quota for this 
table namespace
// Table is disabled, table region count won't change during 
restoreSnapshot
tableRegionCount = getRegionCountOfTable(tableName);
int snapshotRegionCount = manifest.getRegionManifestsMap().size();

// Update region count before restoreSnapshot if snapshotRegionCount is 
larger. If we
// updated the region count to a smaller value before retoreSnapshot 
and the retoreSnapshot
// fails, we may fail to reset the region count to its original value 
if the namespace
// region count quota is consumed by other tables during the 
restoreSnapshot, such as
// region split or table create under the same namespace.
if (tableRegionCount > 0 && tableRegionCount < snapshotRegionCount) {
  checkAndUpdateNamespaceRegionQuota(snapshotRegionCount, tableName);
}

restoreSnapshot(snapshot, snapshotTableDesc);

// Update the region count after restoreSnapshot succeeded if 
snapshotRegionCount is
// smaller. This step should not fail because it will reduce the region 
count for table
if (tableRegionCount > 0 && tableRegionCount > snapshotRegionCount) {
  checkAndUpdateNamespaceRegionQuota(snapshotRegionCount, tableName);
}
  } catch (QuotaExceededException e) {
LOG.error("Exception occurred while restoring the snapshot " + 
snapshot.getName()
  + " as table " + tableName.getNameAsString(), e);
// If QEE is thrown before restoreSnapshot, quota information is not 
updated, and we
// should throw the exception directly. If QEE is thrown after 
restoreSnapshot, there
// must be unexpected reasons, we also throw the exception directly
throw e;
  } catch (IOException e) {
if (tableRegionCount > 0) {
  // reset region count for table
  checkAndUpdateNamespaceRegionQuota(tableRegionCount, tableName);
}
LOG.error("Exception occurred while restoring the snapshot " + 
snapshot.getName()
+ " as table " + tableName.getNameAsString(), e);
throw e;
  }
{code}
What's your opinion about this issue? [~ashish singhi]

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-13 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15192813#comment-15192813
 ] 

Ashish Singhi commented on HBASE-15433:
---

OK. Now I understood.
When QEE is thrown we will still end up in updating the region quota which is 
not really required, may be we can avoid that.

{code}
// update region quota after restore if updatedRegionCount is smaller 
740 if (currentRegionCount > 0 && currentRegionCount > 
updatedRegionCount) {
741   checkAndUpdateNamespaceRegionQuota(updatedRegionCount, 
tableName);
742 }
{code}
If this throws exception then there will be another issue, because now the 
snapshot has been successfully restored but in the catch clause we are updating 
the table region count in namespace quota.

Also suggest to rename currentRegionCount to tableRegionCount and 
updatedRegionCount to snapshotRegionCount for better understanding. Please add 
more comments like why are we doing this way. 

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-13 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15192690#comment-15192690
 ] 

Jianwei Cui commented on HBASE-15433:
-

Thanks for your comment and sorry to reply late [~ashish singhi].
{quote}
So in catch clause first let us catch QEE and then IOE. If QEE is caught then 
we will not update the quota information.
{quote}
Yes, we don't need to update the quota information if QEE is caught. However, 
if IOE is caught, this means {{checkAndUpdateNamespaceRegionQuota}} succeeded 
while the following {{restoreSnapshot(SnapshotDescription, HTableDescriptor)}} 
failed, and the quota information has been updated by the region count in the 
snapshot. For example, the original region count is 10 for the table, and there 
are 5 regions in the snapshot, the region count will be updated to 5 in such 
case? and we need to reset the region count to 10 for the table in {{catch}}?

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-11 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190915#comment-15190915
 ] 

Ashish Singhi commented on HBASE-15433:
---

{quote}
If the snapshot contains less regions than the current table's, 
'checkAndUpdateNamespaceRegionQuota' will update the region count for the 
table, we need to reset the region count in 'catch' block if 'restoreSnapshot' 
throw exception?
{quote}
Sorry I am not getting your point. 
Correct me If I am wrong. The issue what I think is if 
{{checkAndUpdateNamespaceRegionQuota}} throws exception then also we were 
updating the table information in the namespace quota which was not correct. So 
in catch clause first let us catch QEE and then IOE. If QEE is caught then we 
will not update the quota information.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-11 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190901#comment-15190901
 ] 

Jianwei Cui commented on HBASE-15433:
-

I get your point, Yes, it will be more concise to remove the 'else' keyword:)

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-11 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190895#comment-15190895
 ] 

Jianwei Cui commented on HBASE-15433:
-

Thanks for your comment, will make a new unit test and update the patch.
{quote}
I was thinking of a simple fix like, just catch the QuoteExceededException and 
don't remove the table from namespace quota.
{quote}
If the snapshot contains less regions than the current table's, 
'checkAndUpdateNamespaceRegionQuota' will update the region count for the 
table, we need to reset the region count in 'catch' block if 'restoreSnapshot' 
throw exception?


> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-10 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189255#comment-15189255
 ] 

Ted Yu commented on HBASE-15433:


I meant the 'else' keyword can be omitted since the if block would always 
return.

+1 if new test passes and there is no quota related test failure.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-10 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189244#comment-15189244
 ] 

Ashish Singhi commented on HBASE-15433:
---

You need to generate the patch from source code root folder. Please follow the 
instructions given [here|http://hbase.apache.org/book.html#submitting.patches].

Coming to the fix, why do all these to fix this issue ? 
I was thinking of a simple fix like, just catch the {{QuoteExceededException}} 
and don't remove the table from namespace quota.
Also can you add your test as a new unit test method ?

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk-v2.patch, 
> HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-10 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15188919#comment-15188919
 ] 

Jianwei Cui commented on HBASE-15433:
-

Thanks for your comments [~yuzhih...@gmail.com]. The 'else' block will throw 
exception when the NamespaceStateManager is not initialized, this will make 
sure the NamespaceAuditor in right state when the method is invoked? Will add 
unit test for this case.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-09 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15188712#comment-15188712
 ] 

Ted Yu commented on HBASE-15433:


{code}
112   return state != null ? state.getRegionCountOfTable(tName) : -1;
113 } else {
{code}
the 'else' keyword is not needed above.

Is it possible to add a test ?

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk-v1.patch, HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-09 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187210#comment-15187210
 ] 

Jianwei Cui commented on HBASE-15433:
-

If the snapshot contains less regions than the current table's, 
'checkAndUpdateNamespaceRegionQuota' will reduce the region count of the table. 
The remaining region quota may be consumed by others(such as table creation, 
region split, etc) during the restore procedure, therefore, we can not reset 
the region count for the table in 'catch' block if the restore procedure fails. 
Will update another patch to fix this case.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-09 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187064#comment-15187064
 ] 

Jianwei Cui commented on HBASE-15433:
-

I tried the patch on trunk and will try it on other branches.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
> Attachments: HBASE-15433-trunk.patch
>
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-09 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187030#comment-15187030
 ] 

Ashish Singhi commented on HBASE-15433:
---

Great!

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-09 Thread Jianwei Cui (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187024#comment-15187024
 ] 

Jianwei Cui commented on HBASE-15433:
-

Thanks for your comment [~ashish singhi], I made a patch to fix this case and 
will upload after test passed:)

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15433) SnapshotManager#restoreSnapshot not update table and region count quota correctly when encountering exception

2016-03-09 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187005#comment-15187005
 ] 

Ashish Singhi commented on HBASE-15433:
---

Planning to give a patch ? If not I can try.

> SnapshotManager#restoreSnapshot not update table and region count quota 
> correctly when encountering exception
> -
>
> Key: HBASE-15433
> URL: https://issues.apache.org/jira/browse/HBASE-15433
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>
> In SnapshotManager#restoreSnapshot, the table and region quota will be 
> checked and updated as:
> {code}
>   try {
> // Table already exist. Check and update the region quota for this 
> table namespace
> checkAndUpdateNamespaceRegionQuota(manifest, tableName);
> restoreSnapshot(snapshot, snapshotTableDesc);
>   } catch (IOException e) {
> 
> this.master.getMasterQuotaManager().removeTableFromNamespaceQuota(tableName);
> LOG.error("Exception occurred while restoring the snapshot " + 
> snapshot.getName()
> + " as table " + tableName.getNameAsString(), e);
> throw e;
>   }
> {code}
> The 'checkAndUpdateNamespaceRegionQuota' will fail if regions in the snapshot 
> make the region count quota exceeded, then, the table will be removed in the 
> 'catch' block. This will make the current table count and region count 
> decrease, following table creation or region split will succeed even if the 
> actual quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

43 matches

Mail list logo