[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756883#comment-16756883
]
Hudson commented on HBASE-21225:
Results for branch branch-2.1
[build #817 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/817/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/817//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/817//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/817//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756871#comment-16756871
]
Hudson commented on HBASE-21225:
Results for branch branch-2.0
[build #1301 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1301/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1301//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1301//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1301//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743392#comment-16743392
]
Sakthi commented on HBASE-21225:
[~elserj], I think this backport can be done after HBASE-21034 is backported to
branch-2.0 & branch-2.1.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742833#comment-16742833
]
Hudson commented on HBASE-21225:
Results for branch branch-2
[build #1611 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1611/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1611//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1611//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1611//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742812#comment-16742812
]
Hudson commented on HBASE-21225:
Results for branch master
[build #720 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/720/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/720//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/720//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/720//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742643#comment-16742643
]
Sakthi commented on HBASE-21225:
Sure [~elserj], I'll take a look at both the branches. Will update here. Thanks
:)
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742631#comment-16742631
]
Josh Elser commented on HBASE-21225:
LGTM, pushing. Thanks for bearing with me, Sakthi.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740922#comment-16740922
]
Sakthi commented on HBASE-21225:
Hey [~elserj], the failed UT looks unrelated. It passed locally. Rest of the QA
looks fine.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740914#comment-16740914
]
Hadoop QA commented on HBASE-21225:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
15s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
5s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 54s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
28s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}148m 23s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}207m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestHbck |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21225 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954636/hbase-21225.master.005.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux cf63acc3b75c 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3d2580cd6d |
| maven | version: Apache Maven 3.5.4
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740814#comment-16740814
]
Josh Elser commented on HBASE-21225:
Looks OK. Let's see what QA says.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740799#comment-16740799
]
Sakthi commented on HBASE-21225:
Oh yes. It would be wise to have the previous tests. I have restored the tests
and have modified testTableSpaceAndRPCQuotaRemoved,
testNamespaceSpaceAndRPCQuotaRemoved. Have uploaded the new patch, [~elserj].
Let me know :)
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740766#comment-16740766
]
Josh Elser commented on HBASE-21225:
[~jatsakthi], the current patch looks OK, but I can't help but feel like you
have removed test coverage for just a space quota and just a throttle quota.
Normally I wouldn't care too much because deleting a space/throttle in the case
where there are also other quotas is more complicated, but we've had "broken"
logic around this once already.
Can we restore {{testTableSpaceQuotaRemoved}} and {{testTableRPCQuotaRemoved}}?
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740761#comment-16740761
]
Sakthi commented on HBASE-21225:
How does the patch look [~elserj]? Any further scope for improvements?
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740709#comment-16740709
]
Josh Elser commented on HBASE-21225:
{quote}the reason I was trying to remove the "remove" attribute was, I saw that
throttle quota's implementation didn't seem to have a "remove" attribute and
wanted to keep both the implementations as similar as possible
{quote}
IMO, the implementation of the ThrottleQuota is strange. It's really
complicated for no apparent reason to me :). It's not a gold-standard to hold
other code to for me.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740300#comment-16740300
]
Hadoop QA commented on HBASE-21225:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
39s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
39s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 47s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
18s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}155m 9s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}208m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.multiwal.TestReplicationKillMasterRSCompressedWithMultipleWAL
|
| |
hadoop.hbase.replication.multiwal.TestReplicationKillMasterRSCompressedWithMultipleAsyncWAL
|
| | hadoop.hbase.replication.TestReplicationDroppedTables |
| | hadoop.hbase.client.TestHbck |
| | hadoop.hbase.replication.TestReplicationKillMasterRSCompressed |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21225 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954547/hbase-21225.master.004.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d1607e0b9375
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740139#comment-16740139
]
Sakthi commented on HBASE-21225:
Have updated the patch with the change. Merged all existing tests with checks
for this Jira into 2 tests(testTableWithSpaceAndRPCQuota,
testNamespaceWithSpaceAndRPCQuota) in TestMasterQuotasObserver.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740079#comment-16740079
]
Sakthi commented on HBASE-21225:
Thanks for your detailed review [~elserj]. One, I hadn't thought of backward
compatibility issues. Two, the reason I was trying to remove the "remove"
attribute was, I saw that throttle quota's implementation didn't seem to have a
"remove" attribute and wanted to keep both the implementations as similar as
possible. But like you mentioned, the removal would require too much of work
(for little to no gain).
{quote}I'm pretty sure this is what the intention was. If REMOVE was set, the
merged quota object should rip it out when constructing the "merged" view of
all quotas for the entity (table or ns).
{quote}
Will update the patch for this.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739976#comment-16739976
]
Josh Elser commented on HBASE-21225:
bq. Have uploaded a patch with "remove" attribute from SpaceQuota protobuf
removed
Let me also clarify: you can try to do this, but it's going to be tricky to
unwind correctly.
* You must _never_ remove fields from a protobuf message. This causes things to
really break down
https://developers.google.com/protocol-buffers/docs/proto#reserved
* You would need to add something to MasterQuotaManager to "fix" the
hbase:quota table that might have a REMOVE in it already
* You would still need special handling in the Master to handle older clients
that do not have the change that you're proposing in this Jira issue.
Perhaps I'm missing some fundamental reason why you want to remove this
attribute instead of make it work, but, for now, I'll assume it's just that
hadn't thought about the backwards compatibility issues yet :)
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739967#comment-16739967
]
Josh Elser commented on HBASE-21225:
bq. Have uploaded a patch with "remove" attribute from SpaceQuota protobuf
removed. Waiting for the unit tests reports from 'Hadoop QA'.
I still believe this is the wrong approach. If you start removing things from
the protobuf, you're going to break backwards compatibility and have to write
an upgrade step for serialized data that might have this attribute (albeit,
unwanted).
bq. As the "mergedQuota" is of type GlobalQuotaSettingsImpl and it's
constructor takes in Throttle.Builder & SpaceQuota.Builder as parameter, we can
pass in null values to both these parameters as and when required (whenever
"remove" is set?)
I'm pretty sure this is what the intention was. If {{REMOVE}} was set, the
merged quota object should rip it out when constructing the "merged" view of
all quotas for the entity (table or ns).
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736292#comment-16736292
]
Sakthi commented on HBASE-21225:
Hey [~elserj], do you mind having a look at the patch when possible. Thanks.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725319#comment-16725319
]
Sakthi commented on HBASE-21225:
The failed UTs don't look related. [~elserj] & [~nihaljain.cs] do you guys
mind reviewing the patch?
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724172#comment-16724172
]
Hadoop QA commented on HBASE-21225:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} hbase-client: The patch generated 4 new + 17 unchanged
- 0 fixed = 21 total (was 17) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
26s{color} | {color:red} hbase-server: The patch generated 6 new + 2 unchanged
- 1 fixed = 8 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
44s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 0s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
2m 20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
42s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
44s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}260m 36s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}341m 37s{color} |
{color:black} {color} |
\\
\\
||
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724170#comment-16724170
]
Hadoop QA commented on HBASE-21225:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
49s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
31s{color} | {color:red} hbase-client: The patch generated 1 new + 17 unchanged
- 0 fixed = 18 total (was 17) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
3s{color} | {color:red} hbase-server: The patch generated 6 new + 3 unchanged -
0 fixed = 9 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 29s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
21s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}266m 26s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}328m 18s{color} |
{color:black} {color} |
\\
\\
||
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723892#comment-16723892
]
Sakthi commented on HBASE-21225:
Have uploaded a patch with "remove" attribute from SpaceQuota protobuf removed.
Waiting for the unit tests reports from 'Hadoop QA'.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722767#comment-16722767
]
Sakthi commented on HBASE-21225:
Will try to modify and come up with a patch, if possible.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722766#comment-16722766
]
Sakthi commented on HBASE-21225:
I am starting to think here, if we can get rid of the "REMOVE" attribute(which
was added in HBASE-17259) in the protobuf (like the ThrottleRequest doesn't
have one), and can still make sure that the remove functionality is available
for public APIs.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722261#comment-16722261
]
Hadoop QA commented on HBASE-21225:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
50s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 17s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}128m
37s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21225 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12941663/hbase-21225.master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c3836540fd0a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ac0b3bb547 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15291/testReport/ |
| Max. process+thread count | 4561 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15291/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Having RPC & Space
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651154#comment-16651154
]
Sakthi commented on HBASE-21225:
A gentle nudge, [~elserj]
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642882#comment-16642882
]
Sakthi commented on HBASE-21225:
My observation:
{code:java}
QuotaSettings newQuota = QuotaSettings.buildFromProto(req);
GlobalQuotaSettingsImpl mergedQuota = currentQuota.merge(newQuota);
if (mergedQuota == null) {
quotaOps.delete();
} else {
quotaOps.update(mergedQuota);
}
{code}
The way to delete a quota is merging the new "delete" quota request with
whatever the current quotasettings the entity has and creating a new
GlobalQuotaSettingsImpl. While writing out data to the hbase:quota,
GlobalQuotaSettingsImpl.toQuotas() is called to create a Quotas object which is
converted to a byte array to update in the hbase:quota.
Few ways to handle the situation:
* As the "mergedQuota" is of type GlobalQuotaSettingsImpl and it's constructor
takes in Throttle.Builder & SpaceQuota.Builder as parameter, we can pass in
null values to both these parameters as and when required (whenever "remove" is
set?)
* Or, pass remove objects(i.e. spaceBuilder with only remove set to true) to
the constructor
## In the constructor of GlobalQuotaSettingsImpl, we can handle as to not set
"QuotaProtos.SpaceQuota spaceProto"data member, if "remove" objects are passed.
## Or, let the GlobalQuotaSettingsImpl be initialized with those objects. We
can later handle the case when we are trying to write out the data using
GlobalQuotaSettingsImpl.toQuotas().
{code:java}
protected Quotas toQuotas() {
QuotaProtos.Quotas.Builder builder = QuotaProtos.Quotas.newBuilder();
if (getThrottleProto() != null) {
builder.setThrottle(getThrottleProto());
}
if (getSpaceProto() != null) {
builder.setSpace(getSpaceProto());
}
return builder.build();
}
{code}
Here, along with the null checking we can also put the "remove set?" check as
well.
* Or, are we planning to wait little bit more till the end point where we are
actually writing the data? In that case, I wasn't able to figure out how would
we do that because I could see that we are using ".writeTo" of protobuf to
convert quotas into byte arrays. As here:
{code:java}
protected static byte[] quotasToData(final Quotas data) throws IOException {
ByteArrayOutputStream stream = new ByteArrayOutputStream();
stream.write(ProtobufMagic.PB_MAGIC);
data.writeTo(stream);
return stream.toByteArray();
}{code}
I prefer the first one, that way all the checking is contained in the merge
logic.
[~elserj], your thoughts, please?
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
>
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636364#comment-16636364
]
Sakthi commented on HBASE-21225:
[~elserj] thanks for your comments.
{quote}1. Why all of the modification/removal of existing tests?
{quote}
Actually I just added two test cases (to specifically test table/namespaces
with both quotas) and did just rename the previous ones to clearly specify that
those tests are to for drops with a single quota set on the entity. Looks like
the git diff shows previous test cases also to be modified (which is not the
case).
{quote}2. The purpose of the REMOVE => true entry is to denote that HBase
should remove the SpaceQuota when it writes that out. With this change, it
seems like you're leaving the REMOVE attribute in the protobuf message, but
completely ignoring it which is confusing.
{quote}
Yes, you are right.
{quote}IMO, the bug is that GlobalSettingsQuotaImpl is not correctly removing
the SpaceQuota when REMOVE is set to true.
{quote}
As of now, the bug seems to be in the following code peice:
{code:java}
if (throttleBuilder == null &&
(spaceBuilder == null || (spaceBuilder.hasRemove() &&
spaceBuilder.getRemove()))
&& bypassGlobals == null) {
return null;
}
{code}
When the throttlebuilder is not set and spacebuilder is set to remove, then
null is returned as the "merged" quota. But when throttlebuilder is set and
spacebuilder is set to remove, then it doesn't enter this condition and rather
passes over to this return statement:
{code:java}
return new GlobalQuotaSettingsImpl(
getUserName(), getTableName(), getNamespace(),
(throttleBuilder == null ? null : throttleBuilder.build()),
bypassGlobals,
(spaceBuilder == null ? null : spaceBuilder.build()));{code}
which just checks the "null" condition of spacebuilder and creates a new
GlobalQuotaSettingsImpl object that actually makes the "remove=>true" entry
stay in the table. Thinking along the lines to just fix how this works, instead
of just checking that spaceBuilder == null, we can also check if remove is set,
in this return statement.
{code:java}
return new GlobalQuotaSettingsImpl(
getUserName(), getTableName(), getNamespace(),
(throttleBuilder == null ? null : throttleBuilder.build()),
bypassGlobals,
(spaceBuilder == null || (spaceBuilder.hasRemove() &&
spaceBuilder.getRemove())? null : spaceBuilder.build()));{code}
This will produce the desired results.
{quote}Seems the logic I had initially written around an RPC and Space quota on
the same table/namespace is lacking. Does that make sense?{quote}
I will have to dig deeper. Will loop back with ideas, Josh.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632004#comment-16632004
]
Josh Elser commented on HBASE-21225:
[~jatsakthi], please update the description with an explanation of the problem,
not just the reproduction steps. In HBASE-20705, you seemed to have some notion
of why this still doesn't work in all cases.
A couple of comments:
1. Why all of the modification/removal of existing tests?
{code:java}
if (quotaToMerge.getRemove()) {
- // Update the builder to propagate the removal
- spaceBuilder.setRemove(true).clearSoftLimit().clearViolationPolicy();
+ // To prevent the "REMOVE => true" row in
QuotaTableUtil.QUOTA_TABLE_NAME
+ spaceBuilder = null;{code}
2. The purpose of the {{REMOVE => true}} entry is to denote that HBase should
remove the SpaceQuota when it writes that out. With this change, it seems like
you're leaving the {{REMOVE}} attribute in the protobuf message, but completely
ignoring it which is confusing.
IMO, the bug is that GlobalSettingsQuotaImpl is not correctly removing the
SpaceQuota when REMOVE is set to true. Seems the logic I had initially written
around an RPC and Space quota on the same table/namespace is lacking. Does that
make sense?
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch
>
>
> A part of HBASE-20705 is still unresolved
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631821#comment-16631821
]
Hadoop QA commented on HBASE-21225:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
55s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 53s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}129m 36s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}182m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21225 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12941663/hbase-21225.master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7a4fc891e108 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 22ac655704 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/14527/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14527/testReport/ |
| Max. process+thread count | 4598 (vs. ulimit of 1) |
| modules | C: hbase-server U:
