[
https://issues.apache.org/jira/browse/HBASE-27784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault updated HBASE-27784:
--------------------------------------
Description:
The below is the original idea that started this work, but not what we actually
landed on. See the first comment from [~rmdmattingly] and the release note for
that.
Old description:
{quote}Currently we provide the ability to define quotas for namespaces,
tables, or users. On multi-tenant clusters, users may be broken down into
groups based on their use-case. For us this comes down to 2 main cases:
# Hadoop jobs – it would be good to be able to limit all hadoop jobs in
aggregate
# Proxy APIs - this is common where upstream callers don't hit hbase directly,
instead they go through one of many proxy api's. For us we have a custom auth
plugin which sets the username to the upstream caller name. But it would still
be useful to be able to limit all usage from some particular proxy API in
aggregate.
I think this could build upon the idea for Connection attributes in
HBASE-27657. Basically when a Connection is established we can set an attribute
(i.e. quotaGrouping=hadoop or quotaGrouping=MyProxyAPI). In QuotaCache, we can
add a {{getQuotaGroupLimiter(String groupName)}} and also allow someone to
define quotas using {{set_quota TYPE => THROTTLE, GROUP => 'hadoop', LIMIT =>
'100M/sec'}}
I need to do more investigation into whether we'd want to return a simple group
limiter (more similar to table/namespace handling) or treat it more like the
USER limiters which returns a QuotaState (so you can limit by-group-by-table).
We need to consider how GROUP quotas interact with USER quotas. If a user has a
quota defined, and that user is also part of a group with a quota defined, does
the request need to honor both quotas? Maybe we provide a GROUP_BYPASS setting,
similar to GLOBAL_BYPASS?
{quote}
was:
Currently we provide the ability to define quotas for namespaces, tables, or
users. On multi-tenant clusters, users may be broken down into groups based on
their use-case. For us this comes down to 2 main cases:
# Hadoop jobs – it would be good to be able to limit all hadoop jobs in
aggregate
# Proxy APIs - this is common where upstream callers don't hit hbase directly,
instead they go through one of many proxy api's. For us we have a custom auth
plugin which sets the username to the upstream caller name. But it would still
be useful to be able to limit all usage from some particular proxy API in
aggregate.
I think this could build upon the idea for Connection attributes in
HBASE-27657. Basically when a Connection is established we can set an attribute
(i.e. quotaGrouping=hadoop or quotaGrouping=MyProxyAPI). In QuotaCache, we can
add a {{getQuotaGroupLimiter(String groupName)}} and also allow someone to
define quotas using {{set_quota TYPE => THROTTLE, GROUP => 'hadoop', LIMIT =>
'100M/sec'}}
I need to do more investigation into whether we'd want to return a simple group
limiter (more similar to table/namespace handling) or treat it more like the
USER limiters which returns a QuotaState (so you can limit by-group-by-table).
We need to consider how GROUP quotas interact with USER quotas. If a user has a
quota defined, and that user is also part of a group with a quota defined, does
the request need to honor both quotas? Maybe we provide a GROUP_BYPASS setting,
similar to GLOBAL_BYPASS?
> support quota user overrides
> ----------------------------
>
> Key: HBASE-27784
> URL: https://issues.apache.org/jira/browse/HBASE-27784
> Project: HBase
> Issue Type: New Feature
> Reporter: Bryan Beaudreault
> Assignee: Ray Mattingly
> Priority: Major
>
> The below is the original idea that started this work, but not what we
> actually landed on. See the first comment from [~rmdmattingly] and the
> release note for that.
>
> Old description:
> {quote}Currently we provide the ability to define quotas for namespaces,
> tables, or users. On multi-tenant clusters, users may be broken down into
> groups based on their use-case. For us this comes down to 2 main cases:
> # Hadoop jobs – it would be good to be able to limit all hadoop jobs in
> aggregate
> # Proxy APIs - this is common where upstream callers don't hit hbase
> directly, instead they go through one of many proxy api's. For us we have a
> custom auth plugin which sets the username to the upstream caller name. But
> it would still be useful to be able to limit all usage from some particular
> proxy API in aggregate.
> I think this could build upon the idea for Connection attributes in
> HBASE-27657. Basically when a Connection is established we can set an
> attribute (i.e. quotaGrouping=hadoop or quotaGrouping=MyProxyAPI). In
> QuotaCache, we can add a {{getQuotaGroupLimiter(String groupName)}} and also
> allow someone to define quotas using {{set_quota TYPE => THROTTLE, GROUP =>
> 'hadoop', LIMIT => '100M/sec'}}
> I need to do more investigation into whether we'd want to return a simple
> group limiter (more similar to table/namespace handling) or treat it more
> like the USER limiters which returns a QuotaState (so you can limit
> by-group-by-table).
> We need to consider how GROUP quotas interact with USER quotas. If a user has
> a quota defined, and that user is also part of a group with a quota defined,
> does the request need to honor both quotas? Maybe we provide a GROUP_BYPASS
> setting, similar to GLOBAL_BYPASS?
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
