[
https://issues.apache.org/jira/browse/GOBBLIN-1650?focusedWorklogId=774239&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774239
]
ASF GitHub Bot logged work on GOBBLIN-1650:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 24/May/22 21:31
Start Date: 24/May/22 21:31
Worklog Time Spent: 10m
Work Description: umustafi commented on code in PR #3511:
URL: https://github.com/apache/gobblin/pull/3511#discussion_r880967550
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/UserQuotaManager.java:
##########
@@ -159,6 +179,9 @@ public boolean releaseQuota(Dag.DagNode<JobExecutionPlan>
dagNode) {
String proxyUserKey = DagManagerUtils.getUserQuotaKey(proxyUser,
dagNode);
decrementQuotaUsage(proxyUserToJobCount, proxyUserKey);
}
+ String flowGroup =
ConfigUtils.getString(dagNode.getValue().getJobSpec().getConfig(),
+ ConfigurationKeys.FLOW_GROUP_KEY, "");
+ decrementQuotaUsage(flowGroupToJobCount,
DagManagerUtils.getFlowGroupQuotaKey(flowGroup, dagNode));
String serializedRequesters =
DagManagerUtils.getSerializedRequesterList(dagNode);
if (serializedRequesters != null) {
Review Comment:
When should `releaseQuota` be called other than for the tests below to do
manual testing? Is there a time period during which a quota is enforced ie: 5
concurrent jobs allowed for group/user X and then we call `releaseQuota` as
flows finish executing?
Issue Time Tracking
-------------------
Worklog Id: (was: 774239)
Time Spent: 0.5h (was: 20m)
> Allow GaaS to enforce quotas by flowgroup
> -----------------------------------------
>
> Key: GOBBLIN-1650
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1650
> Project: Apache Gobblin
> Issue Type: New Feature
> Reporter: William Lo
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Certain GaaS flows can have a large number of flows that can cause
> instability on dependent services. We want to be able to control the
> throughput of flows from a flowgroup granularity on top of a user
> granularity. We keep the quota configuration separate as there can be many
> users submitting flows to the same flowgroup.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
