[
https://issues.apache.org/jira/browse/GOBBLIN-1650?focusedWorklogId=775147&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-775147
]
ASF GitHub Bot logged work on GOBBLIN-1650:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 26/May/22 17:04
Start Date: 26/May/22 17:04
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3511:
URL: https://github.com/apache/gobblin/pull/3511#discussion_r882889332
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/UserQuotaManager.java:
##########
@@ -102,11 +115,20 @@ public void checkQuota(Dag.DagNode<JobExecutionPlan>
dagNode, boolean onInit) th
}
}
+ int flowGroupQuotaIncrement = incrementJobCountAndCheckQuota(
+ DagManagerUtils.getFlowGroupQuotaKey(flowGroup, dagNode),
flowGroupToJobCount, dagNode, getQuotaForFlowGroup(flowGroup));
+ boolean flowGroupCheck = flowGroupQuotaIncrement >= 0;
+ if (!flowGroupCheck) {
+ requesterMessage.append(String.format(
+ "Quota exceeded for flowgroup %s on executor %s : quota=%s, requests
above quota=%d%n",
+ flowGroup, specExecutorUri, getQuotaForFlowGroup(flowGroup),
Math.abs(flowGroupQuotaIncrement)+1-getQuotaForFlowGroup(flowGroup)));
+ }
+
// Throw errors for reach quota at the end to avoid inconsistent job counts
- if ((!proxyUserCheck || !requesterCheck) && !onInit) {
+ if ((!proxyUserCheck || !requesterCheck || !flowGroupCheck) && !onInit) {
// roll back the increased counts in this block
- String userKey = DagManagerUtils.getUserQuotaKey(proxyUser, dagNode);
- decrementQuotaUsage(proxyUserToJobCount, userKey);
+ decrementQuotaUsage(proxyUserToJobCount,
DagManagerUtils.getUserQuotaKey(proxyUser, dagNode));
Review Comment:
We also decrement if the quota is exceeded which is above. The guard will
prevent a double decrement when the dag is removed.
Concurrent hashmap won't handle the scenario if there is a write-after-read
scenario, where one thread reads (to check) and another thread writes in
between the check. So we either use a lock or the current method, which is to
write then check the modified value after writing since it's returned.
Issue Time Tracking
-------------------
Worklog Id: (was: 775147)
Time Spent: 1h 50m (was: 1h 40m)
> Allow GaaS to enforce quotas by flowgroup
> -----------------------------------------
>
> Key: GOBBLIN-1650
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1650
> Project: Apache Gobblin
> Issue Type: New Feature
> Reporter: William Lo
> Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Certain GaaS flows can have a large number of flows that can cause
> instability on dependent services. We want to be able to control the
> throughput of flows from a flowgroup granularity on top of a user
> granularity. We keep the quota configuration separate as there can be many
> users submitting flows to the same flowgroup.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
