[
https://issues.apache.org/jira/browse/GOBBLIN-1656?focusedWorklogId=778299&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-778299
]
ASF GitHub Bot logged work on GOBBLIN-1656:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 03/Jun/22 20:41
Start Date: 03/Jun/22 20:41
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3516:
URL: https://github.com/apache/gobblin/pull/3516#discussion_r889333264
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/scheduler/GobblinServiceJobScheduler.java:
##########
@@ -322,6 +326,18 @@ public AddSpecResponse onAddSpec(Spec addedSpec) {
return new AddSpecResponse<>(response);
}
+ // Check quota limits against run immediately flows or adhoc flows before
saving the schedule
+ if (!jobConfig.containsKey(ConfigurationKeys.JOB_SCHEDULE_KEY) ||
PropertiesUtils.getPropAsBoolean(jobConfig,
ConfigurationKeys.FLOW_RUN_IMMEDIATELY, "false")) {
+ try {
+ if (quotaManager.isPresent()) {
+ quotaManager.get().checkQuota(dag.getNodes().get(0), false);
Review Comment:
It wouldn't since in the QuotaManager there's a check against duplicate
dagIds to not double increment if the dag was already added. I can add a
comment here to reflect this behavior. It seems a bit hacky but I couldn't
think of another way of handling this scenario since the dagmanager
asynchronous design doesn't make it easy to validate that a flow is running on
the specExecutor before returning a http status code.
Issue Time Tracking
-------------------
Worklog Id: (was: 778299)
Time Spent: 1.5h (was: 1h 20m)
> Return different Http Status on GaaS if Quota is Exceeded
> ---------------------------------------------------------
>
> Key: GOBBLIN-1656
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1656
> Project: Apache Gobblin
> Issue Type: New Feature
> Components: gobblin-service
> Reporter: William Lo
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> GaaS has quota limits for proxy users and flowGroups.
> When a user wants to create a flow that exceeds their specified quota, the
> flow should
> 1) Not be run
> 2) Return a http status code (i.e. 503) due to exceeding the resource. This
> allows clients to implement some wait and retry functionality
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
