umustafi commented on code in PR #3550:
URL: https://github.com/apache/gobblin/pull/3550#discussion_r966376550
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/scheduler/GobblinServiceJobScheduler.java:
##########
@@ -335,7 +335,9 @@ public AddSpecResponse onAddSpec(Spec addedSpec) {
if (quotaManager.isPresent()) {
// QuotaManager has idempotent checks for a dagNode, so this check
won't double add quotas for a flow in the DagManager
try {
+ // todo : we should probably check quota for all of the start nodes
quotaManager.get().checkQuota(dag.getNodes().get(0));
+ ((FlowSpec)
addedSpec).getConfigAsProperties().setProperty(ServiceConfigKeys.GOBBLIN_SERVICE_ADHOC_FLOW,
"true");
Review Comment:
that function notifies the `addSpec` immediately after so I'm not sure we
will use the value directly anywhere else. However, if we do use the in memory
value of the Spec in other places, this property will be useful to add there.
Btw shouldn't this value be set to false after the first iteration of an adhoc
flow in the `DagManager` or Scheduler?
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/scheduler/GobblinServiceJobScheduler.java:
##########
@@ -335,7 +335,9 @@ public AddSpecResponse onAddSpec(Spec addedSpec) {
if (quotaManager.isPresent()) {
// QuotaManager has idempotent checks for a dagNode, so this check
won't double add quotas for a flow in the DagManager
try {
+ // todo : we should probably check quota for all of the start nodes
Review Comment:
do u mean this for all hops in multi-hop flow?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
