umustafi commented on code in PR #3863:
URL: https://github.com/apache/gobblin/pull/3863#discussion_r1465694903


##########
gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/KafkaJobStatusMonitor.java:
##########
@@ -281,6 +282,7 @@ static void 
addJobStatusToStateStore(org.apache.gobblin.configuration.State jobS
       stateStore.put(storeName, tableName, jobStatus);
       if (isNewStateTransitionToFinal(jobStatus, states)) {
         eventProducer.emitObservabilityEvent(jobStatus);
+        eventBus.post(new JobStatusEvent(jobStatus));

Review Comment:
   why is this change here? can u add a comment to explain who the listener is 
here of the job status event?



##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/core/GobblinServiceGuiceModule.java:
##########
@@ -173,6 +180,12 @@ public void configure(Binder binder) {
       binder.bind(FlowTriggerHandler.class);
     }
 
+    binder.bind(DagManagement.class).to(NewDagManager.class);
+    
binder.bind(DagManagementStateStore.class).to(InMemoryDagManagementStateStore.class);
+    binder.bind(DagProcFactory.class);
+    binder.bind(DagProcessingEngine.class);
+    binder.bind(DagTaskStream.class);

Review Comment:
   these should be optionally bound through the config 
`isDagProcessingEngineEnabled `



##########
gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/DagActionStoreChangeMonitor.java:
##########
@@ -250,7 +259,11 @@ private void handleDagAction(DagActionStore.DagAction 
dagAction, boolean isStart
         throw new RuntimeException(String.format("Received LAUNCH dagAction 
while not in multi-active scheduler "
             + "mode for flowAction: %s", dagAction));
       }
-      submitFlowToDagManagerHelper(dagAction, isStartup);
+      if (dagProcessingEngineEnabled) {
+        this.dagTaskStream.addDagAction(dagAction);
+      } else {
+        submitFlowToDagManagerHelper(dagAction, isStartup);
+      }

Review Comment:
   in this PR we should still have it in "test" mode so if dagProcessingEngine 
is labeled we add some logging around the code paths it follows and we can 
trace it but not necessarily change the path yet. What do you think? 



##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/scheduler/GobblinServiceJobScheduler.java:
##########
@@ -210,18 +206,6 @@ public 
GobblinServiceJobScheduler(@Named(InjectionNames.SERVICE_NAME) String ser
     }
   }
 
-  public GobblinServiceJobScheduler(String serviceName, Config config, 
FlowStatusGenerator flowStatusGenerator,
-      Optional<HelixManager> helixManager, Optional<FlowCatalog> flowCatalog, 
TopologyCatalog topologyCatalog,
-      DagManager dagManager, Optional<UserQuotaManager> quotaManager, 
SchedulerService schedulerService,
-      Optional<Logger> log, boolean isWarmStandbyEnabled, Optional 
<FlowTriggerHandler> flowTriggerHandler,
-      SharedFlowMetricsSingleton sharedFlowMetricsSingleton)
-      throws Exception {
-    this(serviceName, config, helixManager, flowCatalog,
-        new Orchestrator(config, topologyCatalog, dagManager, log, 
flowStatusGenerator, flowTriggerHandler,
-            sharedFlowMetricsSingleton, flowCatalog),
-        schedulerService, quotaManager, log, isWarmStandbyEnabled, 
flowTriggerHandler);
-  }
-

Review Comment:
   why did this get removed?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to