[
https://issues.apache.org/jira/browse/GOBBLIN-1863?focusedWorklogId=873885&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-873885
]
ASF GitHub Bot logged work on GOBBLIN-1863:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 31/Jul/23 19:15
Start Date: 31/Jul/23 19:15
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3727:
URL: https://github.com/apache/gobblin/pull/3727#discussion_r1279761914
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/DagManager.java:
##########
@@ -455,6 +470,54 @@ public synchronized void setActive(boolean active) {
}
}
+ /**
+ * Used by the DagManager to launch a new execution for a flow action event
loaded from the DagActionStore upon
+ * setting this instance of the DagManager to active. Because it may be a
completely new DAG not contained in the
+ * dagStore, we compile the flow to generate the dag before calling
addDag(), handling any errors that may result in
+ * the process.
+ */
+ public void handleLaunchFlowEvent(DagActionStore.DagAction action) {
+ if (this.specCompiler.isPresent()) {
+ FlowId flowId = new
FlowId().setFlowGroup(action.getFlowGroup()).setFlowName(action.getFlowName());
Review Comment:
Same comment here as the one above, wanted to double check that all the
checks are being done before submitting these flows which the orchestrator is
doing, otherwise there will be some bugs introduced that are guarded against.
Is it possible to use a helper or a static function?
Issue Time Tracking
-------------------
Worklog Id: (was: 873885)
Time Spent: 0.5h (was: 20m)
> Multi-Active Launch Job Related Issues
> --------------------------------------
>
> Key: GOBBLIN-1863
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1863
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: Urmi Mustafi
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> * DagManager check leader status before addDag bc calling this method from
> non-leader hosts throws a NPE which may caused failed dag event to be emitted
> * also handle LAUNCH type events upon leader change and setting a new
> participant DagManager to be active. Failing to handle these events may be
> causing missed flow launches on any leader change or restart.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
