phet commented on code in PR #3906:
URL: https://github.com/apache/gobblin/pull/3906#discussion_r1543543370
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/proc/LaunchDagProc.java:
##########
@@ -70,41 +51,33 @@ public class LaunchDagProc extends
DagProc<Optional<Dag<JobExecutionPlan>>, Opti
metricContext.newContextAwareGauge(ServiceMetricNames.FLOW_ORCHESTRATION_DELAY,
orchestrationDelayCounter::get));
}
- @Override
- protected DagManager.DagId getDagId() {
- return this.launchDagTask.getDagId();
+ public LaunchDagProc(LaunchDagTask dagTask, FlowCompilationValidationHelper
flowCompilationValidationHelper) {
+ super(dagTask);
+ this.flowCompilationValidationHelper = flowCompilationValidationHelper;
}
@Override
protected Optional<Dag<JobExecutionPlan>> initialize(DagManagementStateStore
dagManagementStateStore)
throws IOException {
try {
- DagActionStore.DagAction dagAction = this.launchDagTask.getDagAction();
- FlowSpec flowSpec = loadFlowSpec(dagManagementStateStore, dagAction);
- flowSpec.addProperty(ConfigurationKeys.FLOW_EXECUTION_ID_KEY,
dagAction.getFlowExecutionId());
+ FlowSpec flowSpec =
dagManagementStateStore.getFlowSpec(FlowSpec.Utils.createFlowSpecUri(getDagId().getFlowId()));
+ flowSpec.addProperty(ConfigurationKeys.FLOW_EXECUTION_ID_KEY,
getDagId().getFlowExecutionId());
return
this.flowCompilationValidationHelper.createExecutionPlanIfValid(flowSpec).toJavaUtil();
} catch (URISyntaxException | SpecNotFoundException | InterruptedException
e) {
throw new RuntimeException(e);
}
}
- private FlowSpec loadFlowSpec(DagManagementStateStore
dagManagementStateStore, DagActionStore.DagAction dagAction)
- throws URISyntaxException, SpecNotFoundException {
- URI flowUri = FlowSpec.Utils.createFlowSpecUri(dagAction.getFlowId());
- return dagManagementStateStore.getFlowSpec(flowUri);
- }
-
@Override
- protected Optional<Dag<JobExecutionPlan>> act(DagManagementStateStore
dagManagementStateStore, Optional<Dag<JobExecutionPlan>> dag)
+ protected void act(DagManagementStateStore dagManagementStateStore,
Optional<Dag<JobExecutionPlan>> dag)
throws IOException {
if (!dag.isPresent()) {
log.warn("Dag with id " + getDagId() + " could not be compiled.");
// todo - add metrics
- return Optional.empty();
+ } else {
+ submitNextNodes(dagManagementStateStore, dag.get());
+ orchestrationDelayCounter.set(System.currentTimeMillis() -
DagManagerUtils.getFlowExecId(dag.get()));
Review Comment:
I realize this is purely for a metric, so not as relevant as when [this
comment](https://github.com/apache/gobblin/pull/3899#discussion_r1540638957)
about hard-coded "clocks".
if you do wish to address this case, however, I'd advise to define `long
DMSS::getCurrentTimeMillis()`.
any test wanting to insist on a known clock could then override DMSS and
override that method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]