[
https://issues.apache.org/jira/browse/GOBBLIN-2029?focusedWorklogId=912107&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-912107
]
ASF GitHub Bot logged work on GOBBLIN-2029:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 28/Mar/24 19:48
Start Date: 28/Mar/24 19:48
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #3906:
URL: https://github.com/apache/gobblin/pull/3906#discussion_r1543543370
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/proc/LaunchDagProc.java:
##########
@@ -70,41 +51,33 @@ public class LaunchDagProc extends
DagProc<Optional<Dag<JobExecutionPlan>>, Opti
metricContext.newContextAwareGauge(ServiceMetricNames.FLOW_ORCHESTRATION_DELAY,
orchestrationDelayCounter::get));
}
- @Override
- protected DagManager.DagId getDagId() {
- return this.launchDagTask.getDagId();
+ public LaunchDagProc(LaunchDagTask dagTask, FlowCompilationValidationHelper
flowCompilationValidationHelper) {
+ super(dagTask);
+ this.flowCompilationValidationHelper = flowCompilationValidationHelper;
}
@Override
protected Optional<Dag<JobExecutionPlan>> initialize(DagManagementStateStore
dagManagementStateStore)
throws IOException {
try {
- DagActionStore.DagAction dagAction = this.launchDagTask.getDagAction();
- FlowSpec flowSpec = loadFlowSpec(dagManagementStateStore, dagAction);
- flowSpec.addProperty(ConfigurationKeys.FLOW_EXECUTION_ID_KEY,
dagAction.getFlowExecutionId());
+ FlowSpec flowSpec =
dagManagementStateStore.getFlowSpec(FlowSpec.Utils.createFlowSpecUri(getDagId().getFlowId()));
+ flowSpec.addProperty(ConfigurationKeys.FLOW_EXECUTION_ID_KEY,
getDagId().getFlowExecutionId());
return
this.flowCompilationValidationHelper.createExecutionPlanIfValid(flowSpec).toJavaUtil();
} catch (URISyntaxException | SpecNotFoundException | InterruptedException
e) {
throw new RuntimeException(e);
}
}
- private FlowSpec loadFlowSpec(DagManagementStateStore
dagManagementStateStore, DagActionStore.DagAction dagAction)
- throws URISyntaxException, SpecNotFoundException {
- URI flowUri = FlowSpec.Utils.createFlowSpecUri(dagAction.getFlowId());
- return dagManagementStateStore.getFlowSpec(flowUri);
- }
-
@Override
- protected Optional<Dag<JobExecutionPlan>> act(DagManagementStateStore
dagManagementStateStore, Optional<Dag<JobExecutionPlan>> dag)
+ protected void act(DagManagementStateStore dagManagementStateStore,
Optional<Dag<JobExecutionPlan>> dag)
throws IOException {
if (!dag.isPresent()) {
log.warn("Dag with id " + getDagId() + " could not be compiled.");
// todo - add metrics
- return Optional.empty();
+ } else {
+ submitNextNodes(dagManagementStateStore, dag.get());
+ orchestrationDelayCounter.set(System.currentTimeMillis() -
DagManagerUtils.getFlowExecId(dag.get()));
Review Comment:
I realize this is purely for a metric, so not as relevant as when [this
comment](https://github.com/apache/gobblin/pull/3899#discussion_r1540638957)
about hard-coded "clocks".
if you do wish to address this case, however, I'd advise to define `long
DMSS::getCurrentTimeMillis()`.
any test wanting to insist on a known clock could then extend DMSS to
override that method
Issue Time Tracking
-------------------
Worklog Id: (was: 912107)
Time Spent: 50m (was: 40m)
> code style changes DagProc and related classes
> ----------------------------------------------
>
> Key: GOBBLIN-2029
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2029
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Arjun Singh Bora
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)