[
https://issues.apache.org/jira/browse/GOBBLIN-1930?focusedWorklogId=885741&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-885741
]
ASF GitHub Bot logged work on GOBBLIN-1930:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 17/Oct/23 20:09
Start Date: 17/Oct/23 20:09
Worklog Time Spent: 10m
Work Description: umustafi commented on code in PR #3800:
URL: https://github.com/apache/gobblin/pull/3800#discussion_r1362704719
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/FlowTriggerHandler.java:
##########
@@ -117,13 +117,15 @@ public void handleTriggerEvent(Properties jobProps,
DagActionStore.DagAction flo
this.leaseObtainedCount.inc();
if (persistFlowAction(leaseObtainedStatus)) {
log.info("Successfully persisted lease: [{}, eventTimestamp: {}] ",
leaseObtainedStatus.getFlowAction(),
- leaseObtainedStatus.getEventTimestamp());
+ leaseObtainedStatus.getEventTimeMillis());
return;
}
// If persisting the flow action failed, then we set another trigger
for this event to occur immediately to
// re-attempt handling the event
+ DagActionStore.DagAction updatedFlowAction =
DagActionStore.DagAction.updateFlowExecutionId(flowAction,
+ leaseObtainedStatus.getEventTimeMillis());
scheduleReminderForEvent(jobProps,
- new MultiActiveLeaseArbiter.LeasedToAnotherStatus(flowAction,
leaseObtainedStatus.getEventTimestamp(), 0L),
+ new
MultiActiveLeaseArbiter.LeasedToAnotherStatus(updatedFlowAction, 0L),
Review Comment:
Yes we do, the timestamp within `leaseObtainedStatus` is the agreed upon
time that is synchronized across all hosts while the following param,
`eventTimeMillis` (later called) `triggerEventTimeMillis` is local to the host
and is only used for logging purposes to show us that we switch from local
trigger to the synchronized trigger. Here we are just changing where it's being
stored not the fact that we do update it.
Issue Time Tracking
-------------------
Worklog Id: (was: 885741)
Time Spent: 2h 40m (was: 2.5h)
> Improve Logs & Metrics around Multi-active Launch Handling
> ----------------------------------------------------------
>
> Key: GOBBLIN-1930
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1930
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Improve logging and metrics around multi-active launch flow event handling to
> identify any missing events between the {{MysqlMultiActiveLeaseArbiter}}
> committing the launch event to the {{dagActionStore}} and the
> {{DagActionMonitor}} receiving events for processing. We want to be able to
> distinguish between the following cases ofÂ
> * events that are never received by the {{DagActionMonitor}}
> * events incorrectly filtered out by the {{DagActionMonitor}}
> * any failed submissions of dags to the {{DagManager}} either upon leader
> change or from the {{DagActionChangeMonitor}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
