[
https://issues.apache.org/jira/browse/GOBBLIN-2022?focusedWorklogId=912900&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-912900
]
ASF GitHub Bot logged work on GOBBLIN-2022:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 03/Apr/24 19:52
Start Date: 03/Apr/24 19:52
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #3896:
URL: https://github.com/apache/gobblin/pull/3896#discussion_r1550342146
##########
gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/KafkaJobStatusMonitor.java:
##########
@@ -236,8 +238,9 @@ protected void
processMessage(DecodeableKafkaRecord<byte[],byte[]> message) {
* It fills missing fields in job status and also merge the fields with the
* existing job status in the state store. Merging is required because we
* do not want to lose the information sent by other GobblinTrackingEvents.
- * Returns false if adding this state transitions the job status of the job
to final, otherwise returns false.
- * It will also return false if the job status was already final before
calling this method.
+ * Returns an absent Optional if adding this state transitions the job
status of the job to final, otherwise returns
+ * the updated job status wrapped inside an Optional.
+ * It will also return an absent Optional if the job status was already
final before calling this method.
Review Comment:
"exactly-once guarantee" might be clearest way to phrase.
with not choosing "at-least-once", we must consider what happens if this
method returns (and did successfully update the JobStateStore), but we fail
before completing the update to the DagActionStore.
seems the reevaluate action would be lost. in fact, it would be lost even
if there is never another GTE that might cause the KJSM to work again on this
same job's state.
what can we do to prevent?
Issue Time Tracking
-------------------
Worklog Id: (was: 912900)
Time Spent: 13h 20m (was: 13h 10m)
> create dag proc for taking actions on job completion
> ----------------------------------------------------
>
> Key: GOBBLIN-2022
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2022
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Arjun Singh Bora
> Priority: Major
> Time Spent: 13h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
