[
https://issues.apache.org/jira/browse/GOBBLIN-1996?focusedWorklogId=904678&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-904678
]
ASF GitHub Bot logged work on GOBBLIN-1996:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 13/Feb/24 01:10
Start Date: 13/Feb/24 01:10
Worklog Time Spent: 10m
Work Description: homatthew commented on code in PR #3865:
URL: https://github.com/apache/gobblin/pull/3865#discussion_r1487017070
##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/workflows/metrics/TemporalEventTimer.java:
##########
@@ -70,7 +70,9 @@ private static Instant getCurrentTime() {
}
public static class Factory {
- private static final ActivityOptions DEFAULT_OPTS =
ActivityOptions.newBuilder().build();
+ private static final ActivityOptions DEFAULT_OPTS =
ActivityOptions.newBuilder()
+ .setStartToCloseTimeout(Duration.ofHours(24))
Review Comment:
Correct. the job itself can still run for > 24 hours. But if for some reason
kafka is down, this operation can take an arbitrary amount of time.
There are 2 things to consider here:
1. GaaS won't be able to detect if a job has finished until this activity
succeeds. If Kafka is down, we want to be able to retry this task until kafka
is back.
2. If the job itself is completed but this timer is unable to send the
message, I think 24 hours is a reasonable amount of time to retry for before
giving up. We ideally don't want to have to retry the work just because the
timer failed for a brief period of time (e.g. a few hours of downtime)
Issue Time Tracking
-------------------
Worklog Id: (was: 904678)
Time Spent: 4h (was: 3h 50m)
> Add ability for Yarn app to terminate on finishing of temporal flow
> -------------------------------------------------------------------
>
> Key: GOBBLIN-1996
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1996
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Matthew Ho
> Priority: Major
> Time Spent: 4h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
