[
https://issues.apache.org/jira/browse/GOBBLIN-1968?focusedWorklogId=893093&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-893093
]
ASF GitHub Bot logged work on GOBBLIN-1968:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 30/Nov/23 08:25
Start Date: 30/Nov/23 08:25
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #3829:
URL: https://github.com/apache/gobblin/pull/3829#discussion_r1410315210
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/SafeDatasetCommit.java:
##########
@@ -316,6 +316,7 @@ private void
finalizeDatasetStateBeforeCommit(JobState.DatasetState datasetState
if (taskState.getWorkingState() != WorkUnitState.WorkingState.SUCCESSFUL
&& this.jobContext.getJobCommitPolicy() ==
JobCommitPolicy.COMMIT_ON_FULL_SUCCESS) {
// The dataset state is set to FAILED if any task failed and
COMMIT_ON_FULL_SUCCESS is used
+ log.info("Failed task state for " +
taskState.getWorkunit().getOutputFilePath());
Review Comment:
that's worth exploring. overall we'd like to capture specifically which
ones failed. if we show the particular error associated w/ each failure that's
a bonus!
Issue Time Tracking
-------------------
Worklog Id: (was: 893093)
Time Spent: 20m (was: 10m)
> Gobblin Commit Step Runs on Temporal
> ------------------------------------
>
> Key: GOBBLIN-1968
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1968
> Project: Apache Gobblin
> Issue Type: New Feature
> Components: gobblin-core
> Reporter: William Lo
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Temporal is the next-gen distributed computing platform Gobblin is using to
> replace MapReduce for many of its data movement workflows. We want to
> integrate Gobblin's commit step that tracks task states and reports their
> status with Temporal
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
