[
https://issues.apache.org/jira/browse/GOBBLIN-2066?focusedWorklogId=919212&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-919212
]
ASF GitHub Bot logged work on GOBBLIN-2066:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 13/May/24 22:46
Start Date: 13/May/24 22:46
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #3912:
URL: https://github.com/apache/gobblin/pull/3912#discussion_r1599172058
##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/workflow/impl/ProcessWorkUnitsWorkflowImpl.java:
##########
@@ -64,14 +66,14 @@ private int performWork(WUProcessingSpec workSpec) {
);
if (workunitsProcessed > 0) {
CommitStepWorkflow commitWorkflow = createCommitStepWorkflow();
- int result = commitWorkflow.commit(workSpec);
- if (result == 0) {
+ CommitGobblinStats result = commitWorkflow.commit(workSpec);
+ if (result.getNumCommittedWorkUnits() == 0) {
log.warn("No work units committed at the job level. They could have
been committed at the task level.");
}
return result;
} else {
log.error("No work units processed, so no commit attempted.");
- return 0;
+ return new CommitGobblinStats(new HashMap<>(), 0);
Review Comment:
did I see this earlier as well? I'd probably encapsulate within a static,
for use as:
```
return CommitStats.createEmpty();
```
Issue Time Tracking
-------------------
Worklog Id: (was: 919212)
Time Spent: 1h (was: 50m)
> Add Dataset level metrics in Temporal
> -------------------------------------
>
> Key: GOBBLIN-2066
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2066
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: William Lo
> Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Temporal workflows can have added observability metrics so that workflows can
> be easily understood at-a-glance which is an improvement over the current
> Gobblin system.
> We want to provide the following:
> 1. Emit dataset-level metrics on job metadata as a GobblinTrackingEvent (to
> reach feature parity with existing Gobblin)
> 2. Enhance return types on Temporal so that users and service operators can
> easily view metadata on jobs being run, so that it becomes obvious when a job
> actually commits work and to which datasets without checking the logs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
