[
https://issues.apache.org/jira/browse/GOBBLIN-1639?focusedWorklogId=766320&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-766320
]
ASF GitHub Bot logged work on GOBBLIN-1639:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 04/May/22 22:24
Start Date: 04/May/22 22:24
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3500:
URL: https://github.com/apache/gobblin/pull/3500#discussion_r865432706
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/AbstractJobLauncher.java:
##########
@@ -685,8 +674,23 @@ public void apply(JobListener jobListener, JobContext
jobContext)
}
}
} finally {
- // Stop metrics reporting
+ // Register metrics then stop metrics reporting, metrics will flush and
emit when the metricsContext is closed
if (this.jobContext.getJobMetricsOptional().isPresent()) {
+ if
(jobState.getPropAsBoolean(ConfigurationKeys.GOBBLIN_OUTPUT_JOB_LEVEL_METRICS,
true)) {
+ String workunitCreationGaugeName =
MetricRegistry.name(ServiceMetricNames.GOBBLIN_JOB_METRICS_PREFIX,
+ TimingEvent.LauncherTimings.WORK_UNITS_CREATION,
jobState.getJobName());
+ long workUnitsCreationTime = workUnitsCreationTimer.getDuration() /
TimeUnit.SECONDS.toMillis(1);
Review Comment:
Yeah there might be a better method of handling the metrics rather in
finally, but I do think it is important to always emit metrics for each
execution of a pipeline hence why trying to consolidate them all within a
finally block. Maybe the solution here is to add yet another `try` in the
`finally`? Reason being is that users should also have visibility when their
workflow does not perform work, rather than not receiving information at all
which would not activate alerting.
Issue Time Tracking
-------------------
Worklog Id: (was: 766320)
Time Spent: 1h (was: 50m)
> Remove adhoc job metric emission and also provide a more uniform way of
> emitting per pipeline metrics on Gobblin
> ----------------------------------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1639
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1639
> Project: Apache Gobblin
> Issue Type: Task
> Components: gobblin-service
> Reporter: William Lo
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Gobblin flows still emit certain metrics by default.
> 1. GaaS flows which are adhoc should not be emitting metrics outside of GTE
> 2. Per pipeline metrics are too disorganized right now, need to unify them
> and this can be the first step.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
