ZihanLi58 commented on code in PR #3656:
URL: https://github.com/apache/gobblin/pull/3656#discussion_r1130257877
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/scheduler/GobblinServiceJobScheduler.java:
##########
@@ -312,15 +380,21 @@ private void scheduleSpecsFromCatalog() {
try {
individualGetSpecStartTime = System.nanoTime();
Spec spec = this.flowCatalog.get().getSpecWrapper(uri);
- this.individualGetSpecSpeedNanosValue = System.nanoTime() -
individualGetSpecStartTime;
+ this.individualGetSpecSpeedValue = System.nanoTime() -
individualGetSpecStartTime;
+ totalGetTime += this.individualGetSpecSpeedValue;
addSpecHelperMethod(spec);
+ totalAddSpecTime += this.eachCompleteAddSpecValue; // this is
updated by each call to onAddSpec
+ actualNumFlowsScheduled += 1;
} catch (Exception e) {
// If there is an uncaught error thrown during compilation, log it
and continue adding flows
_log.error("Could not schedule spec uri {} from flowCatalog due to
", uri, e);
}
-
}
+ this.individualGetSpecSpeedValue = -1L;
+ this.totalGetSpecTimeValue = totalGetTime;
+ this.totalAddSpecTimeValue = totalAddSpecTime;
Review Comment:
You might want to add some todo lists here to remove the metrics once the
benchmark is done. For example, I don't think we want to report
totalAddSpecTimeValue as metrics, a log makes more sense to me as we just set
this value once and never change it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
