sv2000 commented on a change in pull request #3365:
URL: https://github.com/apache/gobblin/pull/3365#discussion_r689810471
##########
File path:
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/GobblinHelixTaskStateTracker.java
##########
@@ -61,7 +61,9 @@ public GobblinHelixTaskStateTracker(Properties properties) {
@Override
public void registerNewTask(Task task) {
try {
- this.scheduledReporters.put(task.getTaskId(),
scheduleTaskMetricsUpdater(new TaskMetricsUpdater(task), task));
+ if (GobblinMetrics.isEnabled(task.getTaskState().getWorkunit())) {
+ this.scheduledReporters.put(task.getTaskId(),
scheduleTaskMetricsUpdater(new TaskMetricsUpdater(task), task));
Review comment:
I think a better approach might be to not create one metrics updater per
task, and create a single updater instead, which iterates over all the tasks
and updates records/bytes written.
##########
File path:
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/GobblinHelixTaskStateTracker.java
##########
@@ -61,7 +61,9 @@ public GobblinHelixTaskStateTracker(Properties properties) {
@Override
public void registerNewTask(Task task) {
try {
- this.scheduledReporters.put(task.getTaskId(),
scheduleTaskMetricsUpdater(new TaskMetricsUpdater(task), task));
+ if (GobblinMetrics.isEnabled(task.getTaskState().getWorkunit())) {
+ this.scheduledReporters.put(task.getTaskId(),
scheduleTaskMetricsUpdater(new TaskMetricsUpdater(task), task));
Review comment:
Using the global "metrics.enabled" key to turn off task metrics
reporting seems too heavy. This global config will turn off other reporting as
well, e.g. GTEs, and other metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
