Hi Zhixiong,
I haven't use configure reporter and haven't sought anything exceptions.
Looks like the TaskState.java responsible to collect metrics of counter type as
follows:
// Add task metrics
TaskMetrics taskMetrics = TaskMetrics.get(this);
MetricArray metricArray = new MetricArray();
for (Map.Entry<String, ? extends com.codahale.metrics.Metric> entry :
taskMetrics.getMetricContext().getCounters()
.entrySet()) {
Metric counter = new Metric();
counter.setGroup(MetricGroup.TASK.name());
counter.setName(entry.getKey());
counter.setType(MetricTypeEnum.valueOf(GobblinMetrics.MetricType.COUNTER.name()));
counter.setValue(Long.toString(((Counter) entry.getValue()).getCount()));
metricArray.add(counter);
}
Here the MetricContext object can be fetched but the counters size is 0, why
the metrics added by Counter from MetricContext can't be fetched on this
place? Very strange the gobblin definite metrics also haven't be fetched. But
it is normal under standalone mode.
If it is a bug or need to configure any other something for mapreduce mode?
Thanks,
Zhang Xiuzhu
From: Zhixiong Chen [mailto:[email protected]]
Sent: Tuesday, June 12, 2018 1:19 AM
To: [email protected]; [email protected]
Subject: Re: Gobblin Metrics
Hi Xiuzhu,
Which metric reporter are you using?
Did you see exceptions about sending metrics in mapreduce mode?
Zhixiong,
________________________________
From: Zhang, Xiuzhu(AWF) <[email protected]<mailto:[email protected]>>
Sent: Monday, June 11, 2018 2:19 AM
To: [email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Subject: Gobblin Metrics
Hi,
Concerning gobblin metrics function, I am encounter a strange issue. I write
some self-definite metrics using MetricContext object by InstrumentedExtractor
and InstrumentedDataWriter. Running it at standalone and mapreduce mode, after
job completed just only standalone have the metrics information in
gobblin_task_metrics table I don't understand why haven't the metrics under
mapreduce mode.
Have anyone know where need to care about it?
I use it as follows :
xxxExtractor extends InstrumentedExtractor<xxx, xxx> {
private MetricContext metricContext;
public xxxExtractor(WorkUnitState state){
this.metricContext=this.getMetricContext();
}
Private void xxx(){
Counter
counter=metricContext.counter("xx.xxx.xxx");
recordsCounter.inc(xxx);
}
}
Thanks,
Zhang Xiuzhu
