[
https://issues.apache.org/jira/browse/HADOOP-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720094#action_12720094
]
Sharad Agarwal commented on HADOOP-5931:
----------------------------------------
Had an off line discussion with Devaraj/Eric, the concern raised is that metric
context is an export interface and instead of using it, we should collect the
metrics natively in hadoop. Administrators should not be able to remove this
metric as it may in future used by Jobtracker to make decisions. Right?
Let me clarify a bit. Please note that only time windows are configured in the
metric properties, and not the actual metric name which gets collected. Also a
new context name is defined "tasktracker" (Refer hadoop-metrics.properties in
patch) . So it does not come in between the existing metric contexts. Those can
continue to be chukwa/ganglia etc.
If this doesn't sound like a good idea, I see few options:
1. Give a better name to the added context say "core-mapred", so that
administrators don't override it. It would serve only to add/remove time
windows.
2. Do not use Metrics api. Expose the time window configuration via
mapred-site.xml.
3. Don't expose the configuration at all and have fixed windows, say "last
hour" and "last day".
I went with extending the metrics API because I thought that it would help to
collect any other existing metrics in time windows without making much change
to the code. For example if we want to collect "mapred" metrics in time
windows, then "mapred" context can point to the Composite context, which can be
configured to use multiple contexts, one being time window context.
Thoughts?
> Collect information about number of tasks succeeded / total per time unit for
> a tasktracker.
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-5931
> URL: https://issues.apache.org/jira/browse/HADOOP-5931
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: Sharad Agarwal
> Fix For: 0.21.0
>
> Attachments: 5931_v1.patch, 5931_v2.patch
>
>
> Collecting information of number of tasks succeeded / total per tasktracker
> and being able to see these counts per hour, day and since start time will
> help reason about things like the blacklisting strategy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
