[
https://issues.apache.org/jira/browse/HADOOP-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718268#action_12718268
]
Sharad Agarwal commented on HADOOP-5931:
----------------------------------------
bq. I am assuming the moving window mechanism would be flexible enough to add
new bucket sizes as required.
Yes. I am planning to use and extend metric framework available in core, thru
which custom window/bucket sizes can be defined.
bq. Regarding having the computation on the tasktracker, and reporting the
status via status, one problem is that if we want to change the bucket size, it
would involve a change in the status object.
To avoid that, instead of above fields, we can have say List<MetricInfo>
metrics field in TaskTrackerStatus where MetricInfo could be:
class MetricInfo {
String name;
int tasks;
int tasksSucceeded;
}
Here name would be the name of the metrics. e.q. "lasthour", "lastday" etc.
which could be configured in the metrics property file.
bq. Also, one requirement for this is to store this information on the
JobTracker. Can you describe how this will be stored, mechanics with respect to
lost tasktrackers etc ?
Currently jobtracker doesn't store any information about lost tasktrackers.
Storing info about lost trackers is not trivial and demands a separate jira
issue. Consider the case of tracker getting lost and never coming back or
coming back at different port. The jobtracker data structures need to be
cleaned up for such trackers otherwise those data structures would be lying
forever.
bq. Will this information be available if the JobTracker restarts ?
Yes. Since this info is propagated from Tasktracker, it would be available
after jobtracker restarts.
> Collect information about number of tasks succeeded / total per time unit for
> a tasktracker.
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-5931
> URL: https://issues.apache.org/jira/browse/HADOOP-5931
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
>
> Collecting information of number of tasks succeeded / total per tasktracker
> and being able to see these counts per hour, day and since start time will
> help reason about things like the blacklisting strategy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
