Robert Kanter created MAPREDUCE-4963:
Summary: StatisticsCollector improperly keeps track of Last Day
and Last Hour statistics for new TaskTrackers
Key: MAPREDUCE-4963
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4963
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv1
Affects Versions: 1.1.1
Reporter: Robert Kanter
Assignee: Robert Kanter
The StatisticsCollector keeps track of updates to the Total Tasks Last Day,
Succeed Tasks Last Day, Total Tasks Last Hour, and Succeeded Tasks Last
Hour per Task Tracker which is displayed on the JobTracker web UI. It uses
buckets to manage when to shift task counts from Last Hour to Last Day and
out of Last Day. After the JT has been running for a while, the connected
TTs will have the max number of buckets and will keep shifting them at each
update. If a new TT connects (or an old on rejoins), it won't have the max
number of buckets, but the code that drops the buckets uses the same counter
for all sets of buckets. This means that new TTs will prematurely drop their
buckets and the stats will be incorrect.
example:
# Max buckets is 5
# TaskTracker A has these values in its buckets [4, 2, 0, 3, 10] (i.e. 19)
# A new TaskTracker, B, connects; it has nothing in its buckets: [ ] (i.e. 0)
# TaskTracker B runs 3 tasks and TaskTracker A runs 5
# An update occurs
# TaskTracker A has [2, 0, 3, 10, 5] (i.e. 20)
# TaskTracker B should have [3] but it will drop that bucket after adding it
during the update and instead have [ ] again (i.e. 0)
# TaskTracker B will keep doing that forever and always show 0 in the web UI
We can fix this by not using the same counter for all sets of buckets
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
