speculation framework does not update job progress rate with time causing false
speculation
-------------------------------------------------------------------------------------------
Key: MAPREDUCE-2175
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2175
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Reporter: Joydeep Sen Sarma
When considering task speculation - the job's overall progress rate for the
given task type and the individual task's progress rate are compared. When
doing so - the task's current progress rate is considered. However the job's
overall progress rate is only only updated when a task's progress report is
received.
This causes weirdness if a particular task does not report for a while. The
job's progress rate is not updated - but the task's progress rate decays. For
single task jobs - it causes speculation almost inevitably.
We need to update the job's overall progress rate as the task's progress rate
changes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
