[
https://issues.apache.org/jira/browse/HADOOP-5761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710640#action_12710640
]
Todd Lipcon commented on HADOOP-5761:
-------------------------------------
I haven't been able to produce this bug (was responding to an email from Lance
on core-user), but if you make the assumption that a task can enter such a
state, it seems reasonable that the JT should send a Kill action for that task.
To me, this looks similar to HADOOP-5374 but not quite the same. The task
probably gets into COMMIT_PENDING state in the same way, but the manifestation
was a slightly different stack trace:
{code}
2009-04-30 02:34:40,215 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2
on 54311, call heartbeat(org.apache.hadoop.mapred.tasktrackersta...@1a93388,
false, true, 5341) from 10.253.134.191:42688: error: java.io.IOException:
java.lang.NullPointerException
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.mapred.JobTracker.getTasksToSave(JobTracker.java:2130)
at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:1923)
{code}
(from the original core-user email from Lance)
In the HADOOP-5374 stack trace, the tip variable in getTasksToSave was
non-null, and the NPE occured inside shouldCommit. In this case, tip is null.
Lance also reported this being as being on 19.1.
Unfortunately I don't know any way to reproduce this, but it seems logical that
if a task somehow enters a bad state (by means some unknown race condition) the
JT's interaction with the TT should restore a proper state by killing that task.
> JobTracker and TaskTracker enter infinite loop when TaskTracker reports bad
> taskid
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-5761
> URL: https://issues.apache.org/jira/browse/HADOOP-5761
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Todd Lipcon
>
> If the TaskTracker somehow gets into a state where it has a task in
> COMMIT_PENDING state that the JobTracker does not know about, the JobTracker
> will throw NPEs while processing heartbeats. Due to HADOOP-3987, this causes
> the JT and TT to enter an infinite heartbeat loop with no delays, and the TT
> fails to make progress.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
