[
https://issues.apache.org/jira/browse/MAPREDUCE-746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729570#action_12729570
]
Iyappan Srinivasan commented on MAPREDUCE-746:
----------------------------------------------
The job tracker logs just before this null pointer exception. I have replaced
teh anme of killed task tracker server name with this string "<killed TT>"
"2009-07-10 07:51:43,626 INFO org.apache.hadoop.mapred.JobTracker: Adding
<killed TT> to the blacklist across all jobs
2009-07-10 07:51:43,628 INFO org.apache.hadoop.mapred.JobTracker: Blacklisting
tracker : <killed TT> Reason for blacklisting is : EXCEEDING_FAILURES
2009-07-10 07:51:43,631 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0
on 9020, call heartbeat(org.apache.hadoop.mapred.tasktrackersta...@1fb050c,
false, false, true, 269) from 98.136.99.54:55815: error: java.io.IOException:
java.lang.NullPointerException
java.io.IOException: java.lang.NullPointerException"
> When a task tracker is killed, there is a Null Pointer exception thrown.
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-746
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-746
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Iyappan Srinivasan
> Priority: Minor
>
> When a task tracker is killed, the job completes. But tehre is a null pointer
> exception thrown:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.JobTracker$FaultyTrackersInfo.removeHostCapacity(JobTracker.java:759)
> at
> org.apache.hadoop.mapred.JobTracker$FaultyTrackersInfo.blackListTracker(JobTracker.java:624)
> at
> org.apache.hadoop.mapred.JobTracker$FaultyTrackersInfo.incrementFaults(JobTracker.java:601)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:2337)
> at
> org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2998)
> at
> org.apache.hadoop.mapred.JobInProgress.jobComplete(JobInProgress.java:2584)
> at
> org.apache.hadoop.mapred.JobInProgress.completedTask(JobInProgress.java:2473)
> at
> org.apache.hadoop.mapred.JobInProgress.updateTaskStatus(JobInProgress.java:1047)
> at
> org.apache.hadoop.mapred.JobTracker.updateTaskStatuses(JobTracker.java:3867)
> at
> org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:3079)
> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2817)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:964)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:960)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:958)
> Steps to reproduce the issue:
> 1) Bring up a 5 node cluster.
> 2) set mapred.max.tracker.failures to 1
> 3) Run a sleep command with 5 maps and 5 redcues.
> 4) Kill a task tracker, when map is 80% complete.
> 5) Kill the task tracker, by using "kill -9 process_id".
> 6) At the time of killing, it was running 2 maps and 2 reduces.
> 7) Allow 12 minutes to elapse for that task tracker to go out of job nodes
> list
> 8) Then job completes successfully by giving the task attempts of the lost
> trackers to other nodes.
> 9) The job tracker logs has this exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
