[
https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-1740:
-------------------------------
Target Version/s: 1.1.0
This should be fixed for 1.x as it is pretty serious, and can bring the JT
scheduler down on its knees.
One strace:
{code}
INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8021, call
heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus, false, false, true, ...)
from xyzzy:abcdef: error: java.io.IOException: java.lang.NullPointerException
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1676)
at
org.apache.hadoop.mapred.JobInProgress.getLocalityLevel(JobInProgress.java:3354)
at
org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1752)
at
org.apache.hadoop.mapred.JobInProgress.obtainNewMapTask(JobInProgress.java:1341)
at
org.apache.hadoop.mapred.JobSchedulable.assignTask(JobSchedulable.java:142)
at
org.apache.hadoop.mapred.PoolSchedulable.assignTask(PoolSchedulable.java:160)
at
org.apache.hadoop.mapred.FairScheduler.assignTasks(FairScheduler.java:440)
at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3226)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
{code}
> NPE in getMatchingLevelForNodes when node locations are variable depth
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-1740
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.3
> Reporter: Todd Lipcon
> Attachments: mapreduce-1740.txt
>
>
> In getMatchingLevelForNodes, we assume that both nodes have the same "depth"
> (ie number of path components). If the user provides a topology script that
> assigns one node a path like /foo/bar/baz and another node a path like
> /foo/blah, this function will throw an NPE.
> I'm not sure if there are other places where we assume that all node
> locations have a constant number of paths. If so we should check the output
> of the topology script aggressively to be sure this is the case. Otherwise I
> think we simply need to add && n2 != null to the while loop
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
