Hi Guodong, Thanks a ton for finding and fixing the bug! If you can submit the patch through Apache review board <https://reviews.apache.org> (you might need to sign up for an Apache account), we can get it committed asap!
If you are using "git", you could use "support/post-reviews.py" in our repo to help you submit the patch (or you can just manually upload the diff to review board). Let me know if you have any questions/need help. On Wed, Apr 17, 2013 at 12:09 AM, 王国栋 <[email protected]> wrote: > hi, > > When I am trying to use fairscheduler as the low level scheduler for > hadoop, when I submit a job, I got NULL pointer exception in jobtracker. > > I go through the hadoop code, and I think it is a bug in MesosScheduler, > Because when the job is submitted to jobtracker, the status of job is PREP, > and at this time, fairscheduler will not schedule any task of this job, so > > // Let the underlying task scheduler do the actual task scheduling. > List<Task> tasks = taskScheduler.assignTasks(taskTracker); > tasks will be null. And then the following exception happnes. > > 13/04/17 11:30:14 INFO ipc.Server: IPC Server handler 8 on 9011, call > heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@5f0704e1, false, > false, true, 0) from 10.47.6.24:32978: error: java.io.IOException: > java.lang.NullPointerException > java.io.IOException: java.lang.NullPointerException > at > > org.apache.hadoop.mapred.MesosScheduler.assignTasks(MesosScheduler.java:229) > > > Also, I modify MesosScheduler a little bit, and it seems that the bug is > fixed. > the diff is as follow > > --- a/hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java > +++ b/hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java > @@ -224,13 +224,14 @@ public class MesosScheduler extends TaskScheduler > implements Scheduler { > // Let the underlying task scheduler do the actual task scheduling. > List<Task> tasks = taskScheduler.assignTasks(taskTracker); > > - // Keep track of which TaskTracker contains which tasks. > - for (Task task : tasks) { > - LOG.info("Assigning task : " + task.getTaskID() > - + " to tracker " + tracker); > - mesosTrackers.get(tracker).hadoopTasks.add(task.getTaskID()); > + if(task != null) { > + // Keep track of whenich TaskTracker contains which tasks. > + for (Task TASK_LOSTTRACKER_JVM_HEAP : tasks) { > + LOG.info("Assigning task : " + task.getTaskID() > + + " to tracker " + tracker); > + mesosTrackers.get(tracker).hadoopTasks.add(task.getTaskID()); > + } > } > - > return tasks; > } > > Does anybody can help me to identify this bug? > Thanks a lot. > > Guodong >
