----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10920/#review20327 -----------------------------------------------------------
hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/10920/#comment41868> I don't see how this is possible, this code builds a list of completed tasks for the job, it must be part of the job here or we're doing something very wrong..? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/10920/#comment41874> The logic inside this loop still needs to happen, but it doesn't make sense any longer for it to be inside the loop. We used to do things at the task level, here you're changing it to be at the job level. If you look at where status is used, it's only used for logging: LOG.info("Removing completed task : " + status.getTaskID() + " of tracker " + status.getTaskTracker()); So effectively, you can kill the TaskStatus loop, make sense? Am I missing something? hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java <https://reviews.apache.org/r/10920/#comment41869> Mesos doesn't have the concept of "jobs" so s/hadoopJobs/jobs/ works :) - Ben Mahler On May 7, 2013, 7 p.m., Brenden Matthews wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/10920/ > ----------------------------------------------------------- > > (Updated May 7, 2013, 7 p.m.) > > > Review request for mesos. > > > Description > ------- > > From 5b29cf70eb7b6ee0f45148668ed8848bf9a941a4 Mon Sep 17 00:00:00 2001 > From: Brenden Matthews <[email protected]> > Date: Fri, 26 Apr 2013 11:42:18 -0700 > Subject: [PATCH 10/19] Don't prematurely kill TaskTrackers. > > We assign the JobID to the internal Mesos tracker (for the TaskTracker) > rather than the TaskID. > > In the case where a TaskTracker has only map tasks assigned to it and > they have all completed, we mustn't terminate the tracker until the > entire job has finished, including the map tasks. > --- > .../org/apache/hadoop/mapred/MesosScheduler.java | 371 > ++++++++++---------- > 1 file changed, 189 insertions(+), 182 deletions(-) > > > Diffs > ----- > > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java afe401f > > Diff: https://reviews.apache.org/r/10920/diff/ > > > Testing > ------- > > Used in production at airbnb. > > > Thanks, > > Brenden Matthews > >
