-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11116/#review21326
-----------------------------------------------------------
What differences were you seeing between job.pendingMaps() and this new
technique?
Looking at JobInProgress.pendingMaps():
public synchronized int pendingMaps() {
return numMapTasks - runningMapTasks - failedMapTIPs -
finishedMapTasks + speculativeMapTasks;
}
vs. jobdetails_jsp.printTaskSummary() {
int totalTasks = tasks.length;
int runningTasks = 0;
int finishedTasks = 0;
int killedTasks = 0;
int failedTaskAttempts = 0;
int killedTaskAttempts = 0;
for(int i=0; i < totalTasks; ++i) {
TaskInProgress task = tasks[i];
if (task.isComplete()) {
finishedTasks += 1;
} else if (task.isRunning()) {
runningTasks += 1;
} else if (task.wasKilled()) {
killedTasks += 1;
}
failedTaskAttempts += task.numTaskFailures();
killedTaskAttempts += task.numKilledTasks();
}
int pendingTasks = totalTasks - runningTasks - killedTasks - finishedTasks;
...
}
It seems like the difference here might be between failed_ vs. killed_ and/or
the fact that the latter case uses speculativeMapTasks?
- Ben Mahler
On May 22, 2013, 7:49 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11116/
> -----------------------------------------------------------
>
> (Updated May 22, 2013, 7:49 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> From dc2c7f4116dec16bf0e22ab4c9fccac8f3a45020 Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <[email protected]>
> Date: Mon, 13 May 2013 15:27:45 -0700
> Subject: [PATCH 09/24] Fix TaskTracker pending tasks calculation.
>
> Review: https://reviews.apache.org/r/11116
> ---
> .../org/apache/hadoop/mapred/MesosScheduler.java | 33
> ++++++++++++++++++--
> 1 file changed, 31 insertions(+), 2 deletions(-)
>
>
> Diffs
> -----
>
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
> afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11116/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
