Merto, You seem to be looking for the speculative execution code?
For MR1, in 1.x or 0.20.x and below, you can find that in the JobInProgress#findNewMapTask method (go down to the bit which does speculative execution logic for maps and reduces depending on if the job enabled it for itself). For MR2, in 2.x and above, since it offers a way to have pluggable speculative algorithms, the default implementation (a slightly advanced version of the one in 1.0) is available in the DefaultSpeculator class (under the mapreduce-client-app package). This is used by the MRAppMaster (and is event driven). If you are instead asking about the task timeout monitor (that kills tasks that haven't reported for a period): For MR1 it is in TaskTracker#markUnresponsiveTasks method (that uses timeout values set per task (from a job)). For MR2 similar code is in TaskHeartbeatHandler#PingChecker thread class. This is again in the mapreduce-client-app package. Preemption is a different operation than monitoring slow jobs/tasks. I believe you should be able to, with any decent IDE, follow the above entry points and then be able to drill up/down from there to analyze the code. In case you get stuck somewhere in understanding something around these features, feel free to ask us here! On Fri, Jun 8, 2012 at 9:14 PM, Merto Mertek <masmer...@gmail.com> wrote: > I am interested in which component of hadoop and how is managed slow task > execution. > > Is this a tasktracker, jobtracker or a scheduler? > > In fairscheduling (0.20.205) there exist a preemption mechanism but it just > handles min and fairshares and kills last submited tips.. > > Thanks. -- Harsh J
