Hi, Anyone can tell me what's the difference between "Killed Task Attempts" and "Killed Tasks"? I ran a big job (14820 maps and 0 reduces). In the job-details page, the web GUI reports 62 "killed task attempts". I'm assuming this is due to "speculative execution". Now when I go to the job-history page for the job, it reports 54 "killed tasks" (and 14820 successful map-tasks as expected).
A few questions: * Why 62 killed task attempts vs 54 killed tasks? * Under speculative execution, does hadoop launch a new MapTask with new task-id or does it just launch a new MapTaskAttempt with a new task-attempt-id? * When a MapTaskAttempt fails, and when hadoop tries to re-launch the MapTask, does it create a new task-id or just a new task-attempt-id? * Does 'mapred.map.max.attempts' include all attempts launched due to speculative-execution? Btw this job is basically a trivial no-op job - it just scans around 1TB of data and does nothing else in the map. I looked at the killed tasks' syslog output and I didn't see any errors. -- Harish Mallipeddi http://blog.poundbang.in
