Hi All, I am using Hadoop 1.0.0.
I am trying to write a sample that shows how debug scripts works. To do that, I have written a map reduce job that always fail (by explicitly throwing an exception) see FaultyWordCount.java. I have setup the debug scripts as per http://developer.yahoo.com/hadoop/tutorial/module5.htmlWordcountWithDebugScript.java When I run the job, I get the following error when Hadoop try to run the debug script. -------------------- DEBUG OUT--------------------- bash: /Users/srinath/playground/hadoop-book/hadoop-1.0.0/fail-script: No such file or directory bash: line 0: exec: /Users/srinath/playground/hadoop-book/hadoop-1.0.0/fail-script: cannot execute: No such file or directory I looked at further, and error happens due to following error (found from task tracker logs) 2012-05-26 05:59:57,110 WARN org.apache.hadoop.mapred.TaskTracker: Working Directory of the task attempt_201205260558_0001_m_000000_0 doesnt exist. Caught exception org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/srinath/jobcache/job_201205260558_0001/attempt_201205260558_0001_m_000000_0/work in any of the configured local directories at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160) at org.apache.hadoop.mapred.TaskTracker$TaskInProgress.taskFinished(TaskTracker.java:2871) at org.apache.hadoop.mapred.TaskTracker$TaskInProgress.reportTaskFinished(TaskTracker.java:2772) at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:285) This happens in line 2871 of org.apache.hadoop.mapred.TaskTracker, where LocalDirAllocator.getLocalPathToRead( ..) throws an error. As I understood this happens due to work directory is missing from taskTracker/srinath/jobcache/job_201205260558_0001/attempt_201205260558_0001_m_000000_0/. Rest of the path exists. When is work directory created? is it always created? Any thoughts on what I am doing wrong? Thanks --Srinath -- ============================ Srinath Perera, Ph.D. Senior Software Architect, WSO2 Inc. Visiting Faculty, University of Moratuwa Member, Apache Software Foundation Research Scientist, Lanka Software Foundation Blog: http://srinathsview.blogspot.com/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902
FaultyWordCount.java
Description: Binary data
WordcountWithDebugScript.java
Description: Binary data
