I changed the mapred.job.reuse.jvm.num.tasks property from -1 to 1 using the option
-D mapred.job.reuse.jvm.num.tasks=1 when firing the terasort job and it worked. Not sure why ? This means that hadoop-log-dir permissions isn't an issue. From: Vinod Kumar Vavilapalli [mailto:vino...@hortonworks.com] Sent: Thursday, August 30, 2012 11:10 AM To: hdfs-user@hadoop.apache.org Subject: Re: terasort fails with error It seems like some of your nodes don't have correct permissions on the hadoop-log-dir, you should check that. It may be only a subset of nodes as you mentioned that Teragen seems to work. You can start with the nodes where the tasks were reported as failed. HTH, +Vinod On Aug 30, 2012, at 10:08 AM, Jain, Prem wrote: Folks, I am trying to terasort and seems to fail with the following error ? Teragen seems to work well though 12/08/30 09:53:38 WARN mapred.JobClient: Error reading taskoutputhttp://datanode004.svl.ibm.com:50060/tasklog?plaintext=true&attemptid=attempt_201208300948_0001_m_000060_1&filter=stdout 12/08/30 09:53:38 WARN mapred.JobClient: Error reading task outputhttp:// datanode004.svl.ibm.com:50060/tasklog?plaintext=true&attemptid=attempt_201208300948_0001_m_000060_1&filter=stderr 12/08/30 09:53:39 INFO mapred.JobClient: Task Id : attempt_201208300948_0001_m_000152_1, Status : FAILED ENOENT: No such file or directory at org.apache.hadoop.io.nativeio.NativeIO.open(Native Method) at org.apache.hadoop.io.SecureIOUtils.createForWrite(SecureIOUtils.java:161) at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:296) at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:369) at org.apache.hadoop.mapred.Child.main(Child.java:229 The command I am running is: $ hadoop jar $HADOOP_HOME/hadoop-examples-1.0.0.jar terasort /user/netapp/input-dir /user/netapp/output-dir Thanks in advance Prem
