Hi, To James.
this is the permission of hadoop.tmp.dir. $> ls -al drwxr-xr-x 6 juneng juneng 4096 5월 3 10:37 hadoop-juneng.2 To Harsh. yes, out cluster has 96 occupied reducer slots. and my job is using 90 reduce tasks at one time to complete it. thanks for all. Junyoung Kim ([email protected]) On 05/02/2011 08:32 PM, James Seigel wrote:
What are your permissions on your hadoop.tmp.dir ? James Sent from my mobile. Please excuse the typos. On 2011-05-02, at 1:26 AM, Jun Young Kim<[email protected]> wrote:hi, all. I got so many failures on a reducing step. see this error. java.io.IOException: Failed to delete earlier output of task: attempt_201105021341_0021_r_000001_0 at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:157) at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:173) at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:173) at org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:133) at org.apache.hadoop.mapred.OutputCommitter.commitTask(OutputCommitter.java:233) at org.apache.hadoop.mapred.Task.commit(Task.java:962) at org.apache.hadoop.mapred.Task.done(Task.java:824) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391) at org.apache.hadoop.mapred.Child$4.run(Child.java:217) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742) at org.apache.hadoop.mapred.C this error was happened after adopting MultipleTextOutputFormat class in my job. the job is producing thousands of different output files on a HDFS. anybody can guess reasons? thanks. -- Junyoung Kim ([email protected])
