Hi Matthias, Best I can guess, you have uneven permissions on some of your mapred.local.dir, causing tasks that run using those directories to fail. See if these are all owned by user:group 'mapred:hadoop', and have drwxr-xr-x permissions.
Regards, - Patrick On Tue, Oct 26, 2010 at 3:34 AM, Mathias Walter <mathias.wal...@gmx.net>wrote: > Hi Guys, > > recently I upgraded to the recent Claudera Hadoop distribution. It contains > hadoop-core-0.20.2+737.jar. If I now run my map job, I > get the following exception for a few tasks: > > java.io.IOException: Owner 'mapred' for path > > /hadoop/hdfs5/tmp/taskTracker/mathias.walter/jobcache/job_201010210928_0005/attempt_201010210928_0005_m_000000_0/output/spill437.out > .index did not match expected owner 'mathias.walter' > at > org.apache.hadoop.io.SecureIOUtils.checkStat(SecureIOUtils.java:182) > at > org.apache.hadoop.io.SecureIOUtils.openForRead(SecureIOUtils.java:108) > at org.apache.hadoop.mapred.SpillRecord.<init>(SpillRecord.java:62) > at org.apache.hadoop.mapred.SpillRecord.<init>(SpillRecord.java:55) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1480) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1172) > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:574) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:641) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:315) > at org.apache.hadoop.mapred.Child$4.run(Child.java:217) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063) > at org.apache.hadoop.mapred.Child.main(Child.java:211) > > A total of 8 tasks are running in parallel. They are finished after about 8 > hours, but some of them (19) were crashed with the above > exception. > > Why are so many tasks crashed, but some not? > > -- > Kind regards, > Mathias > >
