The directory it's trying to create is on the local file system of a node (it's temp storage), not in hdfs. Do you have /rawfiles/ set up as temp storage for Hadoop?
-D On Mon, Jan 31, 2011 at 10:29 AM, Kris Coward <[email protected]> wrote: > > So I have a relation apa which when DUMPed, ends up getting output just > fine, but when I run > STORE apa INTO '/rawfiles/f3453efd460348bbaeee2e9496e25871/1294311600/apa' > USING PigStorage(','); > I get the following error: > > java.io.IOException: Mkdirs failed to create > > file:/rawfiles/f3453efd460348bbaeee2e9496e25871/1294311600/apa/_temporary/_attempt_local_0007_m_000000_0 > at > org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:367) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:507) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:414) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:406) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:177) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:96) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:80) > at > org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.<init>(MapTask.java:624) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:352) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176) > 2011-01-31 16:42:31,999 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Submitting job: job_local_0007 to execution engine. > 2011-01-31 16:42:32,501 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 0% complete > 2011-01-31 16:42:37,021 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 100% complete > 2011-01-31 16:42:37,021 [main] ERROR > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 1 map reduce job(s) failed! > 2011-01-31 16:42:37,021 [main] ERROR > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Failed to produce result in: > "file:/rawfiles/f3453efd460348bbaeee2e9496e25871/1294311600/apa" > 2011-01-31 16:42:37,022 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Failed! > > I've manually created the directory > /rawfiles/f3453efd460348bbaeee2e9496e25871/1294311600/apa > from within grunt to verify that it wasn't a permissions problem (and > then removed apa so that STORE wouldn't fail on account of the directory > already existing), and the error persists. > > Any advice on what might be causing this problem? > > Thanks, > Kris > > -- > Kris Coward http://unripe.melon.org/ > GPG Fingerprint: 2BF3 957D 310A FEEC 4733 830E 21A4 05C7 1FEB 12B3 >
