Is this file present on your local FS? file:///data1/home/ltang01/mf/prep/in/toy.txt
On Mon, Oct 15, 2012 at 3:43 PM, lei tang <[email protected]> wrote: > Hi, > > I'm using Pig for my daily job. Pig dose a good job in mapreduce mode in > our internal hadoop cluster. But it signals an error whenever I want to > run it in local mode. I believe this is due to some permission or config > issue. Does anybody know how I can address this problem? Any idea is > appreciated. > > [ltang01@stg-trgt00 in]$ pig -x local > 2012-10-11 17:29:25,707 [main] INFO org.apache.pig.Main - Apache Pig > version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12 > 2012-10-11 17:29:25,708 [main] INFO org.apache.pig.Main - Logging error > messages to: /somewhere/in/pig_1350001765703.log > 2012-10-11 17:29:26,068 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting > to hadoop file system at: file:/// > grunt> A = load 'toy.txt'; > grunt> dump A; > 2012-10-11 17:29:36,056 [main] INFO > org.apache.pig.tools.pigstats.ScriptState - Pig features used in the > script: UNKNOWN > 2012-10-11 17:29:36,218 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - > File concatenation threshold: 100 optimistic? false > 2012-10-11 17:29:36,249 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size before optimization: 1 > 2012-10-11 17:29:36,250 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size after optimization: 1 > 2012-10-11 17:29:36,382 [main] INFO > org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added > to the job > 2012-10-11 17:29:36,403 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 > 2012-10-11 17:29:36,445 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - Setting up single store job > 2012-10-11 17:29:36,486 [main] INFO > org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with > processName=JobTracker, sessionId= > 2012-10-11 17:29:36,493 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 1 map-reduce job(s) waiting for submission. > 2012-10-11 17:29:36,995 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 0% complete > 2012-10-11 17:29:37,002 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - job null has failed! Stop running all dependent jobs > 2012-10-11 17:29:37,002 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 100% complete > 2012-10-11 17:29:37,007 [main] ERROR > org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to > recreate exception from backend error: ENOENT: No such file or directory > at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) > at > > org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:513) > at > org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:325) > at > org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189) > at > > org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126) > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:873) > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:867) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:416) > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1109) > at > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:867) > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:841) > at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378) > at > > org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247) > at > org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279) > at java.lang.Thread.run(Thread.java:679) > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260) > > 2012-10-11 17:29:37,007 [main] ERROR > org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed! > 2012-10-11 17:29:37,007 [main] INFO > org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. Stats > reported below may be incomplete > 2012-10-11 17:29:37,009 [main] INFO > org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics: > > HadoopVersion PigVersion UserId StartedAt FinishedAt > Features > 0.20.2-dev 0.10.0 ltang01 2012-10-11 17:29:36 2012-10-11 > 17:29:37 UNKNOWN > > Failed! > > Failed Jobs: > JobId Alias Feature Message Outputs > N/A A MAP_ONLY Message: ENOENT: No such file or directory > at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) > at > > org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:513) > at > org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:325) > at > org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189) > at > > org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126) > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:873) > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:867) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:416) > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1109) > at > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:867) > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:841) > at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378) > at > > org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247) > at > org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279) > at java.lang.Thread.run(Thread.java:679) > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260) > file:/tmp/temp-916111882/tmp1833113178, > > Input(s): > Failed to read data from "file:///data1/home/ltang01/mf/prep/in/toy.txt" > > Output(s): > Failed to produce result in "file:/tmp/temp-916111882/tmp1833113178" > > Job DAG: > null > > > 2012-10-11 17:29:37,009 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Failed! > 2012-10-11 17:29:37,015 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 1066: Unable to open iterator for alias A > Details at logfile: /data1/home/ltang01/mf/prep/in/pig_1350001765703.log >
