Yes. it is there. Tried the full path as well, still not working. On Mon, Oct 15, 2012 at 3:45 PM, Prashant Kommireddi <[email protected]>wrote:
> Is this file present on your local FS? > file:///data1/home/ltang01/mf/prep/in/toy.txt > > On Mon, Oct 15, 2012 at 3:43 PM, lei tang <[email protected]> wrote: > > > Hi, > > > > I'm using Pig for my daily job. Pig dose a good job in mapreduce mode in > > our internal hadoop cluster. But it signals an error whenever I want to > > run it in local mode. I believe this is due to some permission or config > > issue. Does anybody know how I can address this problem? Any idea is > > appreciated. > > > > [ltang01@stg-trgt00 in]$ pig -x local > > 2012-10-11 17:29:25,707 [main] INFO org.apache.pig.Main - Apache Pig > > version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12 > > 2012-10-11 17:29:25,708 [main] INFO org.apache.pig.Main - Logging error > > messages to: /somewhere/in/pig_1350001765703.log > > 2012-10-11 17:29:26,068 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > Connecting > > to hadoop file system at: file:/// > > grunt> A = load 'toy.txt'; > > grunt> dump A; > > 2012-10-11 17:29:36,056 [main] INFO > > org.apache.pig.tools.pigstats.ScriptState - Pig features used in the > > script: UNKNOWN > > 2012-10-11 17:29:36,218 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - > > File concatenation threshold: 100 optimistic? false > > 2012-10-11 17:29:36,249 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > > - MR plan size before optimization: 1 > > 2012-10-11 17:29:36,250 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > > - MR plan size after optimization: 1 > > 2012-10-11 17:29:36,382 [main] INFO > > org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added > > to the job > > 2012-10-11 17:29:36,403 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > > - mapred.job.reduce.markreset.buffer.percent is not set, set to default > 0.3 > > 2012-10-11 17:29:36,445 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > > - Setting up single store job > > 2012-10-11 17:29:36,486 [main] INFO > > org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with > > processName=JobTracker, sessionId= > > 2012-10-11 17:29:36,493 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > > - 1 map-reduce job(s) waiting for submission. > > 2012-10-11 17:29:36,995 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > > - 0% complete > > 2012-10-11 17:29:37,002 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > > - job null has failed! Stop running all dependent jobs > > 2012-10-11 17:29:37,002 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > > - 100% complete > > 2012-10-11 17:29:37,007 [main] ERROR > > org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to > > recreate exception from backend error: ENOENT: No such file or directory > > at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) > > at > > > > > org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:513) > > at > > > org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:325) > > at > > org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189) > > at > > > > > org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126) > > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:873) > > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:867) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:416) > > at > > > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1109) > > at > > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:867) > > at > org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:841) > > at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378) > > at > > > > > org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247) > > at > > org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279) > > at java.lang.Thread.run(Thread.java:679) > > at > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260) > > > > 2012-10-11 17:29:37,007 [main] ERROR > > org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed! > > 2012-10-11 17:29:37,007 [main] INFO > > org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. Stats > > reported below may be incomplete > > 2012-10-11 17:29:37,009 [main] INFO > > org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics: > > > > HadoopVersion PigVersion UserId StartedAt FinishedAt > > Features > > 0.20.2-dev 0.10.0 ltang01 2012-10-11 17:29:36 2012-10-11 > > 17:29:37 UNKNOWN > > > > Failed! > > > > Failed Jobs: > > JobId Alias Feature Message Outputs > > N/A A MAP_ONLY Message: ENOENT: No such file or > directory > > at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) > > at > > > > > org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:513) > > at > > > org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:325) > > at > > org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189) > > at > > > > > org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126) > > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:873) > > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:867) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:416) > > at > > > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1109) > > at > > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:867) > > at > org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:841) > > at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378) > > at > > > > > org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247) > > at > > org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279) > > at java.lang.Thread.run(Thread.java:679) > > at > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260) > > file:/tmp/temp-916111882/tmp1833113178, > > > > Input(s): > > Failed to read data from "file:///data1/home/ltang01/mf/prep/in/toy.txt" > > > > Output(s): > > Failed to produce result in "file:/tmp/temp-916111882/tmp1833113178" > > > > Job DAG: > > null > > > > > > 2012-10-11 17:29:37,009 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > > - Failed! > > 2012-10-11 17:29:37,015 [main] ERROR org.apache.pig.tools.grunt.Grunt - > > ERROR 1066: Unable to open iterator for alias A > > Details at logfile: /data1/home/ltang01/mf/prep/in/pig_1350001765703.log > > >
