Hi All,My script runs find in map reduce mode, but I get the following error
when I run it in the local mode. I have made sure that the i/p file exists. I
am not sure why map reduce is coming into picture when it is local mode.
pig -x local myscript.pig
2014-01-22 16:14:02,771 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File
concatenation threshold: 100 optimistic? false2014-01-22 16:14:02,805 [main]
INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 32014-01-22 16:14:02,806 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- Merged 2 map-only splittees.2014-01-22 16:14:02,806 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- Merged 2 out of total 3 MR operators.2014-01-22 16:14:02,806 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 12014-01-22 16:14:02,845 [main] INFO
org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to
the job2014-01-22 16:14:02,865 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default
0.32014-01-22 16:14:02,876 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Using reducer estimator:
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator2014-01-22
16:14:02,878 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
- BytesPerReducer=1000000000 maxReducers=999
totalInputFileSize=99408652014-01-22 16:14:02,878 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting Parallelism to 12014-01-22 16:14:02,909 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up multi store job2014-01-22 16:14:02,918 [main] INFO
org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will
not generate code.2014-01-22 16:14:02,918 [main] INFO
org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated
code to distributed cacche2014-01-22 16:14:02,918 [main] INFO
org.apache.pig.data.SchemaTupleFrontend - Distributed cache not supported or
needed in local mode. Setting key [pig.schematuple.local.dir] with code temp
directory: /tmp/1390436042918-02014-01-22 16:14:02,978 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.2014-01-22 16:14:02,991
[JobControl] INFO org.apache.hadoop.util.NativeCodeLoader - Loaded the
native-hadoop library2014-01-22 16:14:02,994 [JobControl] ERROR
org.apache.hadoop.security.UserGroupInformation - PriviledgedActionException
as:userid cause:ENOENT: No such file or directory2014-01-22 16:14:03,479 [main]
INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete2014-01-22 16:14:03,489 [main] WARN
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop
immediately on failure.2014-01-22 16:14:03,489 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job null has failed! Stop running all dependent jobs2014-01-22 16:14:03,490
[main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete2014-01-22 16:14:03,492 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to recreate
exception from backend error: ENOENT: No such file or directory at
org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:699) at
org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:654) at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344) at
org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189) at
org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856) at
org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:415) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824) at
org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378) at
org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601) at
org.apache.pig.backend.hadoop20.PigJobControl.mainLoopAction(PigJobControl.java:157)
at org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:134)
at java.lang.Thread.run(Thread.java:722) at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
2014-01-22 16:14:03,493 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil
- 1 map reduce job(s) failed!2014-01-22 16:14:03,493 [main] INFO
org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. Stats
reported below may be incomplete2014-01-22 16:14:03,495 [main] INFO
org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
