What are you running this on? This is really odd: /proc/<pid>/status does not have information about swap space used(VmSwap).
D On Fri, Oct 19, 2012 at 11:11 AM, lei tang <[email protected]> wrote: > I checked the hadoop config and found that the staging dir specified does > not exist in local path (only in hdfs) and neither do I have write access. > So I changed the staging dir as below, but encounter some weird error > message: > > [ltang01@stg-trgt00 ~]$ pig > -Dmapreduce.jobtracker.staging.root.dir=~/tmp/staging -x local > 2012-10-19 11:06:49,624 [main] INFO org.apache.pig.Main - Apache Pig > version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12 > 2012-10-19 11:06:49,625 [main] INFO org.apache.pig.Main - Logging error > messages to: /data1/home/ltang01/pig_1350670009621.log > 2012-10-19 11:06:50,009 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting > to hadoop file system at: file:/// > grunt> A = load 'toy.txt'; > grunt> dump A; > 2012-10-19 11:06:53,485 [main] INFO > org.apache.pig.tools.pigstats.ScriptState - Pig features used in the > script: UNKNOWN > 2012-10-19 11:06:53,660 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - > File concatenation threshold: 100 optimistic? false > 2012-10-19 11:06:53,693 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size before optimization: 1 > 2012-10-19 11:06:53,693 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size after optimization: 1 > 2012-10-19 11:06:53,886 [main] INFO > org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added > to the job > 2012-10-19 11:06:53,908 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 > 2012-10-19 11:06:53,953 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - Setting up single store job > 2012-10-19 11:06:53,996 [main] INFO > org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with > processName=JobTracker, sessionId= > 2012-10-19 11:06:54,000 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 1 map-reduce job(s) waiting for submission. > 2012-10-19 11:06:54,125 [Thread-5] WARN org.apache.hadoop.mapred.JobClient > - No job jar file set. User classes may not be found. See JobConf(Class) > or JobConf#setJar(String). > 2012-10-19 11:06:54,207 [Thread-5] INFO > org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths > to process : 1 > 2012-10-19 11:06:54,207 [Thread-5] INFO > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input > paths to process : 1 > 2012-10-19 11:06:54,222 [Thread-5] INFO > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input > paths (combined) to process : 1 > 2012-10-19 11:06:54,501 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 0% complete > 2012-10-19 11:06:55,005 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - HadoopJobId: job_local_0001 > 2012-10-19 11:06:55,268 [Thread-6] INFO > org.apache.hadoop.mapreduce.util.ProcessTree - setsid exited with exit code > 0 > 2012-10-19 11:06:55,824 [Thread-6] WARN > org.apache.hadoop.mapreduce.util.ProcfsBasedProcessTree - > /proc/<pid>/status does not have information about swap space used(VmSwap). > Can not track swap usage of a task. > 2012-10-19 11:06:55,825 [Thread-6] INFO org.apache.hadoop.mapred.Task - > Using ResourceCalculatorPlugin : > org.apache.hadoop.mapreduce.util.LinuxResourceCalculatorPlugin@52cab854 > 2012-10-19 11:06:55,853 [Thread-6] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader > - Current split being processed file:/data1/home/ltang01/toy.txt:0+8 > 2012-10-19 11:06:55,914 [Thread-6] INFO org.apache.hadoop.mapred.Task - > Task:attempt_local_0001_m_000000_0 is done. And is in the process of > commiting > 2012-10-19 11:06:55,921 [Thread-6] INFO > org.apache.hadoop.mapred.LocalJobRunner - > 2012-10-19 11:06:55,922 [Thread-6] INFO org.apache.hadoop.mapred.Task - > Task attempt_local_0001_m_000000_0 is allowed to commit now > 2012-10-19 11:06:55,927 [Thread-6] INFO > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output > of task 'attempt_local_0001_m_000000_0' to > file:/tmp/temp487011820/tmp-1248641571 > 2012-10-19 11:06:55,927 [Thread-6] INFO > org.apache.hadoop.mapred.LocalJobRunner - > 2012-10-19 11:06:55,928 [Thread-6] INFO org.apache.hadoop.mapred.Task - > Task 'attempt_local_0001_m_000000_0' done. > 2012-10-19 11:06:55,929 [Thread-6] WARN > org.apache.hadoop.mapred.FileOutputCommitter - Output path is null in > cleanup > 2012-10-19 11:06:56,380 [main] INFO org.apache.hadoop.ipc.Client - > Retrying connect to server: localhost/127.0.0.1:9001. Already tried 0 > time(s). > 2012-10-19 11:06:57,382 [main] INFO org.apache.hadoop.ipc.Client - > Retrying connect to server: localhost/127.0.0.1:9001. Already tried 1 > time(s). > 2012-10-19 11:06:58,383 [main] INFO org.apache.hadoop.ipc.Client - > Retrying connect to server: localhost/127.0.0.1:9001. Already tried 2 > time(s). > 2012-10-19 11:06:59,385 [main] INFO org.apache.hadoop.ipc.Client - > Retrying connect to server: localhost/127.0.0.1:9001. Already tried 3 > time(s). > > > It keeps trying to connect to the local server. I have no idea what this is > about. Can you help? > > - Lei > > On Wed, Oct 17, 2012 at 9:05 PM, Dmitriy Ryaboy <[email protected]> wrote: > >> I think it's trying to find the staging directory set in your >> configuration, not finding it, and isn't able to create it. >> depending on your configs, that could be in different places, but >> usually it's looking under /tmp/mapred . Check permissions there. >> >> D >> >> On Mon, Oct 15, 2012 at 4:35 PM, lei tang <[email protected]> wrote: >> > Yes. it is there. Tried the full path as well, still not working. >> > >> > On Mon, Oct 15, 2012 at 3:45 PM, Prashant Kommireddi < >> [email protected]>wrote: >> > >> >> Is this file present on your local FS? >> >> file:///data1/home/ltang01/mf/prep/in/toy.txt >> >> >> >> On Mon, Oct 15, 2012 at 3:43 PM, lei tang <[email protected]> wrote: >> >> >> >> > Hi, >> >> > >> >> > I'm using Pig for my daily job. Pig dose a good job in mapreduce mode >> in >> >> > our internal hadoop cluster. But it signals an error whenever I want >> to >> >> > run it in local mode. I believe this is due to some permission or >> config >> >> > issue. Does anybody know how I can address this problem? Any idea is >> >> > appreciated. >> >> > >> >> > [ltang01@stg-trgt00 in]$ pig -x local >> >> > 2012-10-11 17:29:25,707 [main] INFO org.apache.pig.Main - Apache Pig >> >> > version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12 >> >> > 2012-10-11 17:29:25,708 [main] INFO org.apache.pig.Main - Logging >> error >> >> > messages to: /somewhere/in/pig_1350001765703.log >> >> > 2012-10-11 17:29:26,068 [main] INFO >> >> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - >> >> Connecting >> >> > to hadoop file system at: file:/// >> >> > grunt> A = load 'toy.txt'; >> >> > grunt> dump A; >> >> > 2012-10-11 17:29:36,056 [main] INFO >> >> > org.apache.pig.tools.pigstats.ScriptState - Pig features used in the >> >> > script: UNKNOWN >> >> > 2012-10-11 17:29:36,218 [main] INFO >> >> > >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - >> >> > File concatenation threshold: 100 optimistic? false >> >> > 2012-10-11 17:29:36,249 [main] INFO >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer >> >> > - MR plan size before optimization: 1 >> >> > 2012-10-11 17:29:36,250 [main] INFO >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer >> >> > - MR plan size after optimization: 1 >> >> > 2012-10-11 17:29:36,382 [main] INFO >> >> > org.apache.pig.tools.pigstats.ScriptState - Pig script settings are >> added >> >> > to the job >> >> > 2012-10-11 17:29:36,403 [main] INFO >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler >> >> > - mapred.job.reduce.markreset.buffer.percent is not set, set to >> default >> >> 0.3 >> >> > 2012-10-11 17:29:36,445 [main] INFO >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler >> >> > - Setting up single store job >> >> > 2012-10-11 17:29:36,486 [main] INFO >> >> > org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics >> with >> >> > processName=JobTracker, sessionId= >> >> > 2012-10-11 17:29:36,493 [main] INFO >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> >> > - 1 map-reduce job(s) waiting for submission. >> >> > 2012-10-11 17:29:36,995 [main] INFO >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> >> > - 0% complete >> >> > 2012-10-11 17:29:37,002 [main] INFO >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> >> > - job null has failed! Stop running all dependent jobs >> >> > 2012-10-11 17:29:37,002 [main] INFO >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> >> > - 100% complete >> >> > 2012-10-11 17:29:37,007 [main] ERROR >> >> > org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to >> >> > recreate exception from backend error: ENOENT: No such file or >> directory >> >> > at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) >> >> > at >> >> > >> >> > >> >> >> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:513) >> >> > at >> >> > >> >> >> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:325) >> >> > at >> >> > >> org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189) >> >> > at >> >> > >> >> > >> >> >> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126) >> >> > at >> org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:873) >> >> > at >> org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:867) >> >> > at java.security.AccessController.doPrivileged(Native Method) >> >> > at javax.security.auth.Subject.doAs(Subject.java:416) >> >> > at >> >> > >> >> > >> >> >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1109) >> >> > at >> >> > >> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:867) >> >> > at >> >> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:841) >> >> > at >> org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378) >> >> > at >> >> > >> >> > >> >> >> org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247) >> >> > at >> >> > >> org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279) >> >> > at java.lang.Thread.run(Thread.java:679) >> >> > at >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260) >> >> > >> >> > 2012-10-11 17:29:37,007 [main] ERROR >> >> > org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) >> failed! >> >> > 2012-10-11 17:29:37,007 [main] INFO >> >> > org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. >> Stats >> >> > reported below may be incomplete >> >> > 2012-10-11 17:29:37,009 [main] INFO >> >> > org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics: >> >> > >> >> > HadoopVersion PigVersion UserId StartedAt FinishedAt >> >> > Features >> >> > 0.20.2-dev 0.10.0 ltang01 2012-10-11 17:29:36 2012-10-11 >> >> > 17:29:37 UNKNOWN >> >> > >> >> > Failed! >> >> > >> >> > Failed Jobs: >> >> > JobId Alias Feature Message Outputs >> >> > N/A A MAP_ONLY Message: ENOENT: No such file or >> >> directory >> >> > at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) >> >> > at >> >> > >> >> > >> >> >> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:513) >> >> > at >> >> > >> >> >> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:325) >> >> > at >> >> > >> org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189) >> >> > at >> >> > >> >> > >> >> >> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126) >> >> > at >> org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:873) >> >> > at >> org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:867) >> >> > at java.security.AccessController.doPrivileged(Native Method) >> >> > at javax.security.auth.Subject.doAs(Subject.java:416) >> >> > at >> >> > >> >> > >> >> >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1109) >> >> > at >> >> > >> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:867) >> >> > at >> >> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:841) >> >> > at >> org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378) >> >> > at >> >> > >> >> > >> >> >> org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247) >> >> > at >> >> > >> org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279) >> >> > at java.lang.Thread.run(Thread.java:679) >> >> > at >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260) >> >> > file:/tmp/temp-916111882/tmp1833113178, >> >> > >> >> > Input(s): >> >> > Failed to read data from >> "file:///data1/home/ltang01/mf/prep/in/toy.txt" >> >> > >> >> > Output(s): >> >> > Failed to produce result in "file:/tmp/temp-916111882/tmp1833113178" >> >> > >> >> > Job DAG: >> >> > null >> >> > >> >> > >> >> > 2012-10-11 17:29:37,009 [main] INFO >> >> > >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> >> > - Failed! >> >> > 2012-10-11 17:29:37,015 [main] ERROR org.apache.pig.tools.grunt.Grunt >> - >> >> > ERROR 1066: Unable to open iterator for alias A >> >> > Details at logfile: >> /data1/home/ltang01/mf/prep/in/pig_1350001765703.log >> >> > >> >> >>
