hadoop fs -ls / BACKSHASH at the end!
And: A= LOAD '/data/test.txt' USING PigStorage(); BACKSLASH before data! -----Original Message----- From: kiranprasad [mailto:[email protected]] Sent: Monday, September 19, 2011 12:10 PM To: [email protected] Subject: Re: ERROR 2118: Input path does not exist Hi I am unable t run the below mentioned command : hadoop fs -ls getting the same output. [kiranprasad.g@pig4 hadoop-0.20.2]$ bin/hadoop fs -ls ls: Cannot access .: No such file or directory. Below is the exception. Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: hdfs://10.0.0.61/home/kiranprasad.g/pig-0.8.1/data/msisdns.txt Regards Kiran.G -----Original Message----- From: Marek Miglinski Sent: Sunday, September 18, 2011 1:09 AM To: [email protected] Subject: RE: ERROR 2118: Input path does not exist I meant that you should use absolute path when you load HDFS path from PIG, so this is not correct: A = LOAD 'data/test.txt' USING PigStorage(); This is correct: A= LOAD '/data/test.txt' USING PigStorage(); If you want to display contents of HDFS, type from terminal: hadoop fs -ls / To display first level structure. hadoop fs -lsr / To display all levels. Sincerely, Marek M. ________________________________________ From: kiranprasad [[email protected]] Sent: Saturday, September 17, 2011 8:46 AM To: [email protected] Subject: Re: ERROR 2118: Input path does not exist When I do hadoop fs -ls I am getting the below [kiranprasad.g@pig4 ~]$ cd hadoop-0.20.2 [kiranprasad.g@pig4 hadoop-0.20.2]$ bin/hadoop fs -ls ls: Cannot access .: No such file or directory. Regards Kiran.G -----Original Message----- From: Damien Hardy Sent: Friday, September 16, 2011 8:34 PM To: [email protected] Subject: Re: ERROR 2118: Input path does not exist What is the result of "hadoop fs -ls hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt" Regards, -- Damien Le 16/09/2011 17:04, kiranprasad a écrit : > Hi > > I am getting the below mentioned exception after I load a file and do > Filter on it. > The file(test.txt) is saved inside PIG home/data/ folder. > > > grunt> A= LOAD 'data/test.txt' USING PigStorage(); B= FOREACH A > grunt> GENERATE $0; DUMP B; > 2011-09-17 01:17:43,408 [main] INFO > org.apache.pig.tools.pigstats.ScriptState - Pig features used in the > script: UNKNOWN > 2011-09-17 01:17:43,409 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > pig.usenewlogicalplan is set to true. New logical plan will be used. > 2011-09-17 01:17:43,652 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: B: > Store(hdfs://10.0.0.61/tmp/temp-754030090/tmp1617007250:org.apache.pig > .impl.io.InterStorage) > - scope-4 Operator Key: scope-4) > 2011-09-17 01:17:43,662 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompile > r - File concatenation threshold: 100 optimistic? false > 2011-09-17 01:17:43,688 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer > yOptimizer > - MR plan size before optimization: 1 > 2011-09-17 01:17:43,689 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer > yOptimizer > - MR plan size after optimization: 1 > 2011-09-17 01:17:43,742 [main] INFO > org.apache.pig.tools.pigstats.ScriptState - Pig script settings are > added to the job > 2011-09-17 01:17:43,754 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro > lCompiler > - mapred.job.reduce.markreset.buffer.percent is not set, set to > default > 0.3 > 2011-09-17 01:17:46,447 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro > lCompiler > - Setting up single store job > 2011-09-17 01:17:46,609 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce > Launcher > - 1 map-reduce job(s) waiting for submission. > 2011-09-17 01:17:47,525 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce > Launcher > - 0% complete > 2011-09-17 01:17:48,158 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce > Launcher > - job null has failed! Stop running all dependent jobs > 2011-09-17 01:17:48,162 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce > Launcher > - 100% complete > 2011-09-17 01:17:48,169 [main] ERROR > org.apache.pig.tools.pigstats.PigStats - ERROR 2997: Unable to > recreate exception from backend error: > org.apache.pig.backend.executionengine.ExecException: ERROR 2118: > Input path does not exist: > hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt > 2011-09-17 01:17:48,173 [main] ERROR > org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed! > 2011-09-17 01:17:48,174 [main] INFO > org.apache.pig.tools.pigstats.PigStats - Script Statistics: > > HadoopVersion PigVersion UserId StartedAt FinishedAt > Features > 0.20.2 0.8.1 kiranprasad.g 2011-09-17 01:17:43 2011-09-17 > 01:17:48 UNKNOWN > > Failed! > > Failed Jobs: > JobId Alias Feature Message Outputs > N/A A,B MAP_ONLY Message: > org.apache.pig.backend.executionengine.ExecException: ERROR 2118: > Input path does not exist: hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:280) > at > org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885) > at > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779) > at > org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730) > at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378) > at > org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247) > at > org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279) > at java.lang.Thread.run(Thread.java:619) > Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: > Input path does not exist: > hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:224) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36) > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:241) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:268) > ... 7 more > hdfs://10.0.0.61/tmp/temp-754030090/tmp1617007250, > > Input(s): > Failed to read data from > "hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt" > > Output(s): > Failed to produce result in > "hdfs://10.0.0.61/tmp/temp-754030090/tmp1617007250" > > Counters: > Total records written : 0 > Total bytes written : 0 > Spillable Memory Manager spill count : 0 Total bags proactively > spilled: 0 Total records proactively spilled: 0 > > Job DAG: > null > > > 2011-09-17 01:17:48,174 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce > Launcher > - Failed! > 2011-09-17 01:17:48,184 [main] ERROR org.apache.pig.tools.grunt.Grunt > - ERROR 1066: Unable to open iterator for alias B Details at logfile: > /home/kiranprasad.g/pig-0.8.1/pig_1316202429844.log > > Any idea where am I making the mistake ? > > > Regards > Kiran.G
