Hey,
'/data/test.txt' is supposed to be on hdfs (if your not executing with -x
local), put it there from your local drive with command:
hadoop fs -put
for ex, create dir and the put:
hadoop fs -mkdir /data
hadoop fs -put /data/test.txt /data/
Sincerely,
Marek M.
________________________________________
From: kiranprasad [kiranprasa...@imimobile.com]
Sent: Tuesday, September 20, 2011 7:47 AM
To: user@pig.apache.org
Subject: Re: ERROR 2118: Input path does not exist
Hi Marek
I got the response as below
[kiranprasad.g@pig4 bin]$ ./hadoop fs -ls /
Found 1 items
drwxr-xr-x - kiranprasad.g supergroup 0 2011-09-19 19:23 /tmp
but after loading (A= LOAD '/data/test.txt' USING PigStorage();),
I am getting the same exception.
Message: org.apache.pig.backend.executionengine.
ExecException: ERROR 2118: Input path does not exist:
hdfs://10.0.0.61/data/msis
dns.txt
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInput
Format.getSplits(PigInputFormat.java:280)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:7
79)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobCont
rol.java:247)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:27
9)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException:
Input pa th does not
exist: hdfs://10.0.0.61/data/msisdns.txt
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:224)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextI
nputFormat.listStatus(PigTextInputFormat.java:36)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileI
nputFormat.java:241)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInput
Format.getSplits(PigInputFormat.java:268)
... 7 more
where is the default path of the data.txt configured ?
Regards
Kiran.G
-----Original Message-----
From: Marek Miglinski
Sent: Monday, September 19, 2011 3:14 PM
To: user@pig.apache.org
Subject: RE: ERROR 2118: Input path does not exist
hadoop fs -ls /
BACKSHASH at the end!
And:
A= LOAD '/data/test.txt' USING PigStorage();
BACKSLASH before data!
-----Original Message-----
From: kiranprasad [mailto:kiranprasa...@imimobile.com]
Sent: Monday, September 19, 2011 12:10 PM
To: user@pig.apache.org
Subject: Re: ERROR 2118: Input path does not exist
Hi
I am unable t run the below mentioned command : hadoop fs -ls getting the
same output.
[kiranprasad.g@pig4 hadoop-0.20.2]$ bin/hadoop fs -ls
ls: Cannot access .: No such file or directory.
Below is the exception.
Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
Input path does not exist:
hdfs://10.0.0.61/home/kiranprasad.g/pig-0.8.1/data/msisdns.txt
Regards
Kiran.G
-----Original Message-----
From: Marek Miglinski
Sent: Sunday, September 18, 2011 1:09 AM
To: user@pig.apache.org
Subject: RE: ERROR 2118: Input path does not exist
I meant that you should use absolute path when you load HDFS path from PIG,
so this is not correct:
A = LOAD 'data/test.txt' USING PigStorage(); This is correct:
A= LOAD '/data/test.txt' USING PigStorage();
If you want to display contents of HDFS, type from terminal:
hadoop fs -ls /
To display first level structure.
hadoop fs -lsr /
To display all levels.
Sincerely,
Marek M.
________________________________________
From: kiranprasad [kiranprasa...@imimobile.com]
Sent: Saturday, September 17, 2011 8:46 AM
To: user@pig.apache.org
Subject: Re: ERROR 2118: Input path does not exist
When I do hadoop fs -ls I am getting the below
[kiranprasad.g@pig4 ~]$ cd hadoop-0.20.2
[kiranprasad.g@pig4 hadoop-0.20.2]$ bin/hadoop fs -ls
ls: Cannot access .: No such file or directory.
Regards
Kiran.G
-----Original Message-----
From: Damien Hardy
Sent: Friday, September 16, 2011 8:34 PM
To: user@pig.apache.org
Subject: Re: ERROR 2118: Input path does not exist
What is the result of "hadoop fs -ls
hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt"
Regards,
--
Damien
Le 16/09/2011 17:04, kiranprasad a écrit :
> Hi
>
> I am getting the below mentioned exception after I load a file and do
> Filter on it.
> The file(test.txt) is saved inside PIG home/data/ folder.
>
>
> grunt> A= LOAD 'data/test.txt' USING PigStorage(); B= FOREACH A
> grunt> GENERATE $0; DUMP B;
> 2011-09-17 01:17:43,408 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> script: UNKNOWN
> 2011-09-17 01:17:43,409 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> pig.usenewlogicalplan is set to true. New logical plan will be used.
> 2011-09-17 01:17:43,652 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: B:
> Store(hdfs://10.0.0.61/tmp/temp-754030090/tmp1617007250:org.apache.pig
> .impl.io.InterStorage)
> - scope-4 Operator Key: scope-4)
> 2011-09-17 01:17:43,662 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompile
> r - File concatenation threshold: 100 optimistic? false
> 2011-09-17 01:17:43,688 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer
> yOptimizer
> - MR plan size before optimization: 1
> 2011-09-17 01:17:43,689 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer
> yOptimizer
> - MR plan size after optimization: 1
> 2011-09-17 01:17:43,742 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are
> added to the job
> 2011-09-17 01:17:43,754 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> lCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to
> default
> 0.3
> 2011-09-17 01:17:46,447 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> lCompiler
> - Setting up single store job
> 2011-09-17 01:17:46,609 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - 1 map-reduce job(s) waiting for submission.
> 2011-09-17 01:17:47,525 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - 0% complete
> 2011-09-17 01:17:48,158 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - job null has failed! Stop running all dependent jobs
> 2011-09-17 01:17:48,162 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - 100% complete
> 2011-09-17 01:17:48,169 [main] ERROR
> org.apache.pig.tools.pigstats.PigStats - ERROR 2997: Unable to
> recreate exception from backend error:
> org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
> Input path does not exist:
> hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt
> 2011-09-17 01:17:48,173 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> 2011-09-17 01:17:48,174 [main] INFO
> org.apache.pig.tools.pigstats.PigStats - Script Statistics:
>
> HadoopVersion PigVersion UserId StartedAt FinishedAt
> Features
> 0.20.2 0.8.1 kiranprasad.g 2011-09-17 01:17:43 2011-09-17
> 01:17:48 UNKNOWN
>
> Failed!
>
> Failed Jobs:
> JobId Alias Feature Message Outputs
> N/A A,B MAP_ONLY Message:
> org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
> Input path does not exist:
> hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:280)
> at
> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
> at
> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
> at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
> at
> org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
> at
> org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException:
> Input path does not exist:
> hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:224)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:241)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:268)
> ... 7 more
> hdfs://10.0.0.61/tmp/temp-754030090/tmp1617007250,
>
> Input(s):
> Failed to read data from
> "hdfs://10.0.0.61/user/kiranprasad.g/data/msisdn.txt"
>
> Output(s):
> Failed to produce result in
> "hdfs://10.0.0.61/tmp/temp-754030090/tmp1617007250"
>
> Counters:
> Total records written : 0
> Total bytes written : 0
> Spillable Memory Manager spill count : 0 Total bags proactively
> spilled: 0 Total records proactively spilled: 0
>
> Job DAG:
> null
>
>
> 2011-09-17 01:17:48,174 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - Failed!
> 2011-09-17 01:17:48,184 [main] ERROR org.apache.pig.tools.grunt.Grunt
> - ERROR 1066: Unable to open iterator for alias B Details at logfile:
> /home/kiranprasad.g/pig-0.8.1/pig_1316202429844.log
>
> Any idea where am I making the mistake ?
>
>
> Regards
> Kiran.G
