I found that if a file is present in all the nodes in the given path in
localFS, then reading is possible.
But is there a way to read if the file is present only in certain nodes ??
[There should be a way !!]
*NEED: Wanted to do some filter ops in HDFS file, create a local file of
the result, create an RDD out of it operate *
Is there any way out ??
Thanks in advance !
On Fri, May 9, 2014 at 12:18 AM, Sai Prasanna <ansaiprasa...@gmail.com>wrote:
> Hi Everyone,
>
> I think all are pretty busy, the response time in this group has slightly
> increased.
>
> But anyways, this is a pretty silly problem, but could not get over.
>
> I have a file in my localFS, but when i try to create an RDD out of it,
> tasks fails with file not found exception is thrown at the log files.
>
> *var file = sc.textFile("file:///home/sparkcluster/spark/input.txt");*
> *file.top(1);*
>
> input.txt exists in the above folder but still Spark coudnt find it. Some
> parameters need to be set ??
>
> Any help is really appreciated. Thanks !!
>
