why does it need to be local file? why not do some filter ops on hdfs file
and save to hdfs, from where you can create rdd?
you can read a small file in on driver program and use sc.parallelize to
turn it into RDD
On May 16, 2014 7:01 PM, Sai Prasanna ansaiprasa...@gmail.com wrote:
I found that
Hi Everyone,
I think all are pretty busy, the response time in this group has slightly
increased.
But anyways, this is a pretty silly problem, but could not get over.
I have a file in my localFS, but when i try to create an RDD out of it,
tasks fails with file not found exception is thrown at
I found that if a file is present in all the nodes in the given path in
localFS, then reading is possible.
But is there a way to read if the file is present only in certain nodes ??
[There should be a way !!]
*NEED: Wanted to do some filter ops in HDFS file, create a local file of
the result,
