Re: Read a HDFS file from Spark source code
Hi Sean,
I was following this link;
http://mund-consulting.com/Blog/Posts/file-operations-in-HDFS-using-java.aspx
But, I was facing FileSystem ambiguity error. I really don't have any idea
as to how to go about doing this.
Can you please help me how to start off with this?
On Wed, Nov 12, 2014 at 11:26 AM, Samarth Mailinglist <
mailinglistsama...@gmail.com> wrote:
> Instead of a file path, use a HDFS URI.
> For example: (In Python)
>
>
>
> data = sc.textFile("hdfs://localhost/user/someuser/data")
>
>
>
> On Wed, Nov 12, 2014 at 10:12 AM, rapelly kartheek <
> kartheek.m...@gmail.com> wrote:
>
>> Hi
>>
>> I am trying to access a file in HDFS from spark "source code". Basically,
>> I am tweaking the spark source code. I need to access a file in HDFS from
>> the source code of the spark. I am really not understanding how to go about
>> doing this.
>>
>> Can someone please help me out in this regard.
>> Thank you!!
>> Karthik
>>
>
>
Re: Read a HDFS file from Spark source code
Instead of a file path, use a HDFS URI.
For example: (In Python)
data = sc.textFile("hdfs://localhost/user/someuser/data")
On Wed, Nov 12, 2014 at 10:12 AM, rapelly kartheek
wrote:
> Hi
>
> I am trying to access a file in HDFS from spark "source code". Basically,
> I am tweaking the spark source code. I need to access a file in HDFS from
> the source code of the spark. I am really not understanding how to go about
> doing this.
>
> Can someone please help me out in this regard.
> Thank you!!
> Karthik
>
Read a HDFS file from Spark source code
Hi I am trying to access a file in HDFS from spark "source code". Basically, I am tweaking the spark source code. I need to access a file in HDFS from the source code of the spark. I am really not understanding how to go about doing this. Can someone please help me out in this regard. Thank you!! Karthik
