: reading distributed cache returns null pointer
The DistributedCache behavior is not symmetrical in local mode vs
distributed mode.
As I replied earlier, you need to use
DistributedCache.getCacheFiles() in distributed mode.
In your code, you can put a check:
if (getLocalCacheFiles()) returns
> From: Hemanth Yamijala
> To: common-user@hadoop.apache.org
> Sent: Fri, July 9, 2010 10:21:19 AM
> Subject: Re: reading distributed cache returns null pointer
>
> Hi,
>
> > Thanks for the information. I got your point. What I specifically wa
should work on
fully-distributed mode as well. What possibilities can be there for this
behavior?
Cheers
From: Hemanth Yamijala
To: common-user@hadoop.apache.org
Sent: Fri, July 9, 2010 10:21:19 AM
Subject: Re: reading distributed cache returns null pointer
__
> From: Rahul Jain
> To: common-user@hadoop.apache.org
> Sent: Fri, July 9, 2010 12:19:44 AM
> Subject: Re: reading distributed cache returns null pointer
>
> Yes, distributed cache writes files to the local file system for each mapper
> / reducer. So you should be
from the local
file system, not the HDFS?
Thanks again
From: Rahul Jain
To: common-user@hadoop.apache.org
Sent: Fri, July 9, 2010 12:19:44 AM
Subject: Re: reading distributed cache returns null pointer
Yes, distributed cache writes files to the local file system
ln(str);
> }
>
> Thanks
>
>
>
> From: Rahul Jain
> To: common-user@hadoop.apache.org
> Sent: Thu, July 8, 2010 8:15:58 PM
> Subject: Re: reading distributed cache returns null pointer
>
> I am not sure why you are
reamfs=hdfs.open(my_path);
while((str=fs.readLine())!=null)
System.out.println(str);
}
Thanks
From: Rahul Jain
To: common-user@hadoop.apache.org
Sent: Thu, July 8, 2010 8:15:58 PM
Subject: Re: reading distrib
I am not sure why you are using getFileClassPaths() API to access files...
here is what works for us:
Add the file(s) to distributed cache using:
DistributedCache.addCacheFile(p.toUri(), conf);
Read the files on the mapper using:
URI[] uris = DistributedCache.getCacheFiles(conf);
// access one o
Hello all,
As a new user of hadoop, I am having some problems with understanding some
things. I am writing a program to load a file to the distributed cache and read
this file in each mapper. In my driver program, I have added the file to my
distributed cache using:
Path p=new
