Hi Shi
Try out the following,it could get things working
Use DistributedCache.getCacheFiles() instead of
DistributedCache.getLocalCacheFiles()
public void setup(JobConf job)
{
DistributedCache.getLocalCacheFiles(job)
.
.
.
}
If that also doesn't seem to work and if you have the file name in handy
which you populated on cache, use it the normal java way
job submission:
hadoop jar ............ -files /home/x/y/testFile.txt <input> <output>
in your setup() method, just retrieve the files like
public void setup(JobConf job)
{
File file = new File("testFile.txt");
}
Regards
Bejoy.K.S
On Fri, Dec 16, 2011 at 4:09 AM, Shi Yu <[email protected]> wrote:
> Hi,
>
> I am using 0.20.X branch. However, I need to use the new API because it
> has the cleanup(context) method in Mapper. However, I am confused about
> how to load the cached files in mapper. I could load the DistributedCache
> files using old API (JobConf), but in new API it always returns null. I
> read some previous discussions that "on 0.20.X branch, calling
> DistributedCache using old API is encouraged." My question is: Is it
> possible to use DistributedCache using new API, or the only possible way is
> to upgrade to higher version? Or I use a wrong method in my code?
> Thanks!
>
>
> The relevant code is something as follows:
>
> class Map extends Mapper <K1, V1, K2, V2>{
>
> public void setup(Mapper<K1,V1, K2, V2>.Context context) throws
> IOException{
> try{
> Path[] localFiles;
> localFiles = DistributedCache.**getLocalCacheFiles(context.**
> getConfiguration());
> String file1 = localFiles[0].toUri.getPath(); // return null?
> ....
>
> }
>
>
> }
>
> }
>
> public int run(String[] args) throws IOException, URISyntaxException{
> ...
> //set the configuration
>
> Job job = new Job(conf,"Myjob");
> DistributeCache.addCacheFile(**new URI(hdfs://server:9000/file1),
> job.getConfiguration);
>
> }
>
> BR,
>
>
> Shi
>
>
