Hi,
I am using 0.20.X branch. However, I need to use the new API because it
has the cleanup(context) method in Mapper. However, I am confused about
how to load the cached files in mapper. I could load the
DistributedCache files using old API (JobConf), but in new API it
always returns null. I read some previous discussions that "on 0.20.X
branch, calling DistributedCache using old API is encouraged." My
question is: Is it possible to use DistributedCache using new API, or
the only possible way is to upgrade to higher version? Or I use a
wrong method in my code? Thanks!
The relevant code is something as follows:
class Map extends Mapper <K1, V1, K2, V2>{
public void setup(Mapper<K1,V1, K2, V2>.Context context) throws IOException{
try{
Path[] localFiles;
localFiles =
DistributedCache.getLocalCacheFiles(context.getConfiguration());
String file1 = localFiles[0].toUri.getPath(); // return null?
....
}
}
}
public int run(String[] args) throws IOException, URISyntaxException{
...
//set the configuration
Job job = new Job(conf,"Myjob");
DistributeCache.addCacheFile(new URI(hdfs://server:9000/file1),
job.getConfiguration);
}
BR,
Shi
