Thanks.
This:
>
> FileSystem fs = FileSystem.get(conf);
> DistributedCache.addFileToClassPath(new
> Path("hdfs://localhost:9000/my.app/batch.jar"), conf, fs);
didn't work, but the uber-jar /lib subfolder did work.
thanks again,
Nick
On Apr 9, 2012, at 2:44 PM, Harsh J wrote:
> Nick,
>
> For jars, you need to use DistributedCache.addFileToClassPath(…). The
> archive-adding function is only to be used when you want the file
> you're adding to also be extracted into the task's working directory.
>
> Also, you don't need to use DistributedFileSystem classes (its not
> supposed to be publicly usable).
>
> Therefore, you may do:
>
> FileSystem fs = FileSystem.get(conf);
> DistributedCache.addFileToClassPath(new
> Path("hdfs://localhost:9000/my.app/batch.jar"), conf, fs);
>
> Or you may also pack your jar into your driver jar under a lib/
> subfolder inside the jar itself (making it an "über" jar, as they call
> it), and just submit that.
>
> Either of these approaches will get you going.
>
> On Mon, Apr 9, 2012 at 11:08 PM, Nick Collier <nick.coll...@verizon.net>
> wrote:
>> Hi,
>>
>> Using Hadoop 1.0.1 I'm trying to use the DistributedCache to add additional
>> jars to the classpath used by my Mappers but I can't get it to work. In the
>> run(String[] args) method of my Tool implementation, I've tried:
>>
>> FileSystem fs = DistributedFileSystem.get(conf);
>> DistributedCache.addArchiveToClassPath(new
>> Path("hdfs://localhost:9000/my.app/batch.jar"), conf, fs);
>>
>> and
>>
>> FileSystem fs = DistributedFileSystem.get(conf);
>> DistributedCache.addArchiveToClassPath(new Path("/my.app/batch.jar"), conf,
>> fs);
>>
>> and using a local path,
>>
>> DistributedCache.addArchiveToClassPath(new Path("./lib/batch.jar"), conf);
>>
>> I've also tried using the addFileToClassPath alternative but also without
>> success. I'm running in pseudo-distributed mode. Adding the additional jars
>> with -libjars does work, but I would like to get this working as well as it
>> will be easier to maintain.
>>
>> Any help is much appreciated,
>>
>> Nick
>>
>>
>>
>
>
>
> --
> Harsh J
