gotcha this makes sense
On Wed, Jan 29, 2014 at 4:44 PM, praveenesh kumar <[email protected]>wrote: > @Jay - Plus if you see DistributedCache class, these methods have been > added inside the Job class, I am guessing they have kept the functionality > same, just merged DistributedCache class into Job class itself. giving more > methods for developers with less classes to worry about, thus simplifying > the API. I hope that makes sense. > > Regards > Prav > > > On Wed, Jan 29, 2014 at 9:41 PM, praveenesh kumar <[email protected]>wrote: > >> @Jay - I don't know how Job class is replacing the DistributedCache class >> , but I remember trying distributed cache functions like >> >> void *addArchiveToClassPath >> <http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html#addArchiveToClassPath%28org.apache.hadoop.fs.Path%29>* >> (Path<http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/fs/Path.html> >> archive) >> Add an archive path to the current set of classpath entries. >> void *addCacheArchive >> <http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html#addCacheArchive%28java.net.URI%29>* >> (URI<http://download.oracle.com/javase/6/docs/api/java/net/URI.html?is-external=true> >> uri) >> Add a archives to be localized void *addCacheFile >> <http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html#addCacheFile%28java.net.URI%29>* >> (URI<http://download.oracle.com/javase/6/docs/api/java/net/URI.html?is-external=true> >> uri) >> Add a file to be localized >> >> and it works fine. The same way you were using DC before.. Well I am not >> sure what would be the best answer, but if you are trying to use DC , I was >> able to do it with Job class itself. >> >> Regards >> Prav >> >> >> On Wed, Jan 29, 2014 at 9:27 PM, Jay Vyas <[email protected]> wrote: >> >>> Thanks for asking this : Im not sure and didnt realize this until you >>> mentioned it! >>> >>> 1) Prav: You are implying that we would use the "Job" Class... but how >>> could it replace the DC? >>> >>> 2) The point of the DC is to replicate a file so that its present and >>> local on ALL nodes. I didnt know it was deprecated, but there must be >>> some replacement for it - or maybe it just got renamed and moved? >>> >>> SO ... what is the future of the DistributedCache for mapreduce jobs? >>> >>> >>> On Wed, Jan 29, 2014 at 4:22 PM, praveenesh kumar >>> <[email protected]>wrote: >>> >>>> I think you can use the Job class. >>>> >>>> http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html >>>> >>>> Regards >>>> Prav >>>> >>>> >>>> On Wed, Jan 29, 2014 at 9:13 PM, Giordano, Michael < >>>> [email protected]> wrote: >>>> >>>>> I noticed that in Hadoop 2.2.0 >>>>> org.apache.hadoop.mapreduce.filecache.DistributedCache has been >>>>> deprecated. >>>>> >>>>> >>>>> >>>>> (http://hadoop.apache.org/docs/current/api/deprecated-list.html#class) >>>>> >>>>> >>>>> >>>>> Is there a class that provides equivalent functionality? My >>>>> application relies heavily on DistributedCache. >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Mike G. >>>>> >>>>> This communication, along with its attachments, is considered >>>>> confidential and proprietary to Vistronix. It is intended only for the >>>>> use >>>>> of the person(s) named above. Note that unauthorized disclosure or >>>>> distribution of information not generally known to the public is strictly >>>>> prohibited. If you are not the intended recipient, please notify the >>>>> sender immediately. >>>>> >>>> >>>> >>> >>> >>> -- >>> Jay Vyas >>> http://jayunit100.blogspot.com >>> >> >> > -- Jay Vyas http://jayunit100.blogspot.com
