Super! I did miss the fact that Job class indeed inherits from JobContext. This clarifies my issue. Thanks Dave and Harsh.
From: Dave Beech [mailto:d...@paraliatech.com] Sent: Tuesday, November 27, 2012 9:51 AM To: user@hadoop.apache.org Subject: Re: MapReduce APIs AK - look again at that javadoc. Job does a have getConfiguration() method. You may have missed it the first time because it's inherited from a parent class, JobContext. On 27 November 2012 14:23, Kartashov, Andy <andy.kartas...@mpac.ca<mailto:andy.kartas...@mpac.ca>> wrote: Thank man for the response. Much appreciated. Why? Because Job object doesn't have the below method getConfiguration(). See for yourself under mapreduce.Job: http://hadoop.apache.org/docs/r0.20.2/api/index.html or http://hadoop.apache.org/docs/current/api/index.html So, back to my original question. If job isn't instance of Job object, then what object does it belong to in the following statement (DistributedCache.addCacheFile(URI, job.getConfiguration()); From: Mahesh Balija [mailto:balijamahesh....@gmail.com<mailto:balijamahesh....@gmail.com>] Sent: Tuesday, November 27, 2012 12:22 AM To: user@hadoop.apache.org<mailto:user@hadoop.apache.org> Subject: Re: MapReduce APIs Hi AK, I don't really understand what is stopping you to use the job.getConfiguration() method to pass the configuration instance to DistributedCache.addCacheFile(URI, job.getConfiguration()). Only thing you need to do is pass the URI and configuration object (getting it from org.apache.hadoop.mapreduce.Job instance). Best, Mahesh.B. Calsoft Labs. On Mon, Nov 26, 2012 at 8:18 PM, Kartashov, Andy <andy.kartas...@mpac.ca<mailto:andy.kartas...@mpac.ca>> wrote: Harsh, Thanks for the " DistributedCache.addCacheFile(URI, job.getConfiguration());" suggestion. What class is your instance job belongs to? It is not Job class, for sure. So must be JobContext? When I write my driver using new API I write: Job job = new Job(); job.setJarByClass(.... job.setJobName(... job.setSetMapOutputKey... | .. value ......Redeuce..... So, how can I use your piece of code here, i.e? DistributedCache.addCacheFile(URI, job.getConfiguration()); How can I wire JobConf to Job instances? Thanks, AK -----Original Message----- From: Harsh J [mailto:ha...@cloudera.com<mailto:ha...@cloudera.com>] Sent: Saturday, November 24, 2012 2:22 AM To: <user@hadoop.apache.org<mailto:user@hadoop.apache.org>> Subject: Re: MapReduce APIs You could use the org.apache.hadoop.filecache.DistributedCache API as: DistributedCache.addCacheFile(URI, job.getConfiguration()); On Sat, Nov 24, 2012 at 3:06 AM, Kartashov, Andy <andy.kartas...@mpac.ca<mailto:andy.kartas...@mpac.ca>> wrote: > Guys, > > > > I know that there is old and new API for MapReduce. The old API is > found under org.apache.hadoop.mapred and the new is under > org.apache.hadoop.mapreduce > > > > I successfully used both (the old and the new API) writing my > MapReduce drivers. > > > > The problem came up when I tried to use distributed cache. My new API > Job object could not locate > > public void addCacheFile(URI uri) method and I was scratching my head why. > > > > What I did not reaslise is that despite new and oold API there is also > Hadoop 0.20 vs Hadoop 2.0.0 APIs that use exact same packages. > > The old Hadoop.0.20.00 new Mapreduce API class Job simply doesn't > have that method "addCacheFile(URI uri)". > > > > I am running Hadoop 2.0.0. so could not understand why the method was > not inside the class. I ended up rewriting MR job under old API mapred > package and ran soccessfully. > > > > Can anyone shed some light on this? > > > > Thanks > > AK > > > > > > NOTICE: This e-mail message and any attachments are confidential, > subject to copyright and may be privileged. Any unauthorized use, > copying or disclosure is prohibited. If you are not the intended > recipient, please delete and contact the sender immediately. Please > consider the environment before printing this e-mail. AVIS : le > présent courriel et toute pièce jointe qui l'accompagne sont > confidentiels, protégés par le droit d'auteur et peuvent être couverts > par le secret professionnel. Toute utilisation, copie ou divulgation > non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce > courriel, supprimez-le et contactez immédiatement l'expéditeur. > Veuillez penser à l'environnement avant d'imprimer le présent courriel -- Harsh J NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
