Hi Harsh, Awesome, after incorporating these changes in my jobdriver, it worked for me. It looks like I should use Oozie to launch my jobs so that I don't have to copy this code in my driver.
Thanks for your help Manish On Sun, Sep 29, 2013 at 3:51 AM, Harsh J <[email protected]> wrote: > Hm, I think I forgot the bit where Oozie also adds a DT for itself to > use: > https://github.com/apache/oozie/blob/release-4.0.0/core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java#L374 > . > Doing that same additional thing to the job's driver, works just fine > for me. > > On Sun, Sep 29, 2013 at 10:51 AM, Manish Verma > <[email protected]> wrote: > > Hi Harsh, > > > > Thanks for the pointer but it didn't fix my problem. Basically I am > trying > > to read the counters set by the mapper in my reducer. And the reducer has > > trouble reading the mapper's counter using the JobClient. I tested this > > logic in a single node cloudera VM which did not have kerberos installed. > > > > Thanks > > Manish > > > > > > On Sat, Sep 28, 2013 at 11:06 AM, Harsh J <[email protected]> wrote: > >> > >> You'll need to reuse the security tokens of the current job to > >> communicate without requiring a re-auth from the task. See the > >> specific code snippet (of what Oozie does for similar goals) at > >> > >> > https://github.com/apache/oozie/blob/release-4.0.0/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java#L90 > >> and also see MAPREDUCE-4324. > >> > >> On Sat, Sep 28, 2013 at 2:43 AM, Manish Verma > >> <[email protected]> wrote: > >> > I am trying to get the job tracker counters in my reducer. It works on > >> > single node demo hadoop but fails on a real cluster where kerberos is > >> > used > >> > for authentication. > >> > > >> > ******************************************** > >> > RunningJob parentJob = > >> > client.getJob(JobID.forName( > >> > context.getConfiguration().get("mapred.job.id") )); > >> > > >> > Counters counters = parentJob.getCounters(); > >> > ******************************************** > >> > > >> > The call to getCounter() API throws GSSException (No valid credentials > >> > provided - Failed to find any kerberos tgt). > >> > > >> > I launched this job using hadoop jar command. > >> > > >> > Any help would be much appreciated. > >> > > >> > Thanks > >> > Manish > >> > > >> > >> > >> > >> -- > >> Harsh J > > > > > > > > -- > Harsh J >
