Hi Tom,
Thanks for notifying.
This issue really gives me a hard time circumventing the problem...
I even tried the job.submit() and then read the counter asynchronously...
But found out that the values are always zero...
What Amareshwari Sriramadasu suggested works!
I added these 3 parameters in mapred-site.xml :
<property>
<name>mapreduce.jobtracker.retirejobs</name>
<value>false</value>
</property>
<property>
<name>mapreduce.jobtracker.persist.jobstatus.active</name>
<value>true</value>
</property>
<property>
<name>mapreduce.jobtracker.persist.jobstatus.hours</name>
<value>1</value>
</property>
All work fine now.
I'm now testing it with a very large input to see what's the
performance improvement over the 0.20.2
I heard the 0.21 has a new shuffling :), I'm keeping my hope high...
Felix Halim
On Wed, Jul 7, 2010 at 7:19 PM, Tom White <[email protected]> wrote:
> Hi Felix,
>
> Aaron Kimball hit the same problem - it's being discussed at
> https://issues.apache.org/jira/browse/MAPREDUCE-1920.
>
> Thanks for reporting this.
>
> Cheers,
> Tom
>
> On Tue, Jul 6, 2010 at 11:26 AM, Felix Halim <[email protected]> wrote:
>> I tried hadoop 0.21 release candidate.
>>
>> job.waitForCompletion(true);
>> Counters ctrs = job.getCounters();
>> // here ctrs is null
>>
>>
>> In the previous hadoop version 0.20.2 it worked fine for all times.
>>
>> Is this a bug in 0.21 ?
>> Or i'm missing some settings?
>>
>> Thanks,
>>
>> Felix Halim
>>
>
