Hello list readers, i'm still looking for an explanation.
maybe i put it into this way: i run a job and at the end i need to know how many instances of a specific type were written into an output file. i wanted to rely on the counters. but maybe this is not a good idea anyway. or what is the outcome if speculative execution is used? how can i get the number of records contained in a MapFile - without the need to read it? thank you for some input, ud [EMAIL PROTECTED] 04/09/2008 10:35 AM Please respond to [email protected] To [email protected] cc Subject Counters giving double values hello, i tried to track down the problem with wrong counter values. didnt find any information / cases of it. maybe it is a feature i don't understand. the problem is, that in a local installation (localjobrunner) i sometimes (somehow randomly) get wrong counter values as the log() function of the Counters tells me in the log: 20080409044008268 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Job complete: job_local_84 20080409044008268 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Counters: 21 20080409044008268 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Map-Reduce Framework 20080409044008269 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Map input records=43917 20080409044008269 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Map output records=43917 20080409044008269 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Map input bytes=2116832 20080409044008269 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Map output bytes=9310354 20080409044008269 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Combine input records=0 20080409044008269 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Combine output records=0 20080409044008269 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Reduce input groups=83736 20080409044008269 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Reduce input records=87834 20080409044008269 INFO [DefaultQuartzScheduler_Worker-0] JobClient - Reduce output records=86640 what i don't not understand is, why or how it comes, that the number of map output records is different than the reduce input records. maybe somebody has a simple explanation for this effect. i would expect that the number must be same. as you can see the number is (always) exactly the double value of what i expect it should be. and also when i check the output file, the amount of records matches the numbers from the map counters and not the double values. thanks, ud
