Here is a related jira: https://issues.apache.org/jira/browse/HADOOP-3126
> -----Original Message----- > From: Devaraj Das [mailto:[EMAIL PROTECTED] > Sent: Wednesday, April 16, 2008 3:56 AM > To: [email protected] > Subject: RE: Counters giving double values > > Also, in those cases where you see wrong counter values, did you validate > the final (reduce) output for correctness (I am just trying to see whether > the problem is with the Counter updates). > > > -----Original Message----- > > From: Devaraj Das [mailto:[EMAIL PROTECTED] > > Sent: Wednesday, April 16, 2008 4:23 PM > > To: [email protected] > > Subject: RE: Counters giving double values > > > > Thanks for the detailed answer. Which hadoop version are you > > on? If you are confident that it is not a problem with your > > app, pls raise a jira. > > > > > > _____ > > > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > > Sent: Wednesday, April 16, 2008 3:25 PM > > To: [email protected] > > Subject: RE: Counters giving double values > > > > > > > > Thanks so far. > > > > key and values are custom implementations. > > > > key implements WritableComparable > > value extends VersionedWritable > > > > btw. The only problem i encounter is that the Counter values > > are wrong. If i > > check the records in the MapFile (re-read it) which is > > written as the output > > of the mapred job, the amount of records is correct and > > represents the halve > > of the reported counter value. > > the same applies for the results of the operations which are > > carried out in > > the reduce(). everything is correct, except the counter values. > > > > the whole thing happens only sometimes. > > > > > > Key serializing / Deserializing, i guess you want to see this > > part of the > > code: > > > > public int language; > > public String term; > > > > public void readFields(DataInput in) throws IOException { > > language = in.readInt(); > > term = Text.readString(in).toString(); > > } > > > > public void write(DataOutput out) throws IOException { > > out.writeInt(language); > > Text.writeString(out, term); > > } > > > > > > > > > >
