Here is a related jira:
https://issues.apache.org/jira/browse/HADOOP-3126


> -----Original Message-----
> From: Devaraj Das [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, April 16, 2008 3:56 AM
> To: [email protected]
> Subject: RE: Counters giving double values
> 
> Also, in those cases where you see wrong counter values, did you
validate
> the final (reduce) output for correctness (I am just trying to see
whether
> the problem is with the Counter updates).
> 
> > -----Original Message-----
> > From: Devaraj Das [mailto:[EMAIL PROTECTED]
> > Sent: Wednesday, April 16, 2008 4:23 PM
> > To: [email protected]
> > Subject: RE: Counters giving double values
> >
> > Thanks for the detailed answer. Which hadoop version are you
> > on? If you are confident that it is not a problem with your
> > app, pls raise a jira.
> >
> >
> >   _____
> >
> > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> > Sent: Wednesday, April 16, 2008 3:25 PM
> > To: [email protected]
> > Subject: RE: Counters giving double values
> >
> >
> >
> > Thanks so far.
> >
> > key and values are custom implementations.
> >
> > key implements WritableComparable
> > value extends VersionedWritable
> >
> > btw. The only problem i encounter is that the Counter values
> > are wrong. If i
> > check the records in the MapFile (re-read it) which is
> > written as the output
> > of the mapred job, the amount of records is correct and
> > represents the halve
> > of the reported counter value.
> > the same applies for the results of the operations which are
> > carried out in
> > the reduce(). everything is correct, except the counter values.
> >
> > the whole thing happens only sometimes.
> >
> >
> > Key serializing / Deserializing, i guess you want to see this
> > part of the
> > code:
> >
> >         public int language;
> >         public String term;
> >
> >         public void readFields(DataInput in) throws IOException {
> >                 language = in.readInt();
> >                 term =         Text.readString(in).toString();
> >         }
> >
> >         public void write(DataOutput out) throws IOException {
> >                 out.writeInt(language);
> >                 Text.writeString(out, term);
> >         }
> >
> >
> >
> >
> >

Reply via email to