What are the types of key1 and key2? What does the readFields() method look like?
-Joey On Sun, Aug 14, 2011 at 10:07 PM, Stan Rosenberg <[email protected]> wrote: > On Sun, Aug 14, 2011 at 9:33 PM, Joey Echeverria <[email protected]> wrote: > >> Does your compareTo() method test object pointer equality? If so, you could >> be getting burned by Hadoop reusing Writable objects. > > > Yes, but only the equality between enum values. Interestingly, when > 'reduce' is called there are three instances of the "same" key. > Two instances are correctly merged and they both come from the same mapper. > The other instance comes from a different mapper, and for > some reason does not get merged. I see the key and the values > (corresponding to the two merged instances) passed as arguments > to 'reduce'; then in subsequent 'reduce' call I see the key and the value > corresponding to the third instance. > > For completeness, here is my 'Key.compareTo': > > public int compareTo(Key o) { > if (this.type != o.type) { > // Type.X < Type.Y > return (this.type == Type.X ? -1 : 1); > } > // otherwise, delegate > if (this.type == Type.X) { > return this.key1.compareTo(o.key1); > } else { > return this.key2.compareTo(o.key2); > } > } > > The 'type' field is an enum with two possible values, say X and Y. Key is > essentially a union type; i.e., at any given time > it's the values in key1 or key2 that are being compared (depending on the > 'type' value). > -- Joseph Echeverria Cloudera, Inc. 443.305.9434
