On 02/04/2011 10:11 AM, Marco Didonna wrote: > On 02/03/2011 07:02 PM, Harsh J wrote: >> For a ValueGrouping comparator to work, your Partitioner must act in >> tandem with it. I do not know if you have implemented a custom >> hashCode() method for your Key class, but your partitioner should look >> like: > > Yes I did and it works like this return "leftElement.hashCode() + > rightElement; " > >> >> return (key.getLeftElement().hashCode() & Integer.MAX_VALUE) % numPartitions; >> > > This was definitely a bug, the result is always the same though :( > >> This will ensure that the to-be grouped data is actually partitioned >> properly too. >> >> The actual sorting (which ought to occur for the full composite key >> field-by-field, and is the only real 'sorter') would be handled by the >> compare() call of your Writable, if you are using a >> WritableComparable. > > I am using a WritableComparable...here's PairOfStringInt > https://gist.github.com/810905 > > Thanks again > >
I finally made it https://gist.github.com/809803 I use the groupingComparator as job.setSortComparatorClass(GroupingComparator.class) I still do not understand what was wrong with the old version of the GroupingComparator and when the key are ordered according to the policy encoded in GroupingComparator. MD
