[ https://issues.apache.org/jira/browse/MAPREDUCE-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated MAPREDUCE-1126: ------------------------------------- Attachment: MAPREDUCE-1126.patch Attaching a patch that makes this work. This depends on HADOOP-6438 which adds the configuration getters/setters discussed earlier. This patch adds the following new files: {code} svn add src/test/mapred/org/apache/hadoop/mapreduce/TestAvroSerialization.java svn add src/test/mapred/org/apache/hadoop/mapreduce/avro/key.avsc svn add src/test/mapred/org/apache/hadoop/mapreduce/avro/val.avsc {code} Tested by running Mapreduce unit tests; added new unit tests that use the Avro serialization systems (generic, specific, and reflect) for intermediate data types. Booted a pseudo-distributed cluster and ran some example jobs, which work. Will mark as patch-available after the dependencies are all met > shuffle should use serialization to get comparator > -------------------------------------------------- > > Key: MAPREDUCE-1126 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1126 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: task > Reporter: Doug Cutting > Assignee: Aaron Kimball > Attachments: MAPREDUCE-1126.patch > > > Currently the key comparator is defined as a Java class. Instead we should > use the Serialization API to create key comparators. This would permit, > e.g., Avro-based comparators to be used, permitting efficient sorting of > complex data types without having to write a RawComparator in Java. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.