Jim Donofrio created MRUNIT-105:
-----------------------------------
Summary: java serialization tests fail to set group comparator
Key: MRUNIT-105
URL: https://issues.apache.org/jira/browse/MRUNIT-105
Project: MRUnit
Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Jim Donofrio
Assignee: Jim Donofrio
Fix For: 1.0.0
>From
>http://mail-archives.apache.org/mod_mbox/incubator-mrunit-dev/201204.mbox/%3C4F927519.2040207%40gmx.net%3E
java.lang.ClassCastException: java.lang.Integer cannot be cast to
org.apache.hadoop.io.WritableComparable
at
org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:119)
at java.util.TreeMap.compare(TreeMap.java:1188)
at java.util.TreeMap.put(TreeMap.java:531)
at
org.apache.hadoop.mrunit.MapReduceDriverBase.shuffle(MapReduceDriverBase.java:172)
at org.apache.hadoop.mrunit.MapReduceDriver.run(MapReduceDriver.java:328)
at
org.apache.hadoop.mrunit.MapReduceDriverBase.runTest(MapReduceDriverBase.java:137)
at org.apache.hadoop.mrunit.TestDriver.runTest(TestDriver.java:158)
at
org.apache.hadoop.mrunit.TestMapReduceDriver.testJavaSerialization(TestMapReduceDriver.java:409)
public int compare(Object a, Object b) {
return compare((WritableComparable)a, (WritableComparable)b);
}
Java 7:
Entry<K,V> t = root;
if (t == null) {
compare(key, key); // type (and possibly null) check
root = new Entry<>(key, value, null);
size = 1;
modCount++;
return null;
}
Java 6:
Entry<K,V> t = root;
if (t == null) {
// TBD:
// 5045147: (coll) Adding null to an empty TreeSet should
// throw NullPointerException
//
// compare(key, key); // type check
root = new Entry<K,V>(key, value, null);
size = 1;
modCount++;
return null;
}
Oops this is a bug in test case not the actual code, good catch. This
problem is not unique to Java 7. The problem only appears in Java 7 because
my java serialization tests only use 1 input value. Java 6's put method in
TreeMap does not call compare when adding to an empty map while Java 7's put
method does call compare when adding to an empty map.
This can be fixed in the test case by setting a
OutputValueGroupingComparator in the conf for Integer. Users that use java
or other non Writable serializations would know they have to set the
relevant comparators.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
