NullPointerException in WritableComparator
------------------------------------------
Key: MAPREDUCE-2564
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2564
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: task
Affects Versions: 0.20.203.0
Environment: java version "1.6.0_23"
Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)
hadoop-0.20.203.0rc1
Reporter: Joseph Shraibman
Priority: Blocker
java.lang.NullPointerException
at
org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:96)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1110)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:70)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1398)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1297)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:698)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:765)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
It is easy to see why this is happening. The WritableComparator is created in
JobConf line 776:
{code:title=JobConf.java}
return
WritableComparator.get(getMapOutputKeyClass().asSubclass(WritableComparable.class));
}
{code}
which calls
{code:title=WritableComparator.java|borderStyle=solid}
protected WritableComparator(Class<? extends WritableComparable> keyClass) {
this(keyClass, false);
}
protected WritableComparator(Class<? extends WritableComparable> keyClass,
boolean createInstances) {
this.keyClass = keyClass;
if (createInstances) {
key1 = newKey();
key2 = newKey();
buffer = new DataInputBuffer();
} else {
key1 = key2 = null;
buffer = null;
}
}
{code}
key1, key2, and buffer end up being null. When compare() is called the NPE is
thrown because buffer is null
{code:title=WritableComparator.java|borderStyle=solid}
public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) {
try {
buffer.reset(b1, s1, l1); // parse key1
key1.readFields(buffer);
buffer.reset(b2, s2, l2); // parse key2
key2.readFields(buffer);
} catch (IOException e) {
throw new RuntimeException(e);
}
return compare(key1, key2); // compare them
}
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
