Thanks for the response Anand, The NullPointerException was by design, I wanted to illustrate that it was not null. Your suspicion about having multiple threads was correct. I traced the error down to the threadpool terminating prior to the last mapper completing its operations.
Sincerely, Matthew Berry -----Original Message----- From: Anand Srivastava [mailto:anand.srivast...@guavus.com] Sent: Tuesday, July 03, 2012 8:31 PM To: <mapreduce-user@hadoop.apache.org> Subject: Re: Null Pointer Excpetion while writing Key on Mapper emit Hi Matt, You StringOne is clearly not null else it would have thrown an exception in the if. You should not be calling equals on String unless you are sure that it is not null; use == instead for nullity checks. This means that one of 'kvbuffer' or 'b' (byte[] argument) to be null. Mind telling us the hadoop version you are using? Any chance that there are two or more threads in your mapper class? Regards, Anand On 04-Jul-2012, at 2:46 AM, Berry, Matt wrote: > While emitting a record from my mapper, I am receiving a > NullPointerException. The stack trace seems to indicate there is a problem > serializing the Key. The key contains a few strings and a few longs. It is a > string that it is having trouble serializing. I've checked the contents of > the string post-mortem and it looks like a valid string (at least, other > strings like it work just fine). Could this be a problem not with the string > itself but the destination of the arraycopy()? > > The Key.write(DataOutput out) function: > > @Override > public void write(DataOutput out) throws IOException { > if(StringOne.equals(null) | StringTwo.equals(null)) { > System.out.println("This is not hit"); > } > out.writeUTF(StringOne); //Line 125 > out.writeUTF(StringTwo); > out.writeLong(LongOne); > out.writeLong(LongTwo); > } > > > The Stack Trace: > > java.lang.NullPointerException > at java.lang.System.arraycopy(Native Method) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1108) > at java.io.DataOutputStream.write(DataOutputStream.java:90) > at java.io.DataOutputStream.writeUTF(DataOutputStream.java:384) > at java.io.DataOutputStream.writeUTF(DataOutputStream.java:306) > at xxxxxx.xxxxxxxxxxxx.xxxx.xxxxx.xxxxxKey.write(xxxxxKey.java:125) > at > org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90) > at > org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:892) > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541) > at > org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > at > xxxxxx.xxxxxxxxxxx.xxxx.xxxxxxxx.xxx.xxxxxxxxxxxxxxxxxxxxxxx.map(xxxxxxxxxxxxxxxxxMapper.java:82) > at > xxxxxx.xxxxxxxxxxx.xxxx.xxxxxxxx.xxx.xxxxxxxxxxxxxxxxxxxxxxx.map(xxxxxxxxxxxxxxxxxMapper.java:63) > at > xxxxxx.xxxxxxxxxxx.xxxx.xxxxxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxxx.xxxxxxxxxx(xxxxxxxxxxxxxxxxxxxxxxxxxxxx.java:131) > at > xxxxxx.xxxxxxxxxxx.xxxx.xxxxx.xxxxxxxxxxxxxx.xxxxxxxxxxxxxxx(xxxxxxxxxxxxxx.java:282) > at > xxxxxx.xxxxxxxxxxx.xxxx.xxxxx.xxxxxxxxxxxxxx.access$000(xxxxxxxxxxxxxx.java:42) > at > xxxxxx.xxxxxxxxxxx.xxxx.xxxxx.xxxxxxxxxxxxxx$xxxxxxxxxxxx.run(xxxxxxxxxxxxx.java:74) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662)
