Hi all,
Quick question. I'm trying to do a very simple MR job just doing nothing...
Just to try to get it run.
But as soon as I set the output value to be KeyValue or Put, I get
exception from the MR framework.
The exception is the following:
java.lang.Exception: java.lang.NullPointerException
at
org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:988)
at
org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391)
at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:80)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:675)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:747)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
If you look into the code, that means that Hadoop is not able to serialize
KeyValue nor Put and so is not able to usethem in the Mapper class.
To validate, I tried this:
SerializationFactory serializationFactory = new
SerializationFactory(conf);
System.out.println
(serializationFactory.getSerializer(KeyValue.class));
System.out.println (serializationFactory.getSerializer(Put.class));
System.out.println (serializationFactory.getSerializer(Cell.class));
And they all return null. Which is consistent with the exception.
So you don't even need to run MR to get it fails. Just a small main with
those 4 lines.
Am I missing something? Like, doing some initialization to help Hadoop to
serialize those classes?
Thanks,
JM
