Hi,
i have been trying to implement custom input and output formats. i
was successful enough in creating an custom output format. but whn i call a
mapreduce method which takes in a file, using custom input format, i get an
exception.
java.lang.NullPointerException
at Beans.Content.write(Content.java:54)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:613)
at CustomInputFormat.SampleCustomInputMap.map(SampleCustomInputMap.java:32)
at CustomInputFormat.SampleCustomInputMap.map(SampleCustomInputMap.java:1)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138)
0 Hello World0 Hello World0 Hello World0
id:: null media:: null url:: null content::null
id:: null media:: null url:: null content::null
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
at
CustomInputFormat.SampleCustomInputMapReduce.run(SampleCustomInputMapReduce.java:53)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at
CustomInputFormat.SampleCustomInputMapReduce.main(SampleCustomInputMapReduce.java:59)
I have attached the following files
Content => Custom object that implements writercomparible
ContentInputFormatV2 => inputformat which implements SequenceFileInputFormat
ContentRecordReader => implementation of RecordReader (Not required though,
it should work w/o it i assume).
SampleCustomInputMap => mapper class
SampleCustomInputReduce => reducer class
SampleCustomInputMapReduce => class which contains the main method and the
job configurations.
data and index => my inputfiles for the main function
For the 1st record it works fine, but for next record and records after
that, i get null. where could i go wrong??
Regards,
Raakhi
