Hi,all. I try to use GenericRecord as value in mapreduce:
public void map(GenericRecord record, AvroCollector<Pair<Utf8,GenericRecord>>
collector, Reporter reporter)
....
collector.collect(new Pair<Utf8,GenericRecord>(new Utf8(key),value));
here, object value is a GenericRecord. Then I run this job, exception comes
out:
11/04/13 02:23:05 WARN mapred.LocalJobRunner: job_local_0001
java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record
incompatible with java.lang.CharSequence
at
org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:189)
at
org.apache.avro.reflect.ReflectDatumWriter.writeString(ReflectDatumWriter.java:85)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:77)
at
org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:102)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
at
org.apache.avro.mapred.AvroSerialization$AvroWrapperSerializer.serialize(AvroSerialization.java:131)
at
org.apache.avro.mapred.AvroSerialization$AvroWrapperSerializer.serialize(AvroSerialization.java:1)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:900)
at
org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:466)
at
org.apache.avro.mapred.HadoopMapper$MapCollector.collect(HadoopMapper.java:69)
at com.ibm.crl.manager.MREngine$AggregationMapper.map(MREngine.java:73)
at com.ibm.crl.manager.MREngine$AggregationMapper.map(MREngine.java:1)
at org.apache.avro.mapred.HadoopMapper.map(HadoopMapper.java:80)
at org.apache.avro.mapred.HadoopMapper.map(HadoopMapper.java:1)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
It seems that the GenericDatumWriter writes my record by using writeString()
instead of writeRecord()..
