Thanks once again Harsh. Thanks & Regards, B Anil Kumar.
On Sat, Feb 8, 2014 at 11:48 PM, Harsh J <[email protected]> wrote: > I've responded to this on your original post over the appropriate > user@avro lists: http://search-hadoop.com/m/icC8CkA4h1 > > On Sat, Feb 8, 2014 at 11:47 PM, AnilKumar B <[email protected]> > wrote: > > Hi, > > > > In my Mapreduce job, I am using AvroKeyValueOutputFormat as one of my > > MultipleOutputs. So I declared my multiple outputs as below. > > > > MultipleOutputs.addNamedOutput(stage2Job, > > SessionConstants.COMPLETED_SESSIONS, > > AvroKeyValueOutputFormat.class, AvroKey.class, AvroValue.class); > > > > And In reducer, I am constructing and emitting GenericData.Record for the > > below schema: > > > > sessionSchema: > > {"namespace": "ci.avro", > > "type": "record", > > "name": "Session", > > "fields": [ > > {"name":"Common", "type": { > > "type": "map", "values":"string"}}, > > {"name":"events", > > "type": { > > "type": "array", > > "items":{ > > "name":"Event", > > "type":"map", > > "values":"string"} > > } > > } > > ] > > } > > > > eventSchema: > > {"namespace": "ci.avro", > > "type": "record", > > "name": "AvroEvent", > > "fields": [ > > {"name":"Event", > > "type": { > > "type": "map", "values":"string" > > } > > } > > ] > > } > > > > > > // > > sessionRecord = new AvroValue<GenericData.Record>(null); > > > > //record generation > > > > GenericData.Record record = new GenericData.Record(sessionSchema); > > GenericData.Record eRecord = new GenericData.Record(eventSchema); > > GenericData.Array<GenericData.Record> eventRecords = > > new GenericData.Array<GenericData.Record>(vc.getEvents().size(), > > sessionSchema.getField("events").schema()); > > record.put("Common", vc.getCommon().getM_parameterMap()); > > for (Event ev : vc.getEvents()) { > > eRecord = new GenericData.Record(eventSchema); > > eRecord.put("Event", ev.getM_parameterMap()); > > eventRecords.add(eRecord); > > } > > record.put("events", eventRecords); > > > > sessionRecord.datum(record); > > > > > > // record emmitted as below > > context.getConfiguration().set(CONF_OUTPUT_KEY_SCHEMA, > > Schema.create(Schema.Type.STRING).toString()); > > context.getConfiguration().set(CONF_OUTPUT_VALUE_SCHEMA, > > sessionSchema.toString()); > > multipleOutputs.write(SessionConstants.COMPLETED_SESSIONS, > > new AvroKey<String>(key.toString()), sessionRecord, > > SessionConstants.COMPLETED_SESSIONS); > > > > But I am getting below exception. I am also declaring > > "avro.schema.output.value" as sessionSchema.toString(). What could be > the > > issue? > > > > I am not understanding why it's trying to cast GenericData$Record to > > java.util.Map. > > > > Exception: > > org.apache.avro.file.DataFileWriter$AppendWriteException: > > java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record > > cannot be cast to java.util.Map > > at > > org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:296) > > at > > > org.apache.avro.mapreduce.AvroKeyValueRecordWriter.write(AvroKeyValueRecordWriter.java:127) > > at > > > org.apache.hadoop.mapreduce.lib.output.MultipleOutputs$RecordWriterWithCounter.write(MultipleOutputs.java:304) > > at > > > org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.write(MultipleOutputs.java:370) > > at > > > com.paypal.ci.CISuperSessionStage2AvroReducer.reduce(CISuperSessionStage2AvroReducer.java:156) > > at > > > com.paypal.ci.CISuperSessionStage2AvroReducer.reduce(CISuperSessionStage2AvroReducer.java:24) > > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177) > > at > > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649) > > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418) > > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) > > at org.apache.hadoop.mapred.Child.main(Child.java:249) > > Caused by: java.lang.ClassCastException: > > org.apache.avro.generic.GenericData$Record cannot be cast to > java.util.Map > > at > > > org.apache.avro.generic.GenericDatumWriter.getMapSize(GenericDatumWriter.java:194) > > at > > > org.apache.avro.generic.GenericDatumWriter.writeMap(GenericDatumWriter.java:173) > > at > > > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:69) > > at > > > org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:143) > > at > > > org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:138) > > at > > > org.apache.avro.reflect.ReflectDatumWriter.writeArray(ReflectDatumWriter.java:64) > > at > > > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:68) > > at > > > org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:143) > > at > > > org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114) > > at > > > org.apache.avro.reflect.ReflectDatumWriter.writeField(ReflectDatumWriter.java:175) > > at > > > org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104) > > at > > > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66) > > at > > > org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:143) > > at > > > org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114) > > at > > > org.apache.avro.reflect.ReflectDatumWriter.writeField(ReflectDatumWriter.java:175) > > at > > > org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104) > > at > > > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66) > > at > > > org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:143) > > at > > > org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58) > > at > > org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:290) > > > > > > > > Thanks & Regards, > > B Anil Kumar. > > Thanks & Regards, > > B Anil Kumar. > > > > -- > Harsh J >
