[
https://issues.apache.org/jira/browse/AVRO-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doug Cutting updated AVRO-780:
------------------------------
Attachment: AVRO-780.patch
It's hard to diagnose this, as the stack trace unfortunately does not show
where the original NullPointerException ocurred. Can you please try again with
the attached patch, which should provide a more informative stack trace?
Thanks!
> union handling in ReflectDatumWriter
> ------------------------------------
>
> Key: AVRO-780
> URL: https://issues.apache.org/jira/browse/AVRO-780
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.5.0
> Environment: Mac with VMWare running Linux training-vm
> 2.6.28-19-server #61-Ubuntu
> Reporter: ey-chih chow
> Attachments: AVRO-780.patch
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> Our avdl schema definition for the record DeviceRow has a field:
> union {array<DynamicColumn4Games>, null} Games__;
> When we migrated our MR jobs from 1.4.0 to 1.5.0, we got following messages:
> ===================================================================================================
> 11/03/10 11:31:02 INFO mapred.TaskInProgress: Error from
> attempt_20110310113041953_0001_m_000000_0: java.lang.NullPointerException: in
> com.ngmoco.hbase.DeviceRow in union null of union in field Games__ of
> com.ngmoco.hbase.DeviceRow
> at
> org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57)
> at
> org.apache.avro.mapred.AvroSerialization$AvroWrapperSerializer.serialize(AvroSerialization.java:131)
> at
> org.apache.avro.mapred.AvroSerialization$AvroWrapperSerializer.serialize(AvroSerialization.java:114)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:900)
> at
> org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:466)
> at
> org.apache.avro.mapred.HadoopMapper$MapCollector.collect(HadoopMapper.java:69)
> at
> com.ngmoco.ngpipes.sourcing.NgActivityGatheringMapper.map(NgActivityGatheringMapper.java:91)
> at
> com.ngmoco.ngpipes.sourcing.NgActivityGatheringMapper.map(NgActivityGatheringMapper.java:1)
> at org.apache.avro.mapred.HadoopMapper.map(HadoopMapper.java:80)
> at org.apache.avro.mapred.HadoopMapper.map(HadoopMapper.java:34)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.NullPointerException: in union null of union in field
> Games__ of com.ngmoco.hbase.DeviceRow
> at
> org.apache.avro.generic.GenericDatumWriter.npe(GenericDatumWriter.java:92)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:86)
> at
> org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:102)
> ... 14 more
> ===================================================================================================
> After we change definition of the field Games__ to:
> union {null, array<DynamicColumn4Games>} Games__;
> The system stop complaining.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
