Hi I tried ByteBuffer and it did not like that either. I finally settled on Byte[] but I'm wondering why its not smart enough to handle that construct. The Avro spec seems to imply that it should but I'm not sure if I'm reading it correctly. Bug maybe?
Thanks Paul On Tue, Nov 22, 2016 at 1:13 AM, Yibing Shi <[email protected]> wrote: > Hi Paul, > > It looks like this problem is because Avro uses ByteBuffer for "bytes" > type. GenericData treats "byte[]" as an array, but ReflectData generates an > union type (["null",{"type":"bytes","java-class":"[B"}]). > > Is it possible for you to wrap your byte[] in a ByteBuffer? > > *Yibing Shi* > *Customer Operations Engineer* > <http://www.cloudera.com> > > On Sat, Nov 19, 2016 at 1:24 AM, Paul Read <[email protected]> wrote: > >> If this is bad form I apologize up front, but I thought I should make >> some additional clarifications: >> >> 1. using GenericDatumWriter/Reader is not applicable for my application >> 2. I tried SpecificDatumWriter/Reader and got IndexRecord Exceptions >> 3. Changing it from byte[] to Byte[] will work but I need it to be as >> fast as possible to handle processing needs so converting it to Byte[] is >> something I would like to avoid. >> >> Thanks >> >> >> On Fri, Nov 18, 2016 at 7:29 AM, Paul Read <[email protected]> wrote: >> >>> Sorry hit the send button accidently...to finish >>> >>> Unit Test >>> >>> public class TestAvroHelper { >>> >>> @Test >>> public void testAvroHelper() { >>> AvroHelper<TestAvroData> helper = new >>> AvroHelper<TestAvroHelper>(TestAvroHelper.class); >>> >>> TestAvroData data = new TestAvroData(); >>> data.setData(new String("hello").toBytes()); >>> >>> byte adata = helper.toAvroBytes(data); >>> TestAvroData data1 = helper.fromAvroBytes(adata); >>> >>> } >>> } >>> >>> This fails with an Union Exception. >>> >>> What am I doing wrong? >>> >>> Thanks >>> >>> >>> >>> On Fri, Nov 18, 2016 at 7:25 AM, Paul Read <[email protected]> wrote: >>> >>>> >>>> I have a AvroHelper that serializes/deserializes about 99% of some very >>>> complex classes, The one class component it cannot seem to handle is a >>>> field that contains a simple byte array. The helper is defined >>>> >>>> public class AvroHelper<T> { >>>> >>>> private Class<T> type; >>>> private Schema schema; >>>> private DatumReader<T> reader; >>>> private DatumWriter<T> writer; >>>> private ByteArrayOutputStream bos; >>>> private Encoder encoder; >>>> private DecoderFactory decoderFactory; >>>> >>>> public AvroHelper(Class<T> type) { >>>> >>>> this.type = type; >>>> this.schema = ReflectData.AllowNull.get().getSchema(type); >>>> this.bos = new ByteArrayOutputStream(); >>>> this.decoderFactory = DencoderFactory.get(); >>>> this.encoder = EncoderFactory.get().binaryEncoder(bos, >>>> null); >>>> this.reader = new ReflectDatumReader<T>(schema); >>>> this.writer = new ReflectDatumWriter<T>(schema); >>>> } >>>> >>>> public byte[] toAvroBytes( T o) throws IOException { >>>> bos.reset(); >>>> writer.write(o, encoder); >>>> encoder.flush(); >>>> return bos.toByteArray(); >>>> } >>>> >>>> public T fromAvroBytes(byte [] raw) throws IOException { >>>> return reader.read(null, decoderFactory.binaryDecoder(raw, >>>> null)); >>>> } >>>> } >>>> >>>> The simple test POJO is >>>> >>>> public class TestAvroData { >>>> private byte [] data; >>>> >>>> public TestAvroData() { >>>> } >>>> >>>> public void setData(byte [] data) { >>>> this.data = data; >>>> } >>>> >>>> public byte[] getData() { >>>> return this.data; >>>> } >>>> } >>>> >>>> Unit Test >>>> >>>> public class TestAvroHelper { >>>> >>>> @Test >>>> public void testAvroHelper() { >>>> AvroHelper<TestAvroData> helper = new >>>> AvroHelper<TestAvroHelper>(TestAvroHelper.class); >>>> >>>> >>>> } >>>> >>> >>> >> >
