Hi Paul,
It looks like this problem is because Avro uses ByteBuffer for "bytes"
type. GenericData treats "byte[]" as an array, but ReflectData generates an
union type (["null",{"type":"bytes","java-class":"[B"}]).
Is it possible for you to wrap your byte[] in a ByteBuffer?
*Yibing Shi*
*Customer Operations Engineer*
<http://www.cloudera.com>
On Sat, Nov 19, 2016 at 1:24 AM, Paul Read <[email protected]> wrote:
> If this is bad form I apologize up front, but I thought I should make some
> additional clarifications:
>
> 1. using GenericDatumWriter/Reader is not applicable for my application
> 2. I tried SpecificDatumWriter/Reader and got IndexRecord Exceptions
> 3. Changing it from byte[] to Byte[] will work but I need it to be as fast
> as possible to handle processing needs so converting it to Byte[] is
> something I would like to avoid.
>
> Thanks
>
>
> On Fri, Nov 18, 2016 at 7:29 AM, Paul Read <[email protected]> wrote:
>
>> Sorry hit the send button accidently...to finish
>>
>> Unit Test
>>
>> public class TestAvroHelper {
>>
>> @Test
>> public void testAvroHelper() {
>> AvroHelper<TestAvroData> helper = new
>> AvroHelper<TestAvroHelper>(TestAvroHelper.class);
>>
>> TestAvroData data = new TestAvroData();
>> data.setData(new String("hello").toBytes());
>>
>> byte adata = helper.toAvroBytes(data);
>> TestAvroData data1 = helper.fromAvroBytes(adata);
>>
>> }
>> }
>>
>> This fails with an Union Exception.
>>
>> What am I doing wrong?
>>
>> Thanks
>>
>>
>>
>> On Fri, Nov 18, 2016 at 7:25 AM, Paul Read <[email protected]> wrote:
>>
>>>
>>> I have a AvroHelper that serializes/deserializes about 99% of some very
>>> complex classes, The one class component it cannot seem to handle is a
>>> field that contains a simple byte array. The helper is defined
>>>
>>> public class AvroHelper<T> {
>>>
>>> private Class<T> type;
>>> private Schema schema;
>>> private DatumReader<T> reader;
>>> private DatumWriter<T> writer;
>>> private ByteArrayOutputStream bos;
>>> private Encoder encoder;
>>> private DecoderFactory decoderFactory;
>>>
>>> public AvroHelper(Class<T> type) {
>>>
>>> this.type = type;
>>> this.schema = ReflectData.AllowNull.get().getSchema(type);
>>> this.bos = new ByteArrayOutputStream();
>>> this.decoderFactory = DencoderFactory.get();
>>> this.encoder = EncoderFactory.get().binaryEncoder(bos, null);
>>> this.reader = new ReflectDatumReader<T>(schema);
>>> this.writer = new ReflectDatumWriter<T>(schema);
>>> }
>>>
>>> public byte[] toAvroBytes( T o) throws IOException {
>>> bos.reset();
>>> writer.write(o, encoder);
>>> encoder.flush();
>>> return bos.toByteArray();
>>> }
>>>
>>> public T fromAvroBytes(byte [] raw) throws IOException {
>>> return reader.read(null, decoderFactory.binaryDecoder(raw,
>>> null));
>>> }
>>> }
>>>
>>> The simple test POJO is
>>>
>>> public class TestAvroData {
>>> private byte [] data;
>>>
>>> public TestAvroData() {
>>> }
>>>
>>> public void setData(byte [] data) {
>>> this.data = data;
>>> }
>>>
>>> public byte[] getData() {
>>> return this.data;
>>> }
>>> }
>>>
>>> Unit Test
>>>
>>> public class TestAvroHelper {
>>>
>>> @Test
>>> public void testAvroHelper() {
>>> AvroHelper<TestAvroData> helper = new
>>> AvroHelper<TestAvroHelper>(TestAvroHelper.class);
>>>
>>>
>>> }
>>>
>>
>>
>
