Re: Sending Apache avro serialized data to thrift server

[Avinash Dongre]

Thanks Martin,
With following code exceptions goes away.

DatumReader<GenericRecord> reader = new
GenericDatumReader<GenericRecord>(schema);

    byte[] avroSerialized = new byte[input.limit() - input.position()];
    input.put(avroSerialized);
    BinaryDecoder decoder =
DecoderFactory.get().binaryDecoder(avroSerialized, null);

    try {
      GenericRecord result = reader.read(null, decoder);
      System.out.println("RESULT : " + result.toString());
    } catch (IOException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }

But I am not getting any values when I do toString, I will investigate
further on what is wrong with my code.

Thanks for your quick help.


Thanks
Avinash



On Mon, Jun 17, 2013 at 11:16 AM, Martin Kleppmann <mar...@rapportive.com>wrote:

> It's hard to tell just by looking at the code -- I suggest you inspect the
> contents of the array you're decoding in each case, and see if there is any
> difference.
>
> Purely guessing, the ByteBuffer that Thrift gives you may have other data
> before and after the Avro record, and calling `bf.array()` will give you
> junk in that case. You may need something more like this (rough outline,
> please read the API docs):
>
> byte[] copy = new byte[bf.limit() - bf.position()];
> bf.read(copy);
> DecoderFactory.get().binaryDecoder(copy, null);
>
>
>
> On 16 June 2013 20:15, Avinash Dongre <dongre.avin...@gmail.com> wrote:
>
>>
>> Hi,
>>> I am trying to send avro serialized data to my thrift server. But when I
>>> try to de-serialized I get exception
>>> org.apache.avro.AvroRuntimeException: Malformed data. Length is
>>> negative: -51
>>>
>>> If I do the same operation in the same vm process everything works fine.
>>>
>>> Here is my small code.
>>>
>>> public static GenericRecord createContentNestedObject() throws Exception
>>> {
>>>     GenericRecord image1 = new GenericData.Record(IMAGE_SCHEMA);
>>>     image1.put("uri", new Utf8("http://javaone.com/keynote_large.jpg";));
>>>     image1.put("width", 0);
>>>     image1.put("height", 0);
>>>     image1.put("size", 2);
>>>     image1.put("title", new Utf8("Javaone Keynote"));
>>>     return image1;
>>>   }
>>>
>>>   // Helper Method to serialize the object from avro to bytebuffer
>>>   public static ByteBuffer serialize(GenericRecord content, Schema
>>> schema)
>>>        throws Exception {
>>>
>>>     ByteArrayOutputStream out = new ByteArrayOutputStream();
>>>     DatumWriter<GenericRecord> writer = new
>>> GenericDatumWriter<GenericRecord>(
>>>         schema);
>>>     // create Binary Encoder
>>>     EncoderFactory ef = new EncoderFactory();
>>>     BinaryEncoder be = ef.binaryEncoder(out, null);
>>>     writer.write(content, be);
>>>      be.flush();
>>>     out.close();
>>>
>>>     return ByteBuffer.wrap(out.toByteArray());
>>>   }
>>>
>>>   public static void main(String[] args) throws Exception {
>>>     try {
>>>       GenericRecord rd = createContentNestedObject();
>>>       bf = serialize(rd, IMAGE_SCHEMA);
>>>
>>>     } catch (Exception e) {
>>>       // TODO Auto-generated catch block
>>>       e.printStackTrace();
>>>     }
>>>
>>>     // Deserialization.
>>>     DatumReader<GenericRecord> reader = new
>>> GenericDatumReader<GenericRecord>(IMAGE_SCHEMA_1);
>>>     ByteArrayOutputStream out = new ByteArrayOutputStream();
>>>     try {
>>>       out.write(bf.array());
>>>     } catch (IOException e1) {
>>>       // TODO Auto-generated catch block
>>>       e1.printStackTrace();
>>>     }
>>>     BinaryDecoder decoder =
>>> DecoderFactory.get().binaryDecoder(out.toByteArray(), null);
>>>     try {
>>>       GenericRecord result = reader.read(null, decoder);
>>>       System.out.println("RESULT : " + result.toString());
>>>     } catch (IOException e) {
>>>       // TODO Auto-generated catch block
>>>       e.printStackTrace();
>>>     }
>>>
>>> This code works. When I send the bf to my thrift api , I am getting the
>>> error,
>>> my thrift api is simple which accept ByteBuffer and try to deserialize
>>> it.
>>>
>>> following is the code in the thrift server api
>>>
>>> DatumReader<GenericRecord> reader = new
>>> GenericDatumReader<GenericRecord>(IMAGE_SCHEMA_1);
>>>     ByteArrayOutputStream out = new ByteArrayOutputStream();
>>>     try {
>>>       out.write(bf.array());
>>>     } catch (IOException e1) {
>>>       // TODO Auto-generated catch block
>>>       e1.printStackTrace();
>>>     }
>>>     BinaryDecoder decoder =
>>> DecoderFactory.get().binaryDecoder(out.toByteArray(), null);
>>>     try {
>>>       GenericRecord result = reader.read(null, decoder);
>>>       System.out.println("RESULT : " + result.toString());
>>>     } catch (IOException e) {
>>>       // TODO Auto-generated catch block
>>>       e.printStackTrace();
>>>     }
>>>
>>>
>>>
>>> What is wrong here.
>>>
>>>
>>> Thanks
>>> Aviansh
>>>
>>>
>>>
>>
>