Hi GZ, You're absolutely right about both "&" and the length/contents. It turned out that it is a bug: as you pointed out the write method ignores some bytes written. So the data is written and ready to be deserialized but the reported length is wrong. If you happen to be building Thrift from the source, the fix will be available on master branch soon.
A very dirty and non-performant workaround is to deserialize the buffer only to know the byte length before sending: wrong_len = cls->write(THRIFT_STRUCT(pData2), ...); correct_len = cls->read(THRIFT_STRUCT(pData2), ...); you'll see correct byte length since read is not affected by this bug. On Sun, Nov 29, 2015 at 4:22 AM GZ <[email protected]> wrote: > Hi Nobuaki, > > I tried this originally but write is expecting a ThriftStruct* rather than > a ThriftStruct. I have tried with both &pData->parent and pData.. the > results are pretty much the same. Below is a Hex printout of the memory > buffer (19 bytes). > > &pData->parent > 01FB5040 | 1081FB01 00000000 52000000 80000000 | ........R....... > 01FB5050 | 010000 > > pData > 01EA4040 | 1071EA01 00000000 52000000 80000000 | .q......R....... > 01EA4050 | 010000 > > > I would expect to see the "XXXXXXX" string serialised into this buffer but > it is not. > > George > > > > Nobuaki Sukegawa <[email protected]> wrote .. > > Hi GZ, > > > > Can you try with "pData->parent" part applied too ? > > Sorry I should have mentioned this too. > > > > > > On Sat, Nov 28, 2015 at 8:25 AM GZ <[email protected]> wrote: > > > > > Hi Nobuaki, > > > > > > Yes - the rdata_write was in the generated c file not the header. > > > > > > I made the changes you suggested, it was just what I was looking for. I > > > have repeated the corrected code to help others but unfortunately the > > > original problem persists.. > > > > > > > > > tbuffer = g_object_new (THRIFT_TYPE_MEMORY_BUFFER, "buf_size", > 1000, > > > NULL); > > > if(tbuffer){ > > > transport = THRIFT_TRANSPORT(tbuffer); > > > thrift_transport_open (transport, &error); > > > > > > protocol = THRIFT_PROTOCOL(g_object_new > > > (THRIFT_TYPE_BINARY_PROTOCOL, "transport", transport, NULL)); > > > if(protocol){ > > > rdata* pData; > > > pData = g_object_new (TYPE_RDATA,"flag",1, > > > "id",27, > > > "start",12000, > > > "done",345435, > > > "notif",34455, > > > > > > "spid","XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",NULL); > > > if(pData){ > > > ThriftStructClass* cls = > > > THRIFT_STRUCT_CLASS(RDATA_GET_CLASS(pData)); > > > len = cls->write(pData,protocol,&error); > > > > > > printf ("%d Bytes written to buffer\n", len); > > > > > > if(tbuffer->buf != NULL){ > > > printHex((const void *)tbuffer->buf, 100); > > > } > > > g_object_unref (pData); > > > } > > > g_object_unref (protocol); > > > } > > > g_object_unref(tbuffer); > > > } > > > > > > > > > The returned buffer length still appears incorrect and the string does > not > > > appear to be serialised into the buffer. > > > > > > I can track the writes through the writing function and I notice that > for > > > the spid string value > > > > > > thrift_protocol_write_field_begin returns 3 bytes > > > thrift_protocol_write_string returns 50 bytes > > > thrift_protocol_write_field_end returns 0 > > > > > > the 50 bytes never get added to the length because it is overriden by > the > > > 0. This is a snippit of the code.. > > > > > > if ((ret = thrift_protocol_write_field_begin (protocol, "spid", > > > T_STRING, 6, error)) < 0) > > > return -1; > > > xfer += ret; > > > if ((ret = thrift_protocol_write_string (protocol, this_object->spid, > > > error)) < 0) > > > return -1; > > > if ((ret = thrift_protocol_write_field_end (protocol, error)) < 0) > > > return -1; > > > xfer += ret; > > > > > > The result is that the write function returns 19 bytes and when I view > the > > > buffer I see no string. I guess I must still be missing something... > The > > > spid string is 46 characters long. I would expect to see this > reflected in > > > the length of the buffer. > > > > > > The idl record is > > > > > > enum Status { > > > ONE = 1, > > > TWO = 2, > > > THREE = 4, > > > } > > > > > > struct rdata { > > > 1: Status flag, > > > 2: i16 id, > > > 3: i64 start, > > > 4: i64 done, > > > 5: i64 notif, > > > 6: string spid > > > } > > > > > > Any idea what might be happenning? > > > > > > Thanks again... > > > > > > George > > > > > > > > > > > > > > > > > > > > > Nobuaki Sukegawa <[email protected]> wrote .. > > > > It is strange, the generated header has rdata_write ? > > > > Seeing generated code, it should be something like this: > > > > > > > > ThriftStructClass* cls = THRIFT_STRUCT_CLASS(RDATA_GET_CLASS(pData)); > > > > cls->write(pData->parent, protocol, &error); > > > > > > > > On Thu, Nov 26, 2015 at 8:03 AM GZ <[email protected]> wrote: > > > > > > > > > I am new to thrift and have been trying to serialize a thrift > structure > > > > > using c but have been unable to get it to work. I want to serialize > > > using > > > > > the binary protocol into a memory buffer. I do not need RPC, I > will be > > > > > passing the message into Kafka. Does anyone know the correct way > to do > > > this? > > > > > > > > > > I have looked through all the examples and test code and have come > up > > > with > > > > > the following. It compiles, but is not working correctly. I am > clearly > > > not > > > > > doing it correctly. rdata_write is a method generated by thrift. > > > > > > > > > > GError *error = NULL; > > > > > ThriftTransport *transport = NULL; > > > > > ThriftProtocol *protocol = NULL; > > > > > > > > > > transport=g_object_new (THRIFT_TYPE_MEMORY_BUFFER, "buf_size", > 100, > > > > > NULL); > > > > > if(transport){ > > > > > thrift_transport_open (transport, &error); > > > > > protocol = g_object_new (THRIFT_TYPE_BINARY_PROTOCOL, > > > "transport", > > > > > transport,NULL); > > > > > if(protocol){ > > > > > rdata* pData = g_object_new (TYPE_RDATA,"flag",1, > > > > > "id",27, > > > > > "start",12000, > > > > > "done",345435, > > > > > "notif",34455, > > > > > "spid","This is the > SPID",NULL); > > > > > if(pData){ > > > > > > > > > > /* wanted something like > obj->write(protocol,&error) > > > > > But how to get object? pData does not have write > > > method > > > > > */ > > > > > > > > > > gint32 len = > > > > > rdata_write(pData,(ThriftProtocol*)protocol,&error); > > > > > > > > > > // Print Content of Buffer > > > > > ThriftMemoryBuffer* pMem = > > > (ThriftMemoryBuffer*)transport; > > > > > if(pMem->buf != NULL){ > > > > > printHex((const void *)pMem->buf, len); > > > > > } > > > > > g_object_unref (pData); > > > > > } > > > > > g_object_unref (protocol); > > > > > } > > > > > g_object_unref(transport); > > > > > } > > > > > > > > > > Error handling and support functions skipped for brevity. > > > > > > > > > > Any help appreciated > > > > > > > > >
