Hi Byran,
The Protocol and Transport classes provide nice abstractions but not
really needed by us. A function of the kind "int toByteArray(byte[] dst, int
dstLen)" would have worked for me which does not take care of Endianness, not
does it insert field ids (for backward and forward compatibility) if it
performed 20% faster.
We don't mind enhancing the code generator but are worried about having to keep
porting it on every Thrift upgrade.
I am open to trying a different implementation altogether even if it has less
features but performs better. (auto-generation of robust
serialization-deserialization code for nested structs being a mandatory
feature).
We are going to use these as key-value objects in Hadoop map-reduce and would
write the comparators when required.
Regards,
Anand
On 09-Apr-2012, at 8:19 PM, Bryan Duxbury wrote:
> What kind of "other abstractions" would you like to turn off?
>
> On Sun, Apr 8, 2012 at 10:50 PM, Anand Srivastava <
> [email protected]> wrote:
>
>> Hi,
>> We have been using Thrift objects when we want to serialize to disk and
>> back (in C++ and Java). While profiling, we have noticed that hard written
>> serializers for our specific objects can perform much better (~2x) than
>> Thrift. While we understand it is unreasonable to expect Thrift to match
>> their performance given the generic usecases it solves, we have been were
>> wondering if we can somehow keep using the auto-generated
>> serialization-deserialization code but not pay for the other abstractions
>> provided.
>> So, we don't care about the language/architecture independence provided or
>> the backward compatibility. And suggestions on possible performance
>> improvements as a tradeoff for some features would be useful.
>>
>> We have tried TBinaryProtocol and TCompactProtocol along with
>> TMemoryBuffer.
>>
>> The objects we are interested in as similar to the one below:
>>
>> Java:
>> public class MeasureSet {
>> public double[] simple;
>> public ArrayList<ArrayList<Long>> complex;
>> public ArrayList<ByteBuffer> others;
>> }
>>
>> IDL:
>> struct ThriftMeasureSet {
>> 1:list<double> simple,
>> 2:list<list<i64>> complex,
>> 3:list<binary> others
>> }
>>
>> We have tried Protocol Buffer as well and find their performance to be
>> similar to Thrift. Any pointers as welcome.
>>
>> Thanks and Regards,
>> Anand
>>
