Thanks Feng Xiao and Henner Zeller :) >From what I read in the protocol specification and wire format, I came to the same conclusion: At least on the wire bytes fields seem to be implemented efficiently. Only remaining question will be, if serialization/deserialization is avoiding mutiple copies of the data and handles memory allocation efficiently for big bytes fields. I will try to implement some simple scenarios and see if the performance is sufficient...
On Thursday, January 11, 2018 at 8:09:24 PM UTC+1, Feng Xiao wrote: > > > > On Thu, Jan 11, 2018 at 4:58 AM, <[email protected] <javascript:>> wrote: > >> Thanks for the reply :) >> >> I was planning to use the "bytes" Proto-Type from the proto3 language >> specification (see here: >> https://developers.google.com/protocol-buffers/docs/proto3). >> It seems to be suited to transfer binary blobs. >> >> However in the Techniques section of the ProtocolBuffers documentation ( >> https://developers.google.com/protocol-buffers/docs/techniques) it is >> mentioned, that "Protocol Buffers are not designed to handle large >> messages." without any further explanation. So I wonder if this also >> applies for "bytes"-typed message fields. >> > Using a bytes field of 100MB is fine. That satement "Protocol Buffers are > not designed to handle large messages." is more or less obsolete now > because we see many people sending protos as large as 2GB and they are > supported. > > > >> >> >> On Thursday, January 11, 2018 at 1:20:26 PM UTC+1, Mario Emmenlauer wrote: >>> >>> >>> Hi, >>> >>> On 11.01.2018 13:13, [email protected] wrote: >>> > Hi, >>> > How do Protocol Buffers (C++) perform when messages contain big binary >>> blobs? With big I mean 1 ... 100MByte. >>> >>> My impression was that its not very suitable for that. Its a while >>> ago that I checked but there was no plain binary encoding back then. >>> To send binary files, I encoded them in some ASCII encoding (Base64 >>> I think) and sent them as strings. The encoding takes some CPU, and >>> the data size goes up. >>> >>> I'd be happy to learn otherwise?! And possibly things have changed in >>> the meantime... >>> >>> Cheers, >>> >>> Mario Emmenlauer >>> >>> >>> -- >>> BioDataAnalysis GmbH, Mario Emmenlauer Tel. Buero: +49-89-74677203 >>> Balanstr. 43 >>> <https://maps.google.com/?q=Balanstr.+43&entry=gmail&source=g> >>> mailto: memmenlauer * biodataanalysis.de >>> D-81669 München http://www.biodataanalysis.de/ >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Protocol Buffers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at https://groups.google.com/group/protobuf. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
