On Mon, Jan 10, 2011 at 9:45 PM, Nicolae Mihalache <xproma...@gmail.com>wrote:

> Hello,
> I recently started to use GPB, great software! :)
> But I have noticed in java that it is impossible to create a message
> containing a "bytes" fields without copying some buffers around. For
> example if I have a encoded message of 1MB with a few regular fields
> and one big bytes field, decoding the message will make a copy of the
> entire buffer instead of keeping a reference to it.

We are actually looking at fixing this by allowing ByteStrings to share

> Even worse when encoding: if I read some data from file, does not seem
> possible to put it directly into a ByteString so I have to make first
> a byte[], then copy it into the ByteString and when encoding, it makes
> yet another byte[].

ByteString provides multiple methods of construction.  One is to copy from a
byte array.  Another is to use an OutputStream that writes into a
ByteString.  In future versions, we are looking at making it possible to
concatenate ByteStrings without a copy.

But yes, if you start with a byte[], and you want a ByteString with the same
content, you are going to need to make a copy, because ByteString has to
guarantee immutability.

> So my question: is it possible to make an exception from the
> immutability for the "bytes" fields and use java.nio.ByteBuffers
> instead of ByteStrings?

No, sorry, making any exception to immutability would end up unraveling the
whole library.  You can go from ByteString to ByteBuffer without a copy (by
calling asReadOnlyByteBuffer()), but you can't go the other way, because
there is no way to know given a ByteBuffer pointer whether or not someone
might be able to modify it in the future.

Storing ByteBuffer in message objects directly has additional problems.
 ByteBuffer is a stateful class -- it maintains a pointer to the current
read location, for example.  So a protocol message object with ByteBuffers
inside it would be thread-hostile no matter how you look at it.  This just
leads to too many problems...

You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
For more options, visit this group at 

Reply via email to