Re: [protobuf] ByteString using N bytes from an InputStream?
On Mon, Aug 5, 2013 at 9:31 PM, V.B. vidalborro...@gmail.com wrote: ... Actually, I just now took a closer look at the readChunk() method. Even that method makes an internal copy, so it looks like readChunk() isn't what we are looking for after all. Hmmm. It seems to me that readChunk() has done a redundant copy which can be eliminated (and should be). I don't understand why the wrapper has to contain an extra copy though. Isn't it just copying the data to the destination directly? On Tuesday, August 6, 2013 12:28:56 AM UTC-4, V.B. wrote: Hi Feng Xiao! Thanks for the response. That's actually our backup plan. We were hoping to avoid it, though, since the wrappers would each contain an extra copy of the data internally. Our ideal case is for the data to get copied in a single step directly from an InputStream to a ByteString with no intermediate copies along the way. Question: You would know best... Would the safety of ByteStrings be preserved if the readChunk() method were to be made public? If so, I'll open a feature request on the issue tracker. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out.
Re: [protobuf] ByteString using N bytes from an InputStream?
On Tue, Aug 6, 2013 at 7:18 PM, Feng Xiao xiaof...@google.com wrote: On Mon, Aug 5, 2013 at 9:31 PM, V.B. vidalborro...@gmail.com wrote: ... Actually, I just now took a closer look at the readChunk() method. Even that method makes an internal copy, so it looks like readChunk() isn't what we are looking for after all. Hmmm. It seems to me that readChunk() has done a redundant copy which can be eliminated (and should be). Does ByteString need to worry about hostile InputStreams? The stream could retain a reference to the bytearray it was given in the read() call. Oliver -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out.
Re: [protobuf] ByteString using N bytes from an InputStream?
On Tue, Aug 6, 2013 at 12:26 PM, Oliver Jowett oliver.jow...@gmail.comwrote: On Tue, Aug 6, 2013 at 7:18 PM, Feng Xiao xiaof...@google.com wrote: On Mon, Aug 5, 2013 at 9:31 PM, V.B. vidalborro...@gmail.com wrote: ... Actually, I just now took a closer look at the readChunk() method. Even that method makes an internal copy, so it looks like readChunk() isn't what we are looking for after all. Hmmm. It seems to me that readChunk() has done a redundant copy which can be eliminated (and should be). Does ByteString need to worry about hostile InputStreams? The stream could retain a reference to the bytearray it was given in the read() call. Hmm, I haven't thought of that. Seems it should be the reason why it does that copy. Given that, there isn't any way to construct a ByteString with just one copy. Oliver -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out.
[protobuf] ByteString using N bytes from an InputStream?
Greetings all, We are using version 2.5. What is the most efficient way (*i.e.* single copy operation, no extra byte arrays) to construct a ByteString from a specific number of bytes in an InputStream? The various versions of ByteString.readFrom() drain the stream completely, which is not what we need; any data past *N* bytes should remain in the stream. The ByteString.readChunk() method looks like it will work if we simply give it * N* as the chunkSize parameter. Unfortunately, ByteString.readChunk() is declared private, so that method is not currently an option. Is there another option that I just haven't found in the source code yet? (Thanks for taking the time to read this question.) -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out.
Re: [protobuf] ByteString using N bytes from an InputStream?
On Mon, Aug 5, 2013 at 3:23 PM, V.B. vidalborro...@gmail.com wrote: Greetings all, We are using version 2.5. What is the most efficient way (*i.e.*single copy operation, no extra byte arrays) to construct a ByteString from a specific number of bytes in an InputStream? The various versions of ByteString.readFrom() drain the stream completely, which is not what we need; any data past *N* bytes should remain in the stream. The ByteString.readChunk() method looks like it will work if we simply give it *N* as the chunkSize parameter. Unfortunately, ByteString.readChunk()is declared private, so that method is not currently an option. Is there another option that I just haven't found in the source code yet? How about create an wrapper InputStream that only reads N bytes from the original InputStream and provide the wrapper to BytesString.readFrom()? (Thanks for taking the time to read this question.) -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out.
Re: [protobuf] ByteString using N bytes from an InputStream?
Hi Feng Xiao! Thanks for the response. That's actually our backup plan. We were hoping to avoid it, though, since the wrappers would each contain an extra copy of the data internally. Our ideal case is for the data to get copied in a single step directly from an InputStream to a ByteString with no intermediate copies along the way. Question: You would know best... Would the safety of ByteStrings be preserved if the readChunk() method were to be made public? If so, I'll open a feature request on the issue tracker. On Monday, August 5, 2013 7:32:49 PM UTC-4, Feng Xiao wrote: How about create an wrapper InputStream that only reads N bytes from the original InputStream and provide the wrapper to BytesString.readFrom()? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out.
Re: [protobuf] ByteString using N bytes from an InputStream?
... Actually, I just now took a closer look at the readChunk() method. Even that method makes an internal copy, so it looks like readChunk() isn't what we are looking for after all. Hmmm. On Tuesday, August 6, 2013 12:28:56 AM UTC-4, V.B. wrote: Hi Feng Xiao! Thanks for the response. That's actually our backup plan. We were hoping to avoid it, though, since the wrappers would each contain an extra copy of the data internally. Our ideal case is for the data to get copied in a single step directly from an InputStream to a ByteString with no intermediate copies along the way. Question: You would know best... Would the safety of ByteStrings be preserved if the readChunk() method were to be made public? If so, I'll open a feature request on the issue tracker. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/groups/opt_out.
