Re: [protobuf] Re: Protocol Buffers and asynchronous sockets
Yes, use std::string. The only potential problem is if your messages are very large -- allocating large contiguous blocks of memory (as std::string does) could lead to memory fragmentation. But for small and medium-sized messages, there's no reason not to use std::string as the buffer. Parsing from an std::string (or a simple array -- they're essentially the same) is (slightly) faster than parsing from any other data structure. On Tue, Nov 24, 2009 at 9:07 AM, Evan Jones wrote: > Gilad Ben-Ami wrote: > > Do you think that using std::iostream in the following scenario would > > work / be a good choice? > > 1. read message_length > > 2. buffer message_length bytes into iostream variable. > > 3. when all data is received, use IstreamInputStream to wrap the > > iostream and have it parsed with ParseFromZeroCopyStream() > > If your application doesn't have a buffer already, I recommend using > std::string. AFAIK, the C++ standard library doesn't provide anything > more appropriate. It will do a good enough job, particularly if you > re-use one std::string rather than allocating a new one for each message. > > The reason to use something more complicated is because lots of > applications already have some sort of buffer, and you want to try and > avoid extra copies. > > Evan > > -- > Evan Jones > http://evanjones.ca/ > > -- > > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To post to this group, send email to proto...@googlegroups.com. > To unsubscribe from this group, send email to > protobuf+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/protobuf?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Re: Protocol Buffers and asynchronous sockets
Gilad Ben-Ami wrote: > Do you think that using std::iostream in the following scenario would > work / be a good choice? > 1. read message_length > 2. buffer message_length bytes into iostream variable. > 3. when all data is received, use IstreamInputStream to wrap the > iostream and have it parsed with ParseFromZeroCopyStream() If your application doesn't have a buffer already, I recommend using std::string. AFAIK, the C++ standard library doesn't provide anything more appropriate. It will do a good enough job, particularly if you re-use one std::string rather than allocating a new one for each message. The reason to use something more complicated is because lots of applications already have some sort of buffer, and you want to try and avoid extra copies. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Re: Protocol Buffers and asynchronous sockets
Gilad Ben-Ami wrote:
> So in this case, what is the best method to use PB?
> Should i use SerializeToArray and ParseFromArray instead of using the
> protobuf::io streams?
To use protocol buffers with an asynchronous library, you need to
collect the data for the message is some data structure until you know
it is all there. If performance is not critical the least effort
approach is:
1. Read the message_length from the stream in some way.
2. Create a std::string.
3. Read message_length bytes from the stream, appending them to the
std::string.
4. Use message.ParseFromString() to parse the message.
This can be bad for performance because the data may be copied many
times. If performance is really critical, you basically need to
efficiently collect the bytes into some "buffer data structure." I'm
assuming the ACE library probably provides something that does this?
Then, once you have at least message_length bytes, you parse it via a
ZeroCopyInputStream implementation.
For my asynchronous library, my implementation is approximately:
// assume we read the message_length from input somehow
if (input.availableBytes() < message_length) {
// get called back later
return IO_WAIT;
}
// MyInputWrapper implements google::protobuf::io::ZeroCopyInputStream
MyInputWrapper wrapper(&input, message_length);
MyProtocolBuffer message;
message.ParseFromZeroCopyStream(&wrapper);
I hope this helps,
Evan
--
Evan Jones
http://evanjones.ca/
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
