On Feb 8, 2011, at 13:34 , Kenton Varda wrote:
I handle user messages by passing them as "bytes", embedded in my
own outer message.
This is what I do as well, as does protobuf-socket-rpc:
http://code.google.com/p/protobuf-socket-rpc/source/browse/trunk/proto/rpc.proto
I guess I was thinking that if you already have to do some sort of
"lookup" of the message type that is stored in that byte blob, then
maybe you don't need the streaming extension. For example, you could
just build a library that produces a sequence of byte strings, which
the "user" of the library can then parse appropriately.
I see how you are using it though: it is a friendly wrapper around
this simple "sequence of byte strings" model, that automatically
parses that byte string using the tag and "schema message." This might
be useful for some people.
This is somewhat inefficient currently, as it will require an extra
copy of all those bytes. However, it seems likely that future
improvements to protocol buffers will allow "bytes" fields to share
memory with the original buffer, which will eliminate this concern.
Ah cool. I was considering changing my protocol to be two messages:
the first one is the "descriptor" (eg. your CallRequest message), then
the second would be the "body" of the request, which I would then
parse based on the type passed in the CallRequest.
Note that I expect people will generally only "stream" their top-
level message. Although the proposal allows for streaming sub-
messages as well, I expect that people will normally want to parse
them into message objects which are handled whole. So, you only
have to manually implement the top-level stream, and then you can
invoke some reflective algorithm from there.
Right, but my concern is that I might want to use this streaming API
to write messages into files. In this case, I might have a file
containing the FooStream and another file containing the BarStream.
I'll have to implement both these ::Writer interfaces, or hack the
code generator to generate it for me. Although now that I think about
this, the implementation of these two APIs will be relatively trivial...
features like being able to detect broken streams and "resume" in
the middle are useful.
I'm not sure how this relates. This seems like it should be handled
at a lower layer, like in the InputStream -- if the connection is
lost, it can re-establish and resume, without the parser ever
knowing what happened.
Sorry, just an example of why you might want a different protocol. If
I've streamed 10e9 messages to disk, I don't want this stream to break
if there is some weird corruption in the middle, so I want some
protocol that can "resume" from corruption.
Evan
--
http://evanjones.ca/
--
You received this message because you are subscribed to the Google Groups "Protocol
Buffers" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
