On Sep 8, 6:40 pm, Chris Kuklewicz <[EMAIL PROTECTED]> wrote: > So we both agree there should be a way to delimit messages in a > stream. In your stream is sounds like there is > > (A) The field type + wire type, which here is 1*8+2 = 10 written as a > varint the stream in a single byte since it is <= 127. > (B) The length of the message as a varint > (C) The message body > > And in you C# code reading this one message at a time is support by a > special API command.
Correct. > So this is very close to my proposal (which is in my Haskell code). I > do not have (A) but I do have (B) and (C). > > Does you C# API allow (in some way) the user to read (B)+(C) as a > single message from the stream? Does it allow the user to write such > a thing? You could do it using CodedInputStream/CodedOutputStream directly, but it wouldn't be terribly pleasant. > In my Haskell code could make a wrapper message and an extension key > to allow it to write (A)+(B)+(C). And this could read all the > messages in the stream. But I lack a special API command to read a > single such message from the stream. The (A)+(B)+(C)+(A)+(B)+(C)+... > is wire compatible with such a repeated message but your reading API > is special. Since a special API is needed I chose (B)+(C)+(B)+(C)+(B)+ > (C)+... and if the user wants them all then the user has to loop over > this collecting a single message at a time. > > Where is your C# API documented, in case I wish to support your (A)+(B) > +(C) single message reading behavior? I haven't got as far as docs yet, but the discussion around this is in the earlier "streaming messages" thread. > As for mixing message types in a stream, I think the ability to have a > heterogeneous stream (or a stream embedded in another protocol) will > be needed by someone. So the main question, like with delimited > messages, is whether each application invents its own technique. One > could use a serialized DescriptorProto Message to announce the type > before each message. All the fields are optional, so the herald > message needs only to set the name field. Or perhaps serialize a > (nearly empty) FileDescriptorProto with the (nearly empty) embedded > DescriptorProto would make it more clear which object is being > serialized. But this is getting me off track. I think it's best to concentrate on the simple requirement first, and not guess too much about what would be needed. Use cases for an homogenous stream are easy to come up with - the simplest being logging, for example. > The main point of the message is to see if the C# and Haskell code can > agree on common APIs and encodings for delimited messages in a > continuous stream. The benefit of the A+B+C format is that it's not really coming up with a new encoding, so much as just a new way of reading repeated "field 1" messages. The whole data still ends up being a valid message of the appropriate type. But yes, I agree that a common API would be good to decide. > Here is a generalization question: If we adopt a way to read (A)+(B)+ > (C) then will the generalize beyond message types to groups or to > basic types like Double ? What happens in the special API in your C# > code if one tries to do this or encounters such a stream? My Haskell > API only works with messages. I have not exposed a friendly way to > encode or decode anything else. If you want to encode a stream of doubles directly, I'm not sure that PBs are the right way to go. However, with CodedInputStream and CodedOutputStream I guess it would be feasible. Jon --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to firstname.lastname@example.org To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~----------~----~----~----~------~----~------~--~---