What does pak.ByteSize() return for those two messages? It would also be good to verify that SerializeToArray() is returning true, because if not then there could be an error being missed.
Something seems to be going wrong with the message sizes, because the first data example seems to have extra data on the end while the second one seems to be too short (truncated). By the way, Marc, I tried out your protogen tool just now and it's pretty cool! That's more convenient than having to fiddle around with creating the binary data file and running protoc --decode_raw on it. On Wed, Apr 11, 2018 at 3:25 PM Tony Tony <xxteknolus...@gmail.com> wrote: > Well I just copied whatever text was printed after i logged what was being > serialized. So I probably copied extra bytes not sent from first one (since > first one deserializes fine). It seems like I'm using correct api call to > do serialization (and it works with other data combos/types) > > The second one, I agree is not right. Just gotta figure out why :| > > > > On Tuesday, April 10, 2018 at 11:36:29 PM UTC-4, Marc Gravell wrote: >> >> Oddly neither of those is correct! The actual hex should be: >> >> 08-03-1A-06-08-01-10-63-18-01 >> >> which is the first version minus some trailing data. The second version >> is not valid at all - it isn't just missing data: it is an incomplete and >> invalid payload. Unfortunately I'm not familiar enough with the C++ API to >> advise on whether you're using the wrong method vs this being a bug. >> >> >> On Wed, 11 Apr 2018, 01:12 Tony Tony, <xxtekn...@gmail.com> wrote: >> >>> So I upgraded to 3.5.0, same issue. >>> >>> I guess I should give some more information: I'm basically using >>> protobuf to serialize/deserialize string data I'm sending through a games >>> socket implementation via hooks/code caves in the games network engine. I >>> currently can validate that the bytes after serialization are sent over >>> completely to the client. After looking at the decoded output that you guys >>> recommended, I do see some differences. >>> >>> - When I send type=1, count=99, index=1, the server shows the below >>> bytes after serialization and the client shows these bytes after receiving >>> the data. The decoding tool shows the correct content. >>> >>> 08 03 1A 06 08 01 10 63 18 01 FD FD FD FD >>> >>> - When I send type=1, count=0, index=1, the server shows the below bytes >>> after serialization and the client shows these bytes after receiving the >>> data. The decoding tool basically shows no content, which is obvious from >>> the missing data. >>> >>> 08 03 1A 06 08 >>> >>> The biggest difference is that it seems the during serialization on the >>> server, this combo of data seems to break everything. I guess I'm not sure >>> why this combo of data would break during serialization. I posted some >>> pseudo code of where the problem occurs for that combo of data below. It's >>> just strange that only that combo of data above results in this behavior. I >>> am able to send large strings just fine, plenty of other data types with no >>> issues (well none that I've found yet..) >>> >>> Packet pak; >>> pak.set_type(Packet_Type_set_info); >>> >>> set_info *setter = pak.mutable_set_info(); >>> setter->set_count(99); >>> setter->set_type(0); >>> setter->set_index(1); >>> >>> char* SendBuf = new char[pak.ByteSize()]; >>> pak.SerializeToArray(SendBuf, pak.ByteSize()); >>> //TODO: the bytes seem to be wrong right after this point for the second >>> combo of data >>> //we send before we delete the sendBuf >>> >>> delete[] SendBuf; >>> >>> >>> On Tuesday, April 10, 2018 at 12:41:30 PM UTC-4, Adam Cozzette wrote: >>>> >>>> I think Marc is right that it would be best to isolate the problem by >>>> manually checking that the bytes are valid. >>>> >>>> Tony, how are you framing the messages? Protocol buffers do not >>>> describe their own size and so you have to know what size to expect before >>>> you parse a proto. The typical approach is to frame the serialized messages >>>> by prefixing them with their size. It's easy to get this wrong, though, so >>>> that could potentially be the reason that the messages aren't showing up as >>>> you expect. >>>> >>>> On Tue, Apr 10, 2018 at 7:28 AM Marc Gravell <marc.g...@gmail.com> >>>> wrote: >>>> >>>>> Well, if it was me: the first thing I'd do is isolate whether it is >>>>> serialize or deserialize that is failing - by taking your currently >>>>> serialized data as a flat file or byte dump, and checking whether the >>>>> values are right or wrong. >>>>> >>>>> Depending on the volume and layout of the data, you might be able to >>>>> use protoc's inbuilt decode mechanisms, or (as long as it isn't huge) >>>>> you're welcome to try using https://protogen.marcgravell.com/decode >>>>> >>>>> That would allow you to focus your checks. >>>>> >>>>> Marc >>>>> >>>>> On 10 April 2018 at 14:20, Tony Tony <xxtekn...@gmail.com> wrote: >>>>> >>>>>> c++/windows >>>>>> >>>>>> I'll upgrade protobuf and see if issue goes away. Just wanted to make >>>>>> sure I'm not missing anything. Is there any additional troubleshooting I >>>>>> can look into to troubleshoot further if the recent release reproduce the >>>>>> issue? >>>>>> >>>>>> On Tuesday, April 10, 2018 at 1:46:50 AM UTC-4, Marc Gravell wrote: >>>>>>> >>>>>>> First thought: what language / platform is this? the C++ generated >>>>>>> code is very different to the Java generated code, for example >>>>>>> >>>>>>> Second thought: 2.6 is pretty old; it is very possible that a bug >>>>>>> existed and has been fixed since then (Aug 2014) - does it still happen >>>>>>> with more recent releases? >>>>>>> >>>>>>> The scenario you present seems fine and reasonable. >>>>>>> >>>>>>> Marc >>>>>>> >>>>>>> On 10 April 2018 at 03:46, Tony Tony <xxtekn...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have a strange case I'm dealing with. Using protobuf compiler 2.6. >>>>>>>> >>>>>>>> Every time I send a certain set of values over my socket, it ends >>>>>>>> up with missing data. Although I've compared the string received after >>>>>>>> serialization on the server and on receiving the string on the client >>>>>>>> and >>>>>>>> they are exactly the same, yet the values are all -1. >>>>>>>> >>>>>>>> I've attached a snippet of the proto file below. >>>>>>>> >>>>>>>> At this point these are below combos that always fail >>>>>>>> >>>>>>>> type=99, count= 0, index = 1 (all values are -1 after >>>>>>>> deserialization) >>>>>>>> type=99, count=99, index= 1 (works) >>>>>>>> type=99999, count=999999, index= 1 (works) >>>>>>>> >>>>>>>> - Basically anytime count is 0, all the values seem to be -1. >>>>>>>> However if I make type 0, the same behavior does not occur. Am I >>>>>>>> missing >>>>>>>> something or there is some underlying bug in my code? Again I've >>>>>>>> checked >>>>>>>> multiple times the correct raw string data is being sent (with correct >>>>>>>> size). I use the parseFromString method to parse the receiving string >>>>>>>> data. >>>>>>>> >>>>>>>> - I've sent plenty of other types of data just fine. >>>>>>>> >>>>>>>> - There are other messages defined in the same protobuf packet >>>>>>>> definition file, can that make a difference? >>>>>>>> >>>>>>>> message packet { >>>>>>>> required int32 type = 1 [default = -1]; >>>>>>>> required int32 count = 2 [default = -1]; >>>>>>>> required int32 index = 3 [default = -1]; >>>>>>>> } >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Protocol Buffers" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to protobuf+u...@googlegroups.com. >>>>>>>> To post to this group, send email to prot...@googlegroups.com. >>>>>>>> Visit this group at https://groups.google.com/group/protobuf. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> >>>>>>> Marc >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Protocol Buffers" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to protobuf+u...@googlegroups.com. >>>>>> To post to this group, send email to prot...@googlegroups.com. >>>>>> Visit this group at https://groups.google.com/group/protobuf. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> >>>>> Marc >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Protocol Buffers" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to protobuf+u...@googlegroups.com. >>>>> To post to this group, send email to prot...@googlegroups.com. >>>>> Visit this group at https://groups.google.com/group/protobuf. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Protocol Buffers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to protobuf+u...@googlegroups.com. >>> To post to this group, send email to prot...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/protobuf. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to protobuf+unsubscr...@googlegroups.com. > To post to this group, send email to protobuf@googlegroups.com. > Visit this group at https://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at https://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
