Re: Isn't the choice of std::string (C++ API) as a container for binary serialized bytes dangerous b/c of null termination?
Sure. Actually, someone made that change to the internal code just the other day, so it'll be in SVN soon. You can also do: message.mutable_foo()->assign(data, size); which is equivalent to: message.set_foo(string(data, size)) but avoids the temporary string. On Wed, Feb 25, 2009 at 4:53 PM, Dave Bailey wrote: > > Kenton, > > The generated code for a string field has setters for std::string and > const char *, but do you think you could add a (const char *, int > length) setter as well (like you have for bytes fields)? Right now in > protobuf-perlxs, we have to construct a temporary std::string in order > to set a string field's value from a Perl scalar, which it would be > nice to avoid (a Perl scalar may include embedded '\0' characters) - > see, for example: > > http://code.google.com/p/protobuf-perlxs/issues/detail?id=4&can=1 > > -dave > > On Feb 25, 3:27 pm, Kenton Varda wrote: > > Good to know, thanks. > > > > On Wed, Feb 25, 2009 at 2:41 PM, marc wrote: > > > > > On Feb 25, 2:41 pm, Kenton Varda wrote: > > > > You may be right, but we've done it that way for many years, and it > would > > > be > > > > too hard to change now. > > > > Are there any known STL implementations that use simple > null-terminated > > > > strings? string::size() would have to be O(n) for them, which would > be > > > > unfortunate. > > > > > Thanks Kenton, > > > > > Further digging reveals that std::string is intended to be able to > > > contain any char, including embedded nul chars, which would make a nul > > > terminated implementation pretty much moot. > > > So, I think it's safe to use them for binary data. > > > > > Below is a quote from the thread: > > >http://www.experts-exchange.com/Programming/Languages/CPP/Q_21022457.. > .. > > > > > 21.3 (String library) of the C++ standard clearly says: "For a char- > > > like type charT, the class basic_string describes objetcs that can > > > store a sequence consisting of a varing number of arbitrary char-like > > > objects ... Such a sequence is also called a "string" if the given > > > char-like type is clear from context. > > > Restrictions are only made in 21.4 (Null-terminated sequence > > > utilities) that deal with the cstring functions like strcat etc. > > > > > > On Wed, Feb 25, 2009 at 8:30 AM, marc wrote: > > > > > > > There's nothing in the standard that forbids the use of null as a > > > > > terminator in internal implementations of std::string. Therefore > it > > > > > seems dangerous to use string as a container to store binary data, > > > > > which certainly will contain null bytes. > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Isn't the choice of std::string (C++ API) as a container for binary serialized bytes dangerous b/c of null termination?
Kenton, The generated code for a string field has setters for std::string and const char *, but do you think you could add a (const char *, int length) setter as well (like you have for bytes fields)? Right now in protobuf-perlxs, we have to construct a temporary std::string in order to set a string field's value from a Perl scalar, which it would be nice to avoid (a Perl scalar may include embedded '\0' characters) - see, for example: http://code.google.com/p/protobuf-perlxs/issues/detail?id=4&can=1 -dave On Feb 25, 3:27 pm, Kenton Varda wrote: > Good to know, thanks. > > On Wed, Feb 25, 2009 at 2:41 PM, marc wrote: > > > On Feb 25, 2:41 pm, Kenton Varda wrote: > > > You may be right, but we've done it that way for many years, and it would > > be > > > too hard to change now. > > > Are there any known STL implementations that use simple null-terminated > > > strings? string::size() would have to be O(n) for them, which would be > > > unfortunate. > > > Thanks Kenton, > > > Further digging reveals that std::string is intended to be able to > > contain any char, including embedded nul chars, which would make a nul > > terminated implementation pretty much moot. > > So, I think it's safe to use them for binary data. > > > Below is a quote from the thread: > >http://www.experts-exchange.com/Programming/Languages/CPP/Q_21022457 > > > 21.3 (String library) of the C++ standard clearly says: "For a char- > > like type charT, the class basic_string describes objetcs that can > > store a sequence consisting of a varing number of arbitrary char-like > > objects ... Such a sequence is also called a "string" if the given > > char-like type is clear from context. > > Restrictions are only made in 21.4 (Null-terminated sequence > > utilities) that deal with the cstring functions like strcat etc. > > > > On Wed, Feb 25, 2009 at 8:30 AM, marc wrote: > > > > > There's nothing in the standard that forbids the use of null as a > > > > terminator in internal implementations of std::string. Therefore it > > > > seems dangerous to use string as a container to store binary data, > > > > which certainly will contain null bytes. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Isn't the choice of std::string (C++ API) as a container for binary serialized bytes dangerous b/c of null termination?
Good to know, thanks. On Wed, Feb 25, 2009 at 2:41 PM, marc wrote: > > > > On Feb 25, 2:41 pm, Kenton Varda wrote: > > You may be right, but we've done it that way for many years, and it would > be > > too hard to change now. > > Are there any known STL implementations that use simple null-terminated > > strings? string::size() would have to be O(n) for them, which would be > > unfortunate. > > Thanks Kenton, > > Further digging reveals that std::string is intended to be able to > contain any char, including embedded nul chars, which would make a nul > terminated implementation pretty much moot. > So, I think it's safe to use them for binary data. > > Below is a quote from the thread: > http://www.experts-exchange.com/Programming/Languages/CPP/Q_21022457.html > > 21.3 (String library) of the C++ standard clearly says: "For a char- > like type charT, the class basic_string describes objetcs that can > store a sequence consisting of a varing number of arbitrary char-like > objects ... Such a sequence is also called a "string" if the given > char-like type is clear from context. > Restrictions are only made in 21.4 (Null-terminated sequence > utilities) that deal with the cstring functions like strcat etc. > > > > > > > On Wed, Feb 25, 2009 at 8:30 AM, marc wrote: > > > > > There's nothing in the standard that forbids the use of null as a > > > terminator in internal implementations of std::string. Therefore it > > > seems dangerous to use string as a container to store binary data, > > > which certainly will contain null bytes. > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Isn't the choice of std::string (C++ API) as a container for binary serialized bytes dangerous b/c of null termination?
On Feb 25, 2:41 pm, Kenton Varda wrote: > You may be right, but we've done it that way for many years, and it would be > too hard to change now. > Are there any known STL implementations that use simple null-terminated > strings? string::size() would have to be O(n) for them, which would be > unfortunate. Thanks Kenton, Further digging reveals that std::string is intended to be able to contain any char, including embedded nul chars, which would make a nul terminated implementation pretty much moot. So, I think it's safe to use them for binary data. Below is a quote from the thread: http://www.experts-exchange.com/Programming/Languages/CPP/Q_21022457.html 21.3 (String library) of the C++ standard clearly says: "For a char- like type charT, the class basic_string describes objetcs that can store a sequence consisting of a varing number of arbitrary char-like objects ... Such a sequence is also called a "string" if the given char-like type is clear from context. Restrictions are only made in 21.4 (Null-terminated sequence utilities) that deal with the cstring functions like strcat etc. > > On Wed, Feb 25, 2009 at 8:30 AM, marc wrote: > > > There's nothing in the standard that forbids the use of null as a > > terminator in internal implementations of std::string. Therefore it > > seems dangerous to use string as a container to store binary data, > > which certainly will contain null bytes. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Isn't the choice of std::string (C++ API) as a container for binary serialized bytes dangerous b/c of null termination?
You may be right, but we've done it that way for many years, and it would be too hard to change now. Are there any known STL implementations that use simple null-terminated strings? string::size() would have to be O(n) for them, which would be unfortunate. On Wed, Feb 25, 2009 at 8:30 AM, marc wrote: > > There's nothing in the standard that forbids the use of null as a > terminator in internal implementations of std::string. Therefore it > seems dangerous to use string as a container to store binary data, > which certainly will contain null bytes. > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
