Hi, I have a question. When arena-based strings are expected to be released ？ On Wednesday, September 9, 2020 at 6:17:26 AM UTC+8 [email protected] wrote:
> Our StringPiece type has been made obsolete by the std::string_view type > introduced in C++17, so we will eventually get rid of StringPiece and > replace it with std::string_view (or absl::string_view, which has the same > API but is available in C++11). So if you are making a local modification > to support allocating strings on arenas, it would probably be best to go > straight to std::string_view and avoid our StringPiece type. To handle both > the arena and non-arena case, the simplest solution would be to store both > a std::string_view and a std::string. When arenas are used, you can have > the string_view point into arena-allocated memory, and when arenas are not > used, it can just point to the std::string's data. > > On Thu, Sep 3, 2020 at 3:19 AM X Ah <[email protected]> wrote: > >> Hi Feng, >> I have an API design problem, Since StringPiece doesn't own the data and >> it's impossible to get memory from it, So how should the behavior be if I >> do ParseFromString and the message is not in arena? I think the StringPiece >> field should be empty if current message doesn't own an arena, but it is >> strange for user. Could you introduce how Google internal use StringPiece >> in protobuf? >> Thanks! >> On Saturday, January 16, 2016 at 3:32:39 AM UTC+8 [email protected] >> wrote: >> >>> On Thu, Jan 14, 2016 at 6:06 PM, Austin Schuh <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> I've got an application where I can't allocate memory while using >>>> protobufs. Arenas have been awesome for doing that. I'm able to allocate >>>> a big block of memory at startup time or stack allocate memory for the >>>> arena, and then use that for allocating protobufs. Thanks! >>>> >>>> I'd like to be able to allocate strings in the arena. I'm willing to >>>> do the implementation, and wouldn't mind up-streaming if my implementation >>>> is complete enough and there is interest. It looks like I should start by >>>> implementing ctype=STRING_PIECE and then allocate memory in the arena to >>>> back it. The class in //src/google/protobuf:arenastring.h looks like the >>>> place to do all the operations. It looks like I need to modify the >>>> interface to provide setters and getters to support STRING_PIECE there. >>>> >>>> Is that the right place to start? Is there any more guidance that you >>>> can give me? >>>> >>> Hi Austin, >>> >>> Thanks for contacting us and offering help! >>> >>> You are looking at the right direction. We actually already opensourced >>> the StringPiece implementation not very long ago: >>> >>> https://github.com/google/protobuf/blob/master/src/google/protobuf/stubs/stringpiece.h >>> >>> It's intended to be used to implement "ctype = STRING_PIECE" for string >>> fields and since it's merely a <const char*, size_t> pair, it can be >>> directed at the buffer in the arena. Such features are implemented inside >>> Google but unfortunately it's not opensourced due to dependency issues. We >>> plan to get them out eventually but hasn't have enough time to work on it. >>> Since we already have an internal version of it, we probably won't be able >>> to accept your contributions. I can't give a concrete timeline about when >>> we will get our implementation opensourced also. Sorry for that... >>> >>> If you need this soon, I suggest you try to implement it as simple as >>> possible. Better to only support lite runtime with arena enabled. Some >>> changes you want to make: >>> 1. Make ArenaStringPtr work with StringPiece, or introduce an >>> ArenaStringPiecePtr which might be easier to implement. >>> 2. Update protocol compiler to use ArenaStringPtr/ArenaStringPiecePtr to >>> store ctype=STRING_PIECE fields and expose a StringPiece API: >>> // proto >>> message Foo { >>> string bar = 1 [ctype = STRING_PIECE]; >>> } >>> // generated C++ code >>> message Foo { >>> public: >>> StringPiece bar() const; >>> void set_bar(StringPiece value); // Note that we need to do a deep >>> copy here because StringPiece doesn't own the underlying data. >>> void set_alias_bar(StringPiece value); // Make the field point to the >>> StringPiece data directly. Caller must make sure the underlying data >>> outlives the Foo message. >>> >>> private: >>> ArenaStringPiecePtr bar_; >>> }; >>> >>> Look at the string_field.cc implementation in the compiler directory >>> <https://github.com/google/protobuf/blob/master/src/google/protobuf/compiler/cpp/cpp_string_field.cc> >>> >>> and you can create a string_piece_field.cc implementation based on that. >>> Most of the work will be done here, including not only the generated API >>> but also all the parsing/serialization/copy/constructor/destructor support. >>> >>> That's pretty all that needed to support StringPiece in lite-runtime + >>> arena. A lot more work will be needed to support other combinations >>> (lite-runtime + no arena, full-runtime + arena, full-runtime + non-arena), >>> but since you have a specific targeted platform and we will opensource the >>> StringPiece support eventually, it's probably not worthwhile to invest time >>> to support anything you don't actually need right now. >>> >>> Hope this helps. >>> >>> Regards, >>> Feng >>> >>> >>>> Thanks, >>>> Austin >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Protocol Buffers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at https://groups.google.com/group/protobuf. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Protocol Buffers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/protobuf/ccc70ecc-873a-4297-8102-8e2319ffd760n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/protobuf/ccc70ecc-873a-4297-8102-8e2319ffd760n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/e5e75dc8-ecaf-4ad4-8cad-8276559fee70n%40googlegroups.com.
