We have no plans set to support arena-based strings yet. This would be blocked on finishing the migration to string_view accessors, but *that* is planned for 2022.
On Tuesday, January 18, 2022 at 7:12:16 PM UTC-8 [email protected] wrote: > Hi, I have a question. When arena-based strings are expected to be > released ？ > On Wednesday, September 9, 2020 at 6:17:26 AM UTC+8 [email protected] > wrote: > >> Our StringPiece type has been made obsolete by the std::string_view type >> introduced in C++17, so we will eventually get rid of StringPiece and >> replace it with std::string_view (or absl::string_view, which has the same >> API but is available in C++11). So if you are making a local modification >> to support allocating strings on arenas, it would probably be best to go >> straight to std::string_view and avoid our StringPiece type. To handle both >> the arena and non-arena case, the simplest solution would be to store both >> a std::string_view and a std::string. When arenas are used, you can have >> the string_view point into arena-allocated memory, and when arenas are not >> used, it can just point to the std::string's data. >> >> On Thu, Sep 3, 2020 at 3:19 AM X Ah <[email protected]> wrote: >> >>> Hi Feng, >>> I have an API design problem, Since StringPiece doesn't own the data and >>> it's impossible to get memory from it, So how should the behavior be if I >>> do ParseFromString and the message is not in arena? I think the StringPiece >>> field should be empty if current message doesn't own an arena, but it is >>> strange for user. Could you introduce how Google internal use StringPiece >>> in protobuf? >>> Thanks! >>> On Saturday, January 16, 2016 at 3:32:39 AM UTC+8 [email protected] >>> wrote: >>> >>>> On Thu, Jan 14, 2016 at 6:06 PM, Austin Schuh <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I've got an application where I can't allocate memory while using >>>>> protobufs. Arenas have been awesome for doing that. I'm able to >>>>> allocate >>>>> a big block of memory at startup time or stack allocate memory for the >>>>> arena, and then use that for allocating protobufs. Thanks! >>>>> >>>>> I'd like to be able to allocate strings in the arena. I'm willing to >>>>> do the implementation, and wouldn't mind up-streaming if my >>>>> implementation >>>>> is complete enough and there is interest. It looks like I should start >>>>> by >>>>> implementing ctype=STRING_PIECE and then allocate memory in the arena to >>>>> back it. The class in //src/google/protobuf:arenastring.h looks like the >>>>> place to do all the operations. It looks like I need to modify the >>>>> interface to provide setters and getters to support STRING_PIECE there. >>>>> >>>>> Is that the right place to start? Is there any more guidance that you >>>>> can give me? >>>>> >>>> Hi Austin, >>>> >>>> Thanks for contacting us and offering help! >>>> >>>> You are looking at the right direction. We actually already opensourced >>>> the StringPiece implementation not very long ago: >>>> >>>> https://github.com/google/protobuf/blob/master/src/google/protobuf/stubs/stringpiece.h >>>> >>>> It's intended to be used to implement "ctype = STRING_PIECE" for string >>>> fields and since it's merely a <const char*, size_t> pair, it can be >>>> directed at the buffer in the arena. Such features are implemented inside >>>> Google but unfortunately it's not opensourced due to dependency issues. We >>>> plan to get them out eventually but hasn't have enough time to work on it. >>>> Since we already have an internal version of it, we probably won't be able >>>> to accept your contributions. I can't give a concrete timeline about when >>>> we will get our implementation opensourced also. Sorry for that... >>>> >>>> If you need this soon, I suggest you try to implement it as simple as >>>> possible. Better to only support lite runtime with arena enabled. Some >>>> changes you want to make: >>>> 1. Make ArenaStringPtr work with StringPiece, or introduce an >>>> ArenaStringPiecePtr which might be easier to implement. >>>> 2. Update protocol compiler to use ArenaStringPtr/ArenaStringPiecePtr >>>> to store ctype=STRING_PIECE fields and expose a StringPiece API: >>>> // proto >>>> message Foo { >>>> string bar = 1 [ctype = STRING_PIECE]; >>>> } >>>> // generated C++ code >>>> message Foo { >>>> public: >>>> StringPiece bar() const; >>>> void set_bar(StringPiece value); // Note that we need to do a deep >>>> copy here because StringPiece doesn't own the underlying data. >>>> void set_alias_bar(StringPiece value); // Make the field point to >>>> the StringPiece data directly. Caller must make sure the underlying data >>>> outlives the Foo message. >>>> >>>> private: >>>> ArenaStringPiecePtr bar_; >>>> }; >>>> >>>> Look at the string_field.cc implementation in the compiler directory >>>> <https://github.com/google/protobuf/blob/master/src/google/protobuf/compiler/cpp/cpp_string_field.cc> >>>> >>>> and you can create a string_piece_field.cc implementation based on that. >>>> Most of the work will be done here, including not only the generated API >>>> but also all the parsing/serialization/copy/constructor/destructor support. >>>> >>>> That's pretty all that needed to support StringPiece in lite-runtime + >>>> arena. A lot more work will be needed to support other combinations >>>> (lite-runtime + no arena, full-runtime + arena, full-runtime + non-arena), >>>> but since you have a specific targeted platform and we will opensource the >>>> StringPiece support eventually, it's probably not worthwhile to invest >>>> time >>>> to support anything you don't actually need right now. >>>> >>>> Hope this helps. >>>> >>>> Regards, >>>> Feng >>>> >>>> >>>>> Thanks, >>>>> Austin >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Protocol Buffers" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/protobuf. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Protocol Buffers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/protobuf/ccc70ecc-873a-4297-8102-8e2319ffd760n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/protobuf/ccc70ecc-873a-4297-8102-8e2319ffd760n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/5abb9dcf-0dff-4676-823b-ae8269d5a752n%40googlegroups.com.
