Re: [protobuf] Why doesn't protobuf C++ include smart pointers to submessages?

[Alex Shaver]

The std::weak_ptr does need a shared_ptr, but that doesn't necessarily 
prevent them from creating their own 'weak_reference' class or whatever 
that could check on whether what it pointed to was still valid or not.

The case I'm using it for is I have a top level message, with a repeated 
set of submessages inside of it. Each of those submessages gets an object 
that edits the submessage it points to. So I could possibly set it up so 
that I copy all those submessages to create these editor objects, then have 
some controller go back through the editor objects to copy their messages 
back into the main top level message. But now I also need to set up that 
controller to notice when some other object with access to the top level 
message deletes a submessage, and in return, delete the appropriate editor 
holding a copy of that submessage. 

On Monday, July 24, 2017 at 12:57:04 PM UTC-4, Arpit Baldeva wrote:
>
> A weak_ptr needs a shared_ptr to begin with so all the submessages would 
> end up being a shared_ptr (creating an impression that they are meant for 
> shared ownership). In fact, I feel that Protobuf could perhaps change the 
> generated code to use unique_ptr instead of it's raw pointers to more 
> clearly express the ownership semantics.
>
> For your use case, why can't you just copy the sub message you are 
> interested in? What use case would rule that out as an option?  
>
> On Friday, July 21, 2017 at 11:11:39 AM UTC-7, Alex Shaver wrote:
>>
>> I agree that since the model is that the top message owns its 
>> submessages, the idea of shared ownership isn't necessarily useful. But 
>> something like a weak pointer doesn't change the ownership, it would just 
>> allow objects to be responsive to whether the submessage exists or not. 
>>
>> While holding a reference to the top level message does technically 
>> resolve the problem, I feel like it breaks a bit of modularity; my object 
>> doesn't need to know about its sibling submessages, nor the broader 
>> message. Further, there's the issue of finding my submessage for a specific 
>> object. I could provide both a pointer to the larger object and to the 
>> smaller one, but that just gets us back to the initial problem. So I have 
>> to give it some algorithm to find which message I want. This then becomes 
>> fragile to schema changes to the protobuf (the object must be updated to 
>> some new path algorithm), but more importantly, still leaves the issue of 
>> when the submessage is deleted ambiguous. Let's say it used to search for 
>> the 3rd item in a repeated field, but the 3rd item is deleted somewhere 
>> else. It doesn't know the 3rd Item is gone, because it sees what used to be 
>> the 4th.
>>
>> It may just be that this is something protobuf can't do, and using it we 
>> just have to deal with that fact or do some additional codework so that 
>> every time a submessage is deleted all the things using it are notified and 
>> respond appropriately (with probably some decent unit testing to feel a bit 
>> more confident). But I feel like some kind of weak pointer, even if it was 
>> a class included in the protobuf library and not the std::weak_ptr class, 
>> would really help to resolve some of these issues in a consistent manner.
>>
>> On Friday, July 21, 2017 at 12:18:20 PM UTC-4, Adam Cozzette wrote:
>>>
>>> I think this would be a pretty difficult and invasive change to make, 
>>> because the basic memory model used by C++ protobuf is that each message 
>>> has sole ownership of its submessages, with no notion of shared ownership. 
>>> If we did introduce a change like this, it would also force existing users 
>>> to pay a runtime cost for this feature even though they don't use it. In 
>>> your case I would guess that the best solution would be to avoid hanging 
>>> onto a pointer to the submessage, but instead just hold a pointer or 
>>> reference to the parent message. That way you can always still access the 
>>> submessage or at least observe if it has been cleared, without risk of a 
>>> dangling pointer.
>>>
>>> On Wed, Jul 19, 2017 at 11:08 AM, Alex Shaver  
>>> wrote:
>>>
 If I have a
 message Foo{
Bar bar = 1;
 }   

 I can get a
 Bar* bar = foo.mutable_bar();   

 I'd like to be able to get, at the very least
 std::weak_ptr bar = foo.mutable_bar();   
 if not a std::shared_ptr. 

 I have an issue where an object has to hold a raw pointer to a 
 submessage, and it is possible to delete that submessage (using the 
 message's API elsewhere), in a way that the object isn't aware of, and it 
 attempts to use that stale pointer. This is one of the principle issues 
 addressed with use of smart pointers, and it's unclear why protobuf cannot 
 provide them. I don't understand the Arena allocation enough, if it is a 
 problem there, but it seems like that would at least provide some way for 
 weak_ptr to work. Or even if 

Re: [protobuf] Why doesn't protobuf C++ include smart pointers to submessages?

[Arpit Baldeva]

A weak_ptr needs a shared_ptr to begin with so all the submessages would 
end up being a shared_ptr (creating an impression that they are meant for 
shared ownership). In fact, I feel that Protobuf could perhaps change the 
generated code to use unique_ptr instead of it's raw pointers to more 
clearly express the ownership semantics.

For your use case, why can't you just copy the sub message you are 
interested in? What use case would rule that out as an option?  

On Friday, July 21, 2017 at 11:11:39 AM UTC-7, Alex Shaver wrote:
>
> I agree that since the model is that the top message owns its submessages, 
> the idea of shared ownership isn't necessarily useful. But something like a 
> weak pointer doesn't change the ownership, it would just allow objects to 
> be responsive to whether the submessage exists or not. 
>
> While holding a reference to the top level message does technically 
> resolve the problem, I feel like it breaks a bit of modularity; my object 
> doesn't need to know about its sibling submessages, nor the broader 
> message. Further, there's the issue of finding my submessage for a specific 
> object. I could provide both a pointer to the larger object and to the 
> smaller one, but that just gets us back to the initial problem. So I have 
> to give it some algorithm to find which message I want. This then becomes 
> fragile to schema changes to the protobuf (the object must be updated to 
> some new path algorithm), but more importantly, still leaves the issue of 
> when the submessage is deleted ambiguous. Let's say it used to search for 
> the 3rd item in a repeated field, but the 3rd item is deleted somewhere 
> else. It doesn't know the 3rd Item is gone, because it sees what used to be 
> the 4th.
>
> It may just be that this is something protobuf can't do, and using it we 
> just have to deal with that fact or do some additional codework so that 
> every time a submessage is deleted all the things using it are notified and 
> respond appropriately (with probably some decent unit testing to feel a bit 
> more confident). But I feel like some kind of weak pointer, even if it was 
> a class included in the protobuf library and not the std::weak_ptr class, 
> would really help to resolve some of these issues in a consistent manner.
>
> On Friday, July 21, 2017 at 12:18:20 PM UTC-4, Adam Cozzette wrote:
>>
>> I think this would be a pretty difficult and invasive change to make, 
>> because the basic memory model used by C++ protobuf is that each message 
>> has sole ownership of its submessages, with no notion of shared ownership. 
>> If we did introduce a change like this, it would also force existing users 
>> to pay a runtime cost for this feature even though they don't use it. In 
>> your case I would guess that the best solution would be to avoid hanging 
>> onto a pointer to the submessage, but instead just hold a pointer or 
>> reference to the parent message. That way you can always still access the 
>> submessage or at least observe if it has been cleared, without risk of a 
>> dangling pointer.
>>
>> On Wed, Jul 19, 2017 at 11:08 AM, Alex Shaver  wrote:
>>
>>> If I have a
>>> message Foo{
>>>Bar bar = 1;
>>> }   
>>>
>>> I can get a
>>> Bar* bar = foo.mutable_bar();   
>>>
>>> I'd like to be able to get, at the very least
>>> std::weak_ptr bar = foo.mutable_bar();   
>>> if not a std::shared_ptr. 
>>>
>>> I have an issue where an object has to hold a raw pointer to a 
>>> submessage, and it is possible to delete that submessage (using the 
>>> message's API elsewhere), in a way that the object isn't aware of, and it 
>>> attempts to use that stale pointer. This is one of the principle issues 
>>> addressed with use of smart pointers, and it's unclear why protobuf cannot 
>>> provide them. I don't understand the Arena allocation enough, if it is a 
>>> problem there, but it seems like that would at least provide some way for 
>>> weak_ptr to work. Or even if protobuf had its own custom template class 
>>> that behaved like a weak pointer so that I can at least be sure I don't 
>>> have stale pointers or references floating around in my code.
>>>
>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Protocol Buffers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to protobuf+u...@googlegroups.com.
>>> To post to this group, send email to prot...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/protobuf.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more 

Re: [protobuf] Why doesn't protobuf C++ include smart pointers to submessages?

[Alex Shaver]

I agree that since the model is that the top message owns its submessages, 
the idea of shared ownership isn't necessarily useful. But something like a 
weak pointer doesn't change the ownership, it would just allow objects to 
be responsive to whether the submessage exists or not. 

While holding a reference to the top level message does technically resolve 
the problem, I feel like it breaks a bit of modularity; my object doesn't 
need to know about its sibling submessages, nor the broader message. 
Further, there's the issue of finding my submessage for a specific object. 
I could provide both a pointer to the larger object and to the smaller one, 
but that just gets us back to the initial problem. So I have to give it 
some algorithm to find which message I want. This then becomes fragile to 
schema changes to the protobuf (the object must be updated to some new path 
algorithm), but more importantly, still leaves the issue of when the 
submessage is deleted ambiguous. Let's say it used to search for the 3rd 
item in a repeated field, but the 3rd item is deleted somewhere else. It 
doesn't know the 3rd Item is gone, because it sees what used to be the 4th.

It may just be that this is something protobuf can't do, and using it we 
just have to deal with that fact or do some additional codework so that 
every time a submessage is deleted all the things using it are notified and 
respond appropriately (with probably some decent unit testing to feel a bit 
more confident). But I feel like some kind of weak pointer, even if it was 
a class included in the protobuf library and not the std::weak_ptr class, 
would really help to resolve some of these issues in a consistent manner.

On Friday, July 21, 2017 at 12:18:20 PM UTC-4, Adam Cozzette wrote:
>
> I think this would be a pretty difficult and invasive change to make, 
> because the basic memory model used by C++ protobuf is that each message 
> has sole ownership of its submessages, with no notion of shared ownership. 
> If we did introduce a change like this, it would also force existing users 
> to pay a runtime cost for this feature even though they don't use it. In 
> your case I would guess that the best solution would be to avoid hanging 
> onto a pointer to the submessage, but instead just hold a pointer or 
> reference to the parent message. That way you can always still access the 
> submessage or at least observe if it has been cleared, without risk of a 
> dangling pointer.
>
> On Wed, Jul 19, 2017 at 11:08 AM, Alex Shaver  > wrote:
>
>> If I have a
>> message Foo{
>>Bar bar = 1;
>> }   
>>
>> I can get a
>> Bar* bar = foo.mutable_bar();   
>>
>> I'd like to be able to get, at the very least
>> std::weak_ptr bar = foo.mutable_bar();   
>> if not a std::shared_ptr. 
>>
>> I have an issue where an object has to hold a raw pointer to a 
>> submessage, and it is possible to delete that submessage (using the 
>> message's API elsewhere), in a way that the object isn't aware of, and it 
>> attempts to use that stale pointer. This is one of the principle issues 
>> addressed with use of smart pointers, and it's unclear why protobuf cannot 
>> provide them. I don't understand the Arena allocation enough, if it is a 
>> problem there, but it seems like that would at least provide some way for 
>> weak_ptr to work. Or even if protobuf had its own custom template class 
>> that behaved like a weak pointer so that I can at least be sure I don't 
>> have stale pointers or references floating around in my code.
>>
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to protobuf+u...@googlegroups.com .
>> To post to this group, send email to prot...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Why doesn't protobuf C++ include smart pointers to submessages?

['Adam Cozzette' via Protocol Buffers]

I think this would be a pretty difficult and invasive change to make,
because the basic memory model used by C++ protobuf is that each message
has sole ownership of its submessages, with no notion of shared ownership.
If we did introduce a change like this, it would also force existing users
to pay a runtime cost for this feature even though they don't use it. In
your case I would guess that the best solution would be to avoid hanging
onto a pointer to the submessage, but instead just hold a pointer or
reference to the parent message. That way you can always still access the
submessage or at least observe if it has been cleared, without risk of a
dangling pointer.

On Wed, Jul 19, 2017 at 11:08 AM, Alex Shaver  wrote:

> If I have a
> message Foo{
>Bar bar = 1;
> }
>
> I can get a
> Bar* bar = foo.mutable_bar();
>
> I'd like to be able to get, at the very least
> std::weak_ptr bar = foo.mutable_bar();
> if not a std::shared_ptr.
>
> I have an issue where an object has to hold a raw pointer to a submessage,
> and it is possible to delete that submessage (using the message's API
> elsewhere), in a way that the object isn't aware of, and it attempts to use
> that stale pointer. This is one of the principle issues addressed with use
> of smart pointers, and it's unclear why protobuf cannot provide them. I
> don't understand the Arena allocation enough, if it is a problem there, but
> it seems like that would at least provide some way for weak_ptr to work. Or
> even if protobuf had its own custom template class that behaved like a weak
> pointer so that I can at least be sure I don't have stale pointers or
> references floating around in my code.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.