On Thu, Jul 16, 2015 at 7:46 AM, Gordon Sim <g...@redhat.com> wrote:
> On 07/16/2015 02:40 PM, aconway wrote:
>
>> Can someone who understand the proton use of refcounts please add some
>> doc comments to explain the semantics? Apologies if this is already
>> there and I missed it, tell me to RTFM.
>>
>
> I'm not entirely sure I understand it. However having spent a couple of
> days reading and puzzling over the code, I'll try and offer some answers
> where I think I can, and add some questions of my own.
>
> For comparison here is what "refcount" traditionally means ("tradition"
>> includes CORBA, boost::intrusive_ptr, std::shared_ptr etc.) I'm not
>> saying we have to make proton conform, but we should document how it
>> differs to save confusion.
>>
>> 1. A refcount is a count of the number of references (owning pointers)
>> to an object.
>>
>
> Yes, that broadly speaking is the purpose in proton. However this was a
> recent addition and it is not used uniformly inside the library itself. Not
> only that but the old api, where the application (generally) owned the
> pointers it created until it called pn_xxx_free, is still supported
> alongside the newer mode of use, as e.g. employed in the python binding,
> where the application uses only incref and decref.
>
> 2. Objects are created with refcuont=1, the creator owns the first
>> reference.
>>
>
> This is not always the case and was one of the points I found surprising.
> Though a newly created 'object' does indeed have its ref count set to 1,
> for pn_session() and pn_link(), which are the functions actually used by
> the application to created new sessions or links, the implementation of
> those functions includes an explicit decref, meaning the objects are
> returned with a ref count of 0.
>
Are they really returned with a ref count of 0? I don't think proton
objects can actually exist with a refcount of 0 outside a finalizer. What
should actually happen is that the finalizer for the newly created session
will run and cause the parent of the session or link (the connection or
session) to inspect the child's state. Based on that state it may create a
new reference to the child, e.g. if there is outstanding work on the wire
to be done for that session or link. In the case of a new session or link
this is always the case, so it will always end up creating a new reference
to it, but this should never result in a child with a non zero refcount
being returned or visible in any way to user code.
I suspect this was done to simplify things for the newer mode of use, where
> a wrapper object can always be instantiated and it simply increments the
> ref count. Would be good to have that confirmed though, as to me this is
> the source of confusion and complexity.
>
Yes, this is certainly true, but it is also to accommodate the memory model
the C interface exposes. In the C interface sessions and links are owned by
their parent objects, e.g. freeing the connection frees all the contained
sessions. The way this is accommodated is that the parent object is what
owns the reference to the child by default. What is returned from the
pn_session()/pn_link() calls is a borrowed reference. (This changes when
you do an incref, see below for more details.)
> However this means that in the old mode of use, e.g.
> pn_session()/pn_session_free(), the object may have a refcount of 0 even
> though it has not been freed by the application.
>
As mentioned above, this should never be able to happen outside of a
finalizer. Perhaps what is confusing here is that the finalizer can create
a new reference to the object that is being finalized thereby causing the
pn_decref() to *appear* to be a noop, however what is really happening is
an important state change, the last reference is being deleted, and the
finalizer has decided to create a new reference for other purposes (e.g.
because there is outstanding transport work to be done with that object or
because it is going into a pool) and the state of the object (or related
objects/state) is being changed in key ways to reflect this.
Note that this pattern is not at all unique to proton's refcount system.
All GC systems that have finalizers accommodate these sorts of semantics,
i.e. finalizers causing objects to become live again. This is fundamental
to finalizers since they are just user code and can do whatever they want,
including create new references.
What is confusing about it here is more to do with how this capability is
being used to interact with all the engine data structures that predate
both the refcount system and the collection objects that make use of the
refcount system. I believe ultimately the engine data structures should be
reworked to use the various collection objects now available, at which
point a lot of this will become much more centralized, self contained, and
understandable and likely won't require so much magic (or at least the
magic will be in one place rather than spread around like it is now).
>
> 3. If another owning reference is created, the refcount MUST BE
>> incremented.
>>
>
> This is not currently the case for all internal use which necessitates
> some extra logic and checks.
>
I might be about to restate what you are saying in a different way, but the
refcount is actually incremented in all cases. What is tricky (and this is
admittedly the one part of all this that is unique to proton's refcount
system), is that proton's refcount system provides a hook for invoking
object-specific code whenever a reference is incremented. The session and
link objects override this hook and based on their state they will
sometimes remove the reference their parent is keeping to them. When this
happens it appears that the incref has been a noop if you only examine the
refcount, however once again there are key state changes and what has
really happened is that the refcount has increased momentarily and then
been decreased again by the hooks.
>
> 4. The owner of a reference MUST decrease the refcount on dropping the
>> reference and MUST NOT use the pointer after refcount is decremented.
>> It no longer "owns" a reference.
>>
>> 5. The object MAY be deleted when refcount goes to 0 or MAY be pooled
>> for re-use but it MUST NOT be used by any code (other than the pooling
>> system if there is one)
>>
>
> As above, that is currently not completely true at least for sessions and
> links, where they may be in use while still having a 0 ref count.
>
Hopefully this should be clear by now, but it isn't actually possible to
use the object when there refcount is 0. I think Alan's statement (5) needs
to be modified to include finalizers in general.
>
> 6. You never examine the refcount (except for debugging) Either you own
>> a reference, which means refcount > 0, or you don't, which means you
>> MUST NOT touch the object, even to examine its refcount.
>>
>
> My questions:
>
> * what are the respective roles of the ref count in pn_endpoint_t, and in
> the pni_head_t record associated with pn_session_t and pn_link_t? i.e. why
> are there two separate ref counts for the same object?
>
These are independent mechanisms and logically unrelated although probably
often correlated in practice. The one in pn_head_t is used purely to track
memory ownership. The one in pn_endpoint_t is used to figure out when it is
safe for a given endpoint to emit it's FINAL event.
>
> * pn_link_new() and pn_session() both decrement the count (in the
> pni_head_t record) for the newly created objects. Why?
>
> * what exactly does the 'referenced' flag indicate?
>
> * when a finalize function is exited early due to the return value of
> pni_preserve_child, the reference count is incremented to prevent that
> object being freed by the code in pn_class_decref. Where/how is that then
> decremented again to retrigger the finalizer?
>
I'm going to try to answer all three of the above at once as they are
related. This is all related to the memory management of Connections,
Sessions, Links, and Deliveries, so it helps to have a picture of the
relevant object model in your head:
1 n 1 n 1 n
Connection-----Session-----Link-----Delivery
While we generally think of this as a tree, i.e. Deliveries are children of
Links are children of Sessions are children of Connections, because you can
navigate in both directions, from a memory management perspective this is
actually a graph.
Terminology-wise it is also helpful to keep in mind the difference between
*references* (i.e. a pointer for which there is a corresponding incref) and
*borrowed references*. A borrowed reference is a pointer which does not
have a corresponding incref and depends on the fact that another reference
exists and is only valid while that other reference continues to exist.
With that in mind there are several related problems here that are being
solved:
1) The memory model for C style usage (i.e. the original engine memory
model) only has references pointing from parent to child. This means that
when a parent is freed, all references the application may have retrieved
to any children become invalid since they depend on the borrowed parent
reference. Maintaining compatibility with this model is one key element of
the overall problem.
2) Exposing the engine objects via the python binding requires the engine
objects to match python's memory model, and the C semantics described in
(1) are incompatible with this. In particular, wrapping the borrowed child
reference results in dangling pointers if/when the last reference to the
parent goes away. The net of this is that mapping the engine object model
into python (or any other high level language) effectively requires that
the engine objects support full GC semantics, i.e. child objects can keep
parents alive, parent objects can keep children alive, etc.
3) Because this is all being maintained with reference counting, we need to
avoid reference cycles.
The scheme in place that addresses all three of these is most easily
described in terms of the generic parent/child relationship, however it is
(to some degree) implemented at each level of the tree.
Between any given child and parent, there are always pointers going in both
directions, i.e. a pointer from parent to child and a pointer from child to
parent, however at any given time, only one of those pointers is a
reference (making the other pointer a borrowed reference). Under specific
conditions these two will swap, i.e. the borrowed reference becomes a
reference and the reference becomes a borrowed reference. While this
results in a net change of zero in the reference count, it has a
significant impact on how and when the decref that ultimately collects a
given object occurs.
This scheme is really at the heart of the solution and is what allows
parents to keep children alive, and children to keep parents alive without
having pointer cycles. To understand the details, it's simply necessary to
understand the circumstances under which the reference semantics between
child and parent swap. There are two state variables that control this. The
"freed" boolean which tracks whether pn_xxx_free() has been called on a
given child, as well as the "referenced" boolean which tracks the
directionality of the reference between parent and child. The referenced
boolean is true when an objects parent pointer is a reference, and false
when it is not.
When a given object is externally referenced, e.g. for a
session/link/delivery that is wrapped by a python object that increfed it,
or indeed if it was simply manually increfed via the C API, the incref hook
flips the reference semantics such that the child object holds a reference
to the parent. When the last reference for an object goes away and the
object is being finalized, the finalizer for the child examines the
freed/reference fields and if freed is not true, i.e. the object has not
been freed, then rather than letting the object be collected, it creates a
reference from its parent to itself so long as the parent is live.
The above behavior results in something that solves the 3 problems. For
normal C style usage it behaves as the original C model behaves with the
only references being from parent to child and all child references being
borrowed. As soon as you incref a child, it will flip the directionality of
the parent/child reference and you get the behavior you would expect from a
typical fully GC'ed object graph.
>
> * when should pn_class_free be used v pn_class_decref? (The former does
> not check whether the finalizer increases the reference, the latter does)
>
In general you should only call pn_free or pn_class_free when you allocated
the object and you know it's not possible for there to be any other
references, and you want to assert that is the case. In general pn_free and
pn_class_free should be fully equivalent to asserting the refcount is one
and then calling decref and asserting that the object was actually freed.
The only exception to this is objects that don't support refcounts (see
below).
>
> * pn_class_free() has the precondition that the object passed in should
> have a refcount of 1 (which I understand) or -1. When and why would the
> refcount be -1?
>
A proton class is basically just a collection of function pointers to be
used with a certain kind of void * pointer. Those function pointers include
things like being able to pretty print the contents of the pointer, but
also things like incref/decref. Not all pointers support the notion of
incref/decref and a refcount of -1 is used to signal this. The intent here
is to be able to pass around proton objects, python objects, and even dumb
void * pointers in a uniform way.
--Rafael
