On 10/10/2016 10:38 PM, Chris Angelico wrote:
On Tue, Oct 11, 2016 at 8:14 AM, Larry Hastings <la...@hastings.org> wrote:
These hacks where we play games with the
reference count are mostly removed in my branch.
That's exactly what I would have said, because I was assuming that
refcounts would be accurate. I'm not sure what you mean by "play games

By "playing games with reference counts", I mean code that purposely doesn't follow the rules of reference counting. Sadly, there are special cases that apparently *are* special enough to break the rules. Which made implementing "buffered reference counting" that much harder.

I currently know of two examples of this in CPython. In both instances, an object has a reference to another object, but *deliberately* does not increase the reference count of the object, in order to prevent keeping the other object alive. The implementation relies on the GIL to preserve correctness; without a GIL, it was much harder to ensure this code was correct. (And I'm still not 100% I've done it. More thinking needed.)

Those two examples are:

1. PyWeakReference objects.  The wr_object pointer--the "reference"
   held by the weak reference object--points to an object, but does not
   increment the reference count.  Worse yet, as already observed,
   PyWeakref_GetObject() and PyWeakref_GET_OBJECT() don't increment the
   reference count, an inconvenient API decision from my perspective.
2. "Interned mortal" strings.  When a string is both interned *and*
   mortal, it's stored in the static "interned" dict in
   unicodeobject.c--as both key and value--and then its's DECREF'd
   twice so those two references don't count.  When the string is
   destroyed, unicode_dealloc resurrects the string, reinstating those
   two references, then removes it from the "interned" dict, then
   destroys the string as normal.

To support these, I've implemented what is effectively a secondary, atomic-only reference count. It seems to work. (And yes that means all objects are now 8 bytes bigger. Let me worry about memory consumption later, m'kay?)

Resurrecting object also gave me a headache in the Gilectomy with this buffered reference counting scheme, but I think I have that figured out too. When you resurrect an object, it's generally because you're going to expose it to other subsystems that may incr / decr / otherwise inspect the reference count. Which means that code may buffer reference count changes. Which means you can't immediately destroy the object anymore. So: when you resurrect, you set the new reference count, you also set a flag saying "I've already been resurrected", you pass it in to that other code, you then drop your references with Py_DECREF, and you exit. Your dealloc function will get called again later; you then see you've already done that first resurrection, and you destroy as normal. Curiously enough, the typeobject actually needs to do this twice: once for tp_finalize, once for tp_del. (Assuming I didn't completely misunderstand what the code was doing.)

My struggles continue,


Python-Dev mailing list

Reply via email to