Nick Coghlan wrote: >> Raymond, there is one thing I don't understand in your line of >> reasoning. You say that you prefer explicit finalization, but that >> implicit finalization still needs to be supported. And for that, >> you'd rather drop __del__ and use weakrefs. But why? You say that >> __del__ is harardous, but I can't see how weakrefs are less >> hazardous. > > As I see it, __del__ is more hazardous because it's an attractive > nuisance - it *looks* like it should be easy to use, but I'm willing > to bet that a lot of the __del__ methods implemented in the wild are > either actual or potential bugs. For example, it would be easy for a > maintenance programmer to make a change to include a reference in a > data structure from a child node back to its parent node to address a > problem, and suddenly the application's memory usage goes through the > roof due to uncollectable cycles.
Is that easier or harder to detect such a cycle, compared to accidentally adding a reference to self (through implicit nested scopes, or bound methods) in the finalizer callback? You have to admit that, at best, they are equally hazardous. As things stand *now* (in Python 2.5 I mean), __del__ is easier to understand/teach, easier to debug (gc.garbage vs finalizers silently ignored), and easier to use (no boilerplate in user's code, no additional finalization API which does not even exist). I saw numerous proposal to address these weakref "defects" by adding some kind of finalizer API, by modifying the GC to put uncollectable loops with weakref finalizers in gc.garbage, and so on. Most finalization APIs (including yours) create cycles just by using them, which also mean that you *must* wait for the GC to kick in before the object is finalized, making it useless for several situations where you want implicit finalizations to happen immediately (file.close() just to name one). [and we are speaking of implicit finalization now, I know of 'with']. It would require some effort to make weakref finalizers *barely* as usable as __del__, and will absolutely not solve the problem per-se: the user will still have to pay attention and understand the hoops (different kind of hoops, but still hoops). So, why do we not spend this same time trying to *fix* __del__ instead? If somebody comes up with a sane way to define the semantic for a new finalizer method (like the __close__ proposal), which can be invoked *even* in the case of cycles, would you still prefer to go the weakref way? > Even the initial implementation of > the generator __del__ slot in the *Python 2.5 core* was buggy, > leading to such cycles - if the developers of the Python interpreter > find it hard to get __del__ right, then there's something seriously > wrong with it in its current form. I don't think it's a fair comparison: generator is a pretty complex class, compared to an average class developed in Python which might need a __del__ method. I would also bet that you would get your first attempt of finalization of generators through weakrefs wrong. > By explicitly stating that __del__ will go away in Py3k, with the > current intent being to replace it with explicit finalization (via > with statements) and the implicit finalization offered by weakref > callbacks, it encourages people to look for ways to make the API for > the latter easier to use. > > For example, a "finalizer" factory function could be added to weakref: > > _finalizer_refs = set() > def finalizer(*args, **kwds): > """Create a finalizer from an object, callback and keyword > dictionary""" # Use positional args and a closure to avoid > namespace collisions obj, callback = args > def _finalizer(_ref=None): > """Callable that invokes the finalization callback""" > # Use closure to get at weakref to allow direct invocation > # This creates a cycle, so this approach relies on cyclic GC > # to clean up the finalizer objects! > try: > _finalizer_refs.remove(ref) > except KeyError: > pass > else: > callback(_finalizer) > # Give callback access to keyword arguments > _finalizer.__dict__ = kwds > ref = weakref.ref(obj, _finalizer) > _finalizer_refs.add(ref) > return _finalizer So uhm, am I reading it bad or your implementation (like any other similar API I have seen till now) create a cycle *just* by using it? This finalizer API ofhuscates user code by forcing to use a separate _data object to hold (part of) the context for apparently no good reason, and make the object collectable *only* through the cyclic GC (while __del__ would happily be invoked in simple cases when the object goes out of context). > P.S. the central finalizers list also works a treat for debugging why > objects aren't getting finalized as expected - a simple loop like > "for wr in weakref.finalizers: print gc.get_referrers(wr)" after a > gc.collect() call works pretty well. Yes, this is indeed interesting. One step closer to get at the __del__ feature set :) -- Giovanni Bajo _______________________________________________ Python-3000 mailing list [email protected] http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
