Could there be a clearer separation between the language semantics and
current CPython behavior?
On 2020-01-02 03:53, Yonatan Zunger wrote:
Oh, I'm absolutely thinking about clarity. Something like:
----
This method is called when the instance is about to be destroyed.
Because it may be called by either the ordinary Python execution flow or
by the garbage collector, it has very unusual semantics, and must be
treated with great care.
The precise semantics of when __del__ is called on an object are
implementation-dependent. For example:
* It might be invoked during the normal interpreter flow at a moment
like function return, if the reference count for an object has dropped
to zero;
* It might be invoked by the garbage collector, which happens during the
normal interpreter flow but is called from an arbitrary thread;
* It might be invoked during interpreter shutdown, after various global
variables (including modules!) have already been deleted, or other
threads' interpreters have already stopped.
* It may not be invoked at all; it is /not guaranteed/ that __del__()
methods are called for objects that still exist when the interpreter exits.
Note that similar caveats apply for alternatives to __del__:
weakref.finalize and atexit.register.
Doing any of the following in an implementation of __del__() may easily
lead to bugs, nondeterministic behavior, and inconsistent behavior
across different implementations or versions of Python:
* Assuming that any variables outside self exist, including
modules or global variables;
* Attempting to acquire locks or otherwise block, since they may
(e.g.) be invoked inside a thread which already holds those resources;
* Relying on being able to signal other threads and wait for their
responses (even thread joins), since those threads' interpreters may
have exited;
* Causing failures (e.g. leaving external devices in an invalid
state) if __del__ is /never/ invoked.
In particular, del x does not call x.__del__() directly: the call may be
delayed or not happen at all.
In CPython, del x decrements the reference count for x by one, and
x.__del__() is only called either when x’s reference count reaches zero,
or by the cyclic garbage collector at an unspecified time (e.g. if a
reference cycle prevents the reference count from going to zero).
If a base class has a __del__() method, the derived class’s __del__()
method, if any, must explicitly call it to ensure proper deletion of the
base class part of the instance.
It is possible (though not recommended!) for the __del__() method to
postpone destruction of the instance by creating a new reference to it.
This is called object resurrection. It is implementation-dependent
whether __del__() is called a second time when a resurrected object is
about to be destroyed.
The current CPython implementation usually calls __del__ only once for
each object, but this is not guaranteed in all cases.
On Wed, Jan 1, 2020 at 12:57 PM Andrew Svetlov <andrew.svet...@gmail.com
<mailto:andrew.svet...@gmail.com>> wrote:
If the warning text tells about the delayed execution -- I'm +1.
-1 for something like "just don't use __del__, it is dangerous".
On Wed, Jan 1, 2020, 21:51 Gregory P. Smith <g...@krypto.org
<mailto:g...@krypto.org>> wrote:
On Wed, Jan 1, 2020 at 6:40 AM Andrew Svetlov
<andrew.svet...@gmail.com <mailto:andrew.svet...@gmail.com>> wrote:
__del__ is very useful not for interpreter shutdown but as a
regular
destructor in object's lifecycle.
The reason we should warn people against ever implementing
__del__ is that people rarely actually understand object
lifecycle. Its presence guarantees delayed garbage collection
and that its code will code execute in a context way outside of
normal control flow that all other methods are invoked from.
-gps
Action on the shutdown is another beast.
Personally, I prefer to do all finalization works by
explicit calls instead.
On Wed, Jan 1, 2020 at 2:39 AM Yonatan Zunger
<zun...@humu.com <mailto:zun...@humu.com>> wrote:
>
> Hey everyone,
>
> I just encountered yet another reason to beware of
__del__: when it's called during interpreter shutdown, for
reasons which are kind of obvious in retrospect, if it calls
notify() on a threading.Condition, the waiting thread may or
may not ever actually receive it, and so if it does that and
then tries to join() the thread the interpreter may hang in
a hard-to-debug way.
>
> This isn't something that can reasonably be fixed, and
(like in most cases involving __del__) there's a very simple
fix of using weakref.finalize instead. My question for the
dev list: How would people feel about changing the
documentation for the method to more bluntly warn people
against using it, and refer them to weakref.finalize and/or
atexit.register as an alternative? The text already has an
undertone of "lasciate ogni speranza, voi ch'entrate" but it
may be helpful to be more explicit to avoid people getting
caught in unexpected pitfalls.
>
> Yonatan
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
<mailto:python-dev@python.org>
> To unsubscribe send an email to
python-dev-le...@python.org <mailto:python-dev-le...@python.org>
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/AAZQBWD6PHC4PVNCCPX4A2745SS7B3LS/
> Code of Conduct: http://python.org/psf/codeofconduct/
--
Thanks,
Andrew Svetlov
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
<mailto:python-dev@python.org>
To unsubscribe send an email to python-dev-le...@python.org
<mailto:python-dev-le...@python.org>
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/57CDW4NIYEQ3JEVX2JVCJDA5TXTC5MBR/
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/NKWOXME256DT53PRLU3IMWORT2DZTRHO/
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/7OKNWI32HPTFRKJUZFL5EKSV4MDU4Q5D/
Code of Conduct: http://python.org/psf/codeofconduct/
