[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Eric Cousineau]

Eric Cousineau  added the comment:

See also bpo-40240: "Expose public spelling of _PyGC_FINALIZED and 
_PyGC_SET_FINALIZED?"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Eric Cousineau]

Eric Cousineau  added the comment:

Super late response, but for this part:

> Well... perhaps you could create another PyObject (it's just a wrapper,
> right?) since the old one doesn't have any outside references to it
> remaining.

In certain cases, yes, that would be the case. I do have unittests that
(hackily) check `id(o)` before and after resurrection by using a `weakref`,
so I could relax that contract.

However, in other cases, the "wrapper" object in Python is actually a C++
base class that has been derived from in Python. If ownership of the C++
instance (which should more-or-less own the Python portion) goes solely to C++,
then the Python portion could be garbage collected, and you get a sort-of
slicing effect:
https://github.com/pybind/pybind11/issues/1145

Granted, within this web of issues, one alternative is to always extend the
lifetime of the Python object by the C++ portion. However, that creates a
(opaque?) reference cycle between C++ and CPython, so I am a bit afraid of what
that would do. I would prefer to make memory management conservative if
possible.

Or rather: It's something cool to try out, but I have not yet had time to dig
in :(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Antoine Pitrou]

Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset dc5770b161a5e28eeff73a406cd4eddb0676c5b5 by Antoine Pitrou (Miss 
Islington (bot)) in branch '3.6':
bpo-32377: improve __del__ docs and fix mention about resurrection (GH-4927) 
(#4929)
https://github.com/python/cpython/commit/dc5770b161a5e28eeff73a406cd4eddb0676c5b5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 4b965930e8625f77cb0e821daf5cc40e85b45f84 by Antoine Pitrou in 
branch 'master':
bpo-32377: improve __del__ docs and fix mention about resurrection (#4927)
https://github.com/python/cpython/commit/4b965930e8625f77cb0e821daf5cc40e85b45f84


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4823

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I've proposed a documentation improvement in 
https://github.com/python/cpython/pull/4927 . Please chime in if you see have 
issues with it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +4822
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 19/12/2017 à 17:46, Eric Cousineau a écrit :
> 
> My current hack is to call `_PyGC_SET_FINALIZED(self, 0)` - may I ask if 
> there is a simpler way to do this?

Well... perhaps you could create another PyObject (it's just a wrapper,
right?) since the old one doesn't have any outside references to it
remaining.

Note that calling __del__ only once is also how PyPy works:
http://doc.pypy.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies

If there is some demand we could expose a higher-level spelling of
`_PyGC_SET_FINALIZED(self, 0)`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Eric Cousineau]

Eric Cousineau  added the comment:

You're welcome, and thank you for the prompt response!

I will say that it feels a tad odd to only have `tp_finalize` be called once 
for the entire lifetime of the object, while still having the option of it 
being resurrected.
Is there any way to somehow "un-mark" the object to enable this workflow that I 
would like to have?

My current hack is to call `_PyGC_SET_FINALIZED(self, 0)` - may I ask if there 
is a simpler way to do this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Thanks for the report.  Apparently I forgot to update that piece of 
documentation when PEP 442 was implemented.

--
nosy: +pitrou
stage:  -> needs patch
versions: +Python 3.7 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32377] Difference in ressurrection behavior with __del__ in py2 vs. py3

[Eric Cousineau]

New submission from Eric Cousineau :

Due to how `PyObject_CallFinalizer` is written in python3, `__del__` will only 
*ever* be called once.

In my use case, I am experimenting with a feature in `pybind11` to prevent 
slicing with Python class instances that inherit from pybind11-C++ base 
classes, which involves detecting when an instance loses all reference in 
Python (`Py_REFCNT(...) == 0`) but still has reference in C++ 
(`shared_ptr::count() > 0`), and reviving the Python portion when this 
situation happens.

In python2, I could do this without a hitch, as a resurrected object could have 
its `__del__` method called multiple times (through `tp_dealloc` I believe?). 
But in python3, the object is marked with `_PyGC_SET_FINALIZED(...)`, thus 
preventing `__del__` from being called again.

It'd be nice to either (a) somehow allow `__del__` to be called naturally 
without too much fuss or, at the least, (b) have this reflected in the 
documentation:
https://docs.python.org/3/reference/datamodel.html#object.__del__

See attached `revive_test`. Example execution:

```
$ python2 ./revive_test.py 
Revive
Destroy
[ Done ]

$ python3 ./revive_test.py 
Revive
[ Done ]
```

--
assignee: docs@python
components: Documentation
files: revive_test.py
messages: 308660
nosy: Eric Cousineau, docs@python
priority: normal
severity: normal
status: open
title: Difference in ressurrection behavior with __del__ in py2 vs. py3
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6
Added file: https://bugs.python.org/file47339/revive_test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com