[issue39873] Debug mode: check if objects are valid

[STINNER Victor]

STINNER Victor  added the comment:

This change was mostly an experimentation. But I'm not sure that it does solve 
any issue. The change is more intrusive than what I expected. I prefer to 
abandon PR 18803 and this issue.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39873] Debug mode: check if objects are valid

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9a73705a1d0cb8b89d0a20add2ffa2c4d32950ed by Victor Stinner in 
branch 'master':
bpo-39873: Cleanup _PyObject_CheckConsistency() (GH-18807)
https://github.com/python/cpython/commit/9a73705a1d0cb8b89d0a20add2ffa2c4d32950ed


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39873] Debug mode: check if objects are valid

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18165
pull_request: https://github.com/python/cpython/pull/18807

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39873] Debug mode: check if objects are valid

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 1fb5a9f394a6fdf62e21b96080c3257c959cf8c9 by Victor Stinner in 
branch 'master':
bpo-39873: PyObject_Init() uses PyObject_INIT() (GH-18804)
https://github.com/python/cpython/commit/1fb5a9f394a6fdf62e21b96080c3257c959cf8c9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39873] Debug mode: check if objects are valid

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +inada.naoki, pablogsal, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39873] Debug mode: check if objects are valid

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-38392: "Ensure that objects entering the GC are valid". I fixed 
this one with commit 1b1845569539db5c1a6948a5d32daea381f1e35f.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39873] Debug mode: check if objects are valid

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18160
pull_request: https://github.com/python/cpython/pull/18804

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39873] Debug mode: check if objects are valid

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +18159
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18803

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39873] Debug mode: check if objects are valid

[STINNER Victor]

New submission from STINNER Victor :

One of the worst issue that I had to debug is a crash in the Python garbage 
collector. It is usually a crash in visit_decref(). See my notes:
https://pythondev.readthedocs.io/debug_tools.html#debug-crash-in-garbage-collection-visit-decref

My previous attempt to help debugging such issue failed: bpo-36389 "Add 
gc.enable_object_debugger(): detect corrupted Python objects in the GC". The 
idea was to check frequently if all objects tracked by the GC are valid. The 
problem is that even if the check looked trivial, checking all objects made 
Python way slower. Even when I tried to only check objects of the "young" GC 
generation (generation 0), it was still too slow.

Here I propose a different approach: attempt to only check objects when they 
are accessed. Recently, I started to replace explicit cast to (PyObject *) type 
with an indirection: a new _PyObject_CAST() macro which should be the only way 
to cast any object pointer to (PyObject *).

/* Cast argument to PyObject* type. */
#define _PyObject_CAST(op) ((PyObject*)(op))

This macro is used in many "core" macros like Py_TYPE(op), Py_REFCNT(op), 
Py_SIZE(op), Py_SETREF(op, op2), Py_VISIT(op), etc.

The idea here is to inject code in _PyObject_CAST(op) when Python is built in 
debug mode to ensure that the object is valid. The intent is to detect 
corrupted objects earlier than a garbage collection, to ease debugging C 
extensions.

The checks should be limited to reduce the performance overhead.

Attached PR implemnts this idea.

--
components: Interpreter Core
messages: 363499
nosy: vstinner
priority: normal
severity: normal
status: open
title: Debug mode: check if objects are valid
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com