[issue30156] PYTHONDUMPREFS segfaults on exit

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +9768

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

STINNER Victor  added the comment:

I proposeto leave Python 3.6 and 3.7 unchanged. I don't think that this 
specific bug matters enough to remove to remove an optimization from these 
stable branches. Since the optimization has been added (2015, bpo-23910), I'm 
only aware of two bug reports on PYTHONDUMPREFS (this one, and my duplicate).

If someone considers that the optimization must die in 3.6 and 3.7, please 
reopen the issue (or just add a comment).

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

STINNER Victor  added the comment:


New changeset e972c13624c32d0efdceb08ff83917fb6b488525 by Victor Stinner in 
branch 'master':
bpo-30156: Remove property_descr_get() optimization (GH-9541)
https://github.com/python/cpython/commit/e972c13624c32d0efdceb08ff83917fb6b488525


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

STINNER Victor  added the comment:

Note: I reported a similar issue which has then been marked as a duplicate of 
this one: bpo-34223. Copy of my first message:
---
It seems like Python has an invalid object somewhere. PYTHONDUMPREFS=1 makes 
Python crash at exit.

$ PYTHONDUMPREFS=1 ./python -c pass
(...)
0x7f1992292448 [1] (, )
0x7f1992241aa8 [1] {'__doc__': 'Thread-local dummy'}
0x7f199222c740 [1] (,)
0x9c98a0 [2] 
0x7f1992217dc0 [1] 
Segmentation fault (core dumped)
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

STINNER Victor  added the comment:

Is this optiization really worth it? 7 nanoseconds faster, but leak a weird 
tuple which caused 3 different crashes...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

STINNER Victor  added the comment:

PR 9541: my new attempt to remove the micro-optimization. Commit message:

bpo-30156: Remove property_descr_get() optimization

property_descr_get() uses a "cached" tuple to optimize function
calls. But this tuple can be discovered in debug mode with
sys.getobjects(). Remove the optimization, it's not really worth it
and it causes 3 different crashes last years.

Microbenchmark:

./python -m perf timeit -v \
-s "from collections import namedtuple; P = namedtuple('P', 'x y'); p = 
P(1, 2)" \
--duplicate 1024 "p.x"

Result:

Mean +- std dev: [ref] 32.8 ns +- 0.8 ns -> [patch] 40.4 ns +- 1.3 ns: 1.23x 
slower (+23%)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +8943

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Result on the same computer:
 
Mean +- std dev: [python0] 146 ns +- 5 ns -> [python] 206 ns +- 2 ns: 1.41x 
slower (+41%)

The difference now is smaller (41% instead of 58%) because calling with a tuple 
now is 16% slower.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Maybe the difference is processor depending. I'm going to repeat benchmarking 
on 32-bit processors. Or maybe fast call was optimized in master. Or passing 
via an arguments tuple was pessimized.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

STINNER Victor  added the comment:

Note: if you care of namedtuple performance, Raymond Hettinger wrote that he 
would be interested to reuse the C structseq sequence. I measured that getting 
an attribute by name is faster in structseq than with the current property 
cached tuple hack.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

STINNER Victor  added the comment:

> Removing this micro-optimization makes attribute access in namedtuple more 
> than 1.5 times slower:
> Mean +- std dev: [python.default] 126 ns +- 4 ns -> [python] 200 ns +- 7 ns: 
> 1.58x slower (+58%)

I wrote the PR 3985, it's only 20 ns slower (1.3x slower):

[ref] 80.4 ns +- 3.3 ns -> [fastcall] 103 ns +- 5 ns: 1.28x slower (+28%)

Maybe Python was optimized further in the meanwhile, or the slowdown is higher 
on your computer?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +3961

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The previous result was got when use _PyObject_FastCallDict(). Using 
PyObject_Call() is slightly faster:

Mean +- std dev: [python.default] 127 ns +- 4 ns -> [python] 185 ns +- 9 ns: 
1.45x slower (+45%)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Removing this micro-optimization makes attribute access in namedtuple more than 
1.5 times slower:

Mean +- std dev: [python.default] 126 ns +- 4 ns -> [python] 200 ns +- 7 ns: 
1.58x slower (+58%)

This would be a significant regression.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

STINNER Victor added the comment:

Come on, yet another crash from property_descr_get()??? It's the 3rd time... Do 
we really need this micro-optimization?

Previous bugs and workarounds:

- issue #26811
- issue #24276

Using the FASTCALL calling convention, no temporary tuple is created to pass 
arguments if you use the _PyObject_FastCall() API and if the called function 
supports this calling convention.

Sadly, namedtuple.attr uses operator.itergetter, itemgetter_call() doesn't 
support the FASTCALL, and my tp_fastcall patch was rejected: issue #29259.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The regression was added by the fix for issue26811. PR 1272 applies the 
alternate patch from issue26811. This doesn't harm the performance.

$ ./python.patched -m perf timeit -q --compare-to ./python.default -s "from 
collections import namedtuple; P = namedtuple('P', 'x y'); p = P(1, 2)" 
--duplicate 1000 "p.x"
Mean +- std dev: [python.default] 128 ns +- 7 ns -> [python.patched] 121 ns +- 
7 ns: 1.05x faster (-5%)

I thought about tests, but I don't know what is the best place for them. Seems 
other environment variables that controls debug output are not tested too.

--
nosy: +haypo
stage:  -> patch review
versions: +Python 3.5, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Oren Tirosh]

Oren Tirosh added the comment:

In addition to fixing this - perhaps PYTHONDUMPREFS or something similar should 
be added to test automation? It is apparently capable of uncovering some bugs 
that none of the other reference and recnt debugging tools could find.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1383

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Oren Tirosh]

New submission from Oren Tirosh:

Reproduce: 

Py_DEBUG build
PYTHONDUMPREFS=1 ./python -c pass
(big dump of reference information)
Segmentation fault 

git-bisected to commit 7822f151b68e40376af657d267ff774439d9adb9

--
components: Interpreter Core
messages: 292232
nosy: orent, serhiy.storchaka
priority: normal
severity: normal
status: open
title: PYTHONDUMPREFS segfaults on exit
type: crash
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com