Neil Schemenauer added the comment:
My preference would be for --with-mimalloc=yes in an upcoming release. For
platforms without the required stdatomic.h stuff, they can manually specify
--with-mimalloc=no. That will make them aware that a future release of Python
might no longer build
Neil Schemenauer added the comment:
Thanks, I'm indeed interested. Most credit goes to Christian for advancing
this.
For the missing stdatomic.h, would it be appropriate to have an autoconfig
check for it? Can just disable mimalloc if it doesn't exist
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
Closing since I believe the bug is fixed and there is an appropriate unit test.
--
assignee: -> nascheme
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tra
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +27415
pull_request: https://github.com/python/cpython/pull/29138
___
Python tracker
<https://bugs.python.org/issue45
New submission from Neil Schemenauer :
Some makefile rules don't work if you build in a separate folder.
--
messages: 404671
nosy: nascheme
priority: normal
severity: normal
stage: patch review
status: open
title: Run smelly.py and multissltests.py from $(srcdir)
type: behavior
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
New changeset 1cdac61065e72db60d26e03ef9286d2743d7000e by Miss Islington (bot)
in branch '3.10':
bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051) (GH-29122)
https://github.com/python/cpython/commit/1cdac61065e72db60d26e03ef9286d2743d7000e
Neil Schemenauer added the comment:
New changeset 311910b31a4bd94dc79298388b7cb65ca5546438 by Neil Schemenauer in
branch 'main':
bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051)
https://github.com/python/cpython/commit/311910b31a4bd94dc79298388b7cb65ca5546438
Neil Schemenauer added the comment:
I have not yet been able to reproduce methane's crash. My guess it it's not
related.
An explanation of what I think the impact of this bug is:
The radix tree is used to determine if memory is from obmalloc or from the
system malloc (i.e return value
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +27329
pull_request: https://github.com/python/cpython/pull/29062
___
Python tracker
<https://bugs.python.org/issue45
New submission from Neil Schemenauer :
Given this feedback:
https://github.com/python/cpython/pull/14474/files#r725488766
it is perhaps not so safe to assume that only the lower 48 bits of virtual
addresses are significant. I had the idea that Go made similar assumptions but
now I'm
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +27322
pull_request: https://github.com/python/cpython/pull/29051
___
Python tracker
<https://bugs.python.org/issue45
New submission from Neil Schemenauer :
There is a typo in the radix tree obmalloc code, spotted by Inada Naoki.
-#define MAP_TOP_MASK (MAP_BOT_LENGTH - 1)
+#define MAP_TOP_MASK (MAP_TOP_LENGTH - 1)
This should be fixed both in the main branch and in 3.10.x.
--
assignee: methane
Neil Schemenauer added the comment:
[Larry]
> The one thing I guess I never mentioned is that building and working with the
> prototype was frightful; it had both Python code and C code, and it was
> fragile and hard to get working.
I took Larry's PR and did a fair amount o
Neil Schemenauer added the comment:
Another small correction, _PyType_IS_GC() checks only the type flag
(Py_TPFLAGS_HAVE_GC). _PyObject_IS_GC() checks both the type flag and calls
tp_is_gc(), if it exists. tp_is_gc() is wart, IMHO, and it would be nice if we
can kill it off so only
Change by Neil Schemenauer :
--
nosy: +pablogsal, tim.peters, vstinner
___
Python tracker
<https://bugs.python.org/issue44897>
___
___
Python-bugs-list mailin
Neil Schemenauer added the comment:
Based on some testing, I think an extra type slot is not worth the extra
complication. I made some small improvements to _Py_Dealloc and now the
performance seems a bit better. Basically, I expanded _PyObject_IS_GC() to put
the most common branches
Change by Neil Schemenauer :
Added file: https://bugs.python.org/file50220/perf-annotate-trash.txt
___
Python tracker
<https://bugs.python.org/issue44897>
___
___
Pytho
Neil Schemenauer added the comment:
As I suspected, the performance impact is significant (although pretty small).
Based on Linux perf, it looks like the extra test+branch in _Py_Dealloc adds
about 1% overhead. pyperformance shows something similar, see attached reports
(pypref
Neil Schemenauer added the comment:
> I think in any case we should benchmark this because this will affect *all*
> GC types if I understand correctly and the TS mechanism had shown slowdowns
> before
We definitely need to benchmark. I would guess that adding trashcan protection
Neil Schemenauer added the comment:
> The problem of PyObject_GC_UnTrack() is just the most visible effect of the
> trashcan mecanism: tp_dealloc can be called twice, and this is *not* expected
> by the tp_dealloc API.
The fact that Py_TRASHCAN_BEGIN and Py_TRASHCAN_END can cause t
Neil Schemenauer added the comment:
I wrote a proof-of-concept as bpo-44897. PR 27718 (this issue) might a
slightly better performance but I like the other one better because it doesn't
expose so much implementation detail to extension types. Either of them
require careful review before
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +26218
pull_request: https://github.com/python/cpython/pull/27738
___
Python tracker
<https://bugs.python.org/issue44
New submission from Neil Schemenauer :
This is a WIP/proof-of-concept of doing away with Py_TRASHCAN_BEGIN and
Py_TRASHCAN_END and instead integrating the functionality into _Py_Dealloc.
There are a few advantages:
- all container objects have the risk of overflowing the C stack if a long
Neil Schemenauer added the comment:
I was thinking about this more today and I think the better fix is to actually
build the trashcan mechanism into _Py_Dealloc(). Requiring that types opt-in
to the trashcan mechanism by using the trashcan macros is not ideal.
First, the trashcan macros
Neil Schemenauer added the comment:
Since C99 is now allowed, perhaps we should suggest that declarations go after
Py_TRASHCAN_BEGIN, e.g.
mytype_dealloc(mytype *p)
{
Py_TRASHCAN_BEGIN(p, mytype_dealloc)
... declarations go here
Neil Schemenauer added the comment:
For examples of bugs caused by forgetting the untrack calls, see bpo-31095.
--
___
Python tracker
<https://bugs.python.org/issue44
Neil Schemenauer added the comment:
Extensions that call PyObject_GC_UnTrack before calling Py_TRASHCAN_BEGIN will
still work, they will just take a very minor performance hit. I don't think it
is worth the trouble to introduce new macros for that reason. Extensions that
really care about
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +26201
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/27718
___
Python tracker
<https://bugs.python.org/issu
Neil Schemenauer added the comment:
I'm thinking that the explicit call to PyObject_GC_UnTrack should be made
unnecessary by integrating it into the trashcan code. That way, we avoid
someone else running into this kind of bug in the future. See bpo-44881.
--
nosy: +nascheme
New submission from Neil Schemenauer :
The fix for bpo-33930 includes a somewhat mysterious comment:
// The Py_TRASHCAN mechanism requires that we be able to
// call PyObject_GC_UnTrack twice on an object.
I wonder if we can just integrate the untrack into the body of the trashcan
Change by Neil Schemenauer :
--
resolution: -> rejected
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
It seems to me the old behaviour (don't forward hash) was done for good
reasons. If the referent might go away, it is not valid to use it as a dict
key since the hash and eq result changes. If it can't go away, what reason is
there to use a weakref
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +25161
pull_request: https://github.com/python/cpython/pull/26571
___
Python tracker
<https://bugs.python.org/issue44
New submission from Neil Schemenauer :
Note: This is a proof of concept and not ready for merging as is.
This is based on 'frozen_modules' from Jeethu Rao , via
Larry Hastings. Larry's git branch was:
g...@github.com:larryhastings/cpython.git
Change by Neil Schemenauer :
--
nosy: +nascheme
___
Python tracker
<https://bugs.python.org/issue42972>
___
___
Python-bugs-list mailing list
Unsubscribe:
Neil Schemenauer added the comment:
> If MTE is actually being used, system software assigns "random" values to 4
> of the higher-order bits.
Oh, interesting.
Two ideas about handling that: we could change our assertion check to be
different on ARM platforms that we kno
Neil Schemenauer added the comment:
It seems to not be specific to S390, same kind of failure on the PPC64LE RHEL8
LTO + PGE 3.x tester:
Exception ignored in:
Traceback (most recent call last):
File
"/home/buildbot/buildarea/3.x.cst
Neil Schemenauer added the comment:
Seems this failure might be back. At least, the traceback looks quite similar
to me.
The buildbot failed with this:
heads/master:85b6b70589, Mar 29 2021, 22:53:15
0:05:26 load avg: 3.95 [426/427] test_tokenize passed (56.0 sec) -- running:
test_asyncio
Neil Schemenauer added the comment:
I've merged PR 14474 so you can just test with an up-to-date "main" branch and
see if that fixes the problem. I would expect it should fix the problem since
with the radix tree arena tracking, no memory unsanitary behaviour remains.
-
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
New changeset 85b6b70589c187639aeebc560d67e9cc04abb4d8 by Neil Schemenauer in
branch 'master':
bpo-37448: Use radix tree for pymalloc address_in_range(). (GH-14474)
https://github.com/python/cpython/commit/85b6b70589c187639aeebc560d67e9cc04abb4d8
Neil Schemenauer added the comment:
Not sure the proper place to discuss this but I wonder if putting this stdlib
module names list in the executable is the best idea. The list of available
stdlib modules could change after compiling Python. I understand you don't
want to dynamically
Neil Schemenauer added the comment:
New changeset 87ec26b812e9c4095c017dc60f246eda37b83ab2 by Neil Schemenauer in
branch 'master':
bpo-43372: Use _freeze_importlib for regen-frozen. (GH-24759)
https://github.com/python/cpython/commit/87ec26b812e9c4095c017dc60f246eda37b83ab2
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
New changeset 87ec26b812e9c4095c017dc60f246eda37b83ab2 by Neil Schemenauer in
branch 'master':
bpo-43372: Use _freeze_importlib for regen-frozen. (GH-24759)
https://github.com/python/cpython/commit/87ec26b812e9c4095c017dc60f246eda37b83ab2
Change by Neil Schemenauer :
--
assignee: -> nascheme
___
Python tracker
<https://bugs.python.org/issue43372>
___
___
Python-bugs-list mailing list
Unsubscrib
Change by Neil Schemenauer :
--
pull_requests: +23532
pull_request: https://github.com/python/cpython/pull/24759
___
Python tracker
<https://bugs.python.org/issue42
Change by Neil Schemenauer :
--
pull_requests: +23531
pull_request: https://github.com/python/cpython/pull/24759
___
Python tracker
<https://bugs.python.org/issue43
Neil Schemenauer added the comment:
It seems it is enough to make a new commit. The CI seems to re-base and re-run
the PR. At least, it worked on two of my PRs.
--
___
Python tracker
<https://bugs.python.org/issue43
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +23494
pull_request: https://github.com/python/cpython/pull/24713
___
Python tracker
<https://bugs.python.org/issue43
New submission from Neil Schemenauer :
While I was fixing the regen-frozen issue, I noticed it seems unnecessary to
have regen-stdlib-module-names separate from regen-all. Maybe Victor knows why
it needs to be separate. If it doesn't need to be separate, the CI scripts can
be slightly
Neil Schemenauer added the comment:
I think it may be related to bpo-41561. There is a bug in the Ubuntu tracker
as well:
https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1899878
I agree with the temporary fix to use "ubuntu-18.04" for CI testing.
--
nosy:
Change by Neil Schemenauer :
--
pull_requests: +23492
pull_request: https://github.com/python/cpython/pull/24714
___
Python tracker
<https://bugs.python.org/issue42
Change by Neil Schemenauer :
--
pull_requests: +23491
pull_request: https://github.com/python/cpython/pull/24714
___
Python tracker
<https://bugs.python.org/issue43
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +23490
pull_request: https://github.com/python/cpython/pull/24712
___
Python tracker
<https://bugs.python.org/issue43
New submission from Neil Schemenauer :
In bug #43372, we didn't notice that the code for the __hello__ module was not
re-generated. Things seems to be okay but the line number table was corrupted.
It seems a good idea to add a small test to ensure that doesn't happen again.
I marked
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +23484
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/24708
___
Python tracker
<https://bugs.python.org/issu
Change by Neil Schemenauer :
--
nosy: +nascheme
nosy_count: 7.0 -> 8.0
pull_requests: +23485
pull_request: https://github.com/python/cpython/pull/24708
___
Python tracker
<https://bugs.python.org/issu
Neil Schemenauer added the comment:
I believe the line table format got changed but the frozen code didn't get
re-generated. If you try to call co_lines() on the __hello__ code, Python
crashes.
>>> import __hello__
Hello world!
>>> co = __hello__.__spec__.loader.g
Change by Neil Schemenauer :
--
nosy: +nascheme
nosy_count: 3.0 -> 4.0
pull_requests: +23486
pull_request: https://github.com/python/cpython/pull/24708
___
Python tracker
<https://bugs.python.org/issu
Change by Neil Schemenauer :
Added file: https://bugs.python.org/file49834/perf_compare_radix4x.txt
___
Python tracker
<https://bugs.python.org/issue37448>
___
___
Pytho
Change by Neil Schemenauer :
Added file: https://bugs.python.org/file49833/perf_compare_noradix.txt
___
Python tracker
<https://bugs.python.org/issue37448>
___
___
Pytho
Neil Schemenauer added the comment:
New changeset 44fe32061d60f4bd9c4fa48c24e3e4ba26033dba by Neil Schemenauer in
branch '3.9':
[3.9] bpo-43288: Fix bug in test_importlib test. (GH-24616)
https://github.com/python/cpython/commit/44fe32061d60f4bd9c4fa48c24e3e4ba26033dba
Change by Neil Schemenauer :
--
pull_requests: +23397
pull_request: https://github.com/python/cpython/pull/24616
___
Python tracker
<https://bugs.python.org/issue43
Neil Schemenauer added the comment:
New changeset 84f7afe65c29330f3ff8e318e054b96554a2dede by Neil Schemenauer in
branch 'master':
Fix failed merge of bpo-43288. (GH-24614)
https://github.com/python/cpython/commit/84f7afe65c29330f3ff8e318e054b96554a2dede
Change by Neil Schemenauer :
--
pull_requests: +23395
pull_request: https://github.com/python/cpython/pull/24614
___
Python tracker
<https://bugs.python.org/issue43
Neil Schemenauer added the comment:
New changeset 50288aa8c955f66ab67a7dadf250ea5f4238eb67 by Neil Schemenauer in
branch 'master':
bpo-43288: Fix bug in test_importlib test. (GH-24612)
https://github.com/python/cpython/commit/50288aa8c955f66ab67a7dadf250ea5f4238eb67
Change by Neil Schemenauer :
--
nosy: +nascheme
nosy_count: 9.0 -> 10.0
pull_requests: +23392
pull_request: https://github.com/python/cpython/pull/24612
___
Python tracker
<https://bugs.python.org/issu
Change by Neil Schemenauer :
--
nosy: +nascheme, nascheme
nosy_count: 9.0 -> 10.0
pull_requests: +23392, 23393
pull_request: https://github.com/python/cpython/pull/24612
___
Python tracker
<https://bugs.python.org/issu
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +23391
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/24612
___
Python tracker
<https://bugs.python.org/issu
New submission from Neil Schemenauer :
The FileSystem class is missing a skip() method. If the file system doesn't
support Unicode filenames, the test crashes.
File "/home/nas/src/cpython/Lib/test/test_importlib/fixtures.py", line 221,
in unicode_filename
self.skip("F
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: needs patch -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
New changeset ffa55d21b4a86ad8b4a43a9f597151e526541130 by Neil Schemenauer in
branch 'master':
bpo-39448: Add regen-frozen makefile target. (GH-18174)
https://github.com/python/cpython/commit/ffa55d21b4a86ad8b4a43a9f597151e526541130
Neil Schemenauer added the comment:
Maybe something to do the configuration of the machine, so that TCP/UDP socket
networking if failing? Two things to try:
- give the "python" process a SIGINT signal and you should get a traceback
showing where it is hanging. Or you coul
Neil Schemenauer added the comment:
I think the comments are correct in that it is used to create a new statically
linked interpreter that includes a user provided extension module. We could
include python.o inside the libpythonXX.a file but then I think it breaks if
you are embedding (e.g
Change by Neil Schemenauer :
--
resolution: -> out of date
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
This has been resolved using Py_TYPE() and Py_SET_TYPE().
--
resolution: -> out of date
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
I believe this is not needed anymore. Closing.
--
resolution: -> rejected
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Neil Schemenauer added the comment:
Correction: I think you *cannot* have it both ways.
--
___
Python tracker
<https://bugs.python.org/issue39573>
___
___
Pytho
Neil Schemenauer added the comment:
> I don't understand the rationale for this change in depth, but
> does the benefit outweigh (yet another) backwards incompatibility?
I think you can have it both ways. Do you want a C API that is
stable over a long period of CPython releases or
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +21784
pull_request: https://github.com/python/cpython/pull/22829
___
Python tracker
<https://bugs.python.org/issue42
New submission from Neil Schemenauer :
It is great that access to ob_type has been cleaned up to use an access macro.
There are two spots that still need fixing.
I think we should do something to help avoid this kind of thing slipping into
the code in the future. E.g. a special build flag
Neil Schemenauer added the comment:
I resurrected an old thread on Discourse that seems quite relevant to this PR:
https://discuss.python.org/t/switching-from-refcounting-to-libgc/1641/35?u=nas
--
___
Python tracker
<https://bugs.python.
Neil Schemenauer added the comment:
> immutability vs immutable object headers
Sorry, I meant to write "immortal vs immutable".
--
___
Python tracker
<https://bugs.pytho
Neil Schemenauer added the comment:
A few thoughts
First, the PR is well done. The changes are pretty minimal and are well
localized. Thanks Eddie for writing clean code and responding to review
requests.
My feeling is that, in current state, this still should not get merged or at
least
Neil Schemenauer added the comment:
Eddie mentions in the PR about using memory arenas to contain immortal objects.
I think it could be worth investigating further. With the current PR, the
immortal status is dependent on the value of the refcnt field of the object.
Using immortal arenas
Change by Neil Schemenauer :
--
keywords: +patch
pull_requests: +17558
pull_request: https://github.com/python/cpython/pull/18174
___
Python tracker
<https://bugs.python.org/issue39
New submission from Neil Schemenauer :
Updating the frozen module "__hello__" code inside Python/frozen.c is currently
a manual process. That's a bit tedious since it adds some extra work in the
case that bytecode changes are made. I've created a small script and a
makef
Neil Schemenauer added the comment:
New changeset 392a13bb9346331b087bcd8bb1b37072c126abee by Neil Schemenauer in
branch 'master':
bpo-38006: Add unit test for weakref clear bug (GH-16788)
https://github.com/python/cpython/commit/392a13bb9346331b087bcd8bb1b37072c126abee
Neil Schemenauer added the comment:
> It's fundamentally insane to expect any gc to work perfectly when it may be
> blind to what the object graph _is_.
I'm often amazed it works at all, let alone perfectly. ;-P
This did trigger me to think of another possible problem. I setup my uni
Change by Neil Schemenauer :
--
pull_requests: +16348
pull_request: https://github.com/python/cpython/pull/16788
___
Python tracker
<https://bugs.python.org/issue38
Neil Schemenauer added the comment:
Łukasz, is there some reason you removed old versions (2.7, 3.6, etc)? The bug
is present on those versions of Python and it should be trivial to backport the
fix. If those branches are maintained, I think we should fix it.
Attached is a small test
Change by Neil Schemenauer :
--
pull_requests: +16095
pull_request: https://github.com/python/cpython/pull/16502
___
Python tracker
<https://bugs.python.org/issue33
Neil Schemenauer added the comment:
I worked some on trying to create a unit test this evening. Attached is my
best result so far (gc_weakref_bug_demo.py). It requires that you enable the
'xx' module so we have a container object without tp_traverse. We should
probably add one
Change by Neil Schemenauer :
--
nosy: +benjamin.peterson, larry, ned.deily
versions: +Python 2.7, Python 3.5, Python 3.6, Python 3.7
___
Python tracker
<https://bugs.python.org/issue38
Neil Schemenauer added the comment:
I created GH-16502. I'm not exactly sure of the process of making a revert on
a branch like '3.8' so hopefully it is correct. The code change is exactly
what has been reverted in 3.8. The net effect will be as if the revert was
never done
Change by Neil Schemenauer :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
1 - 100 of 372 matches
Mail list logo
