[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-30 Thread Neil Schemenauer
Change by Neil Schemenauer : -- pull_requests: +16092 pull_request: https://github.com/python/cpython/pull/16502 ___ Python tracker <https://bugs.python.org/issue38

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-30 Thread Neil Schemenauer
Neil Schemenauer added the comment: > Is introducing tp_clear on functions a thing that has ABI consequences? In > other words, if we take our time on this, would it land in 3.8.1 or 3.9.0? I think it should not have ABI consequences. However, I see the addition of tp_clear as

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-30 Thread Neil Schemenauer
Neil Schemenauer added the comment: > Would [func tp_clear] help with memory usage in functions or was BPO-33418 > addressed in another way since? Having a tp_clear for all container objects that can be involved in reference cycles will help the GC free memory. BPO-33418 may be con

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-30 Thread Neil Schemenauer
Neil Schemenauer added the comment: Oops, I linked to wrong PR, my proposed fix is GH-16495. It is the same as what Tim suggests in his last comment. -- ___ Python tracker <https://bugs.python.org/issue38

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-30 Thread Neil Schemenauer
Neil Schemenauer added the comment: > Why setting it back to release blocker? As far as I recall, this issue is not > a Python 3.8 regression. The regression which triggered this old and existing > bug has been fixed, see previous comments. I leave it up to our glorious releas

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-30 Thread Neil Schemenauer
Change by Neil Schemenauer : -- pull_requests: +16083 pull_request: https://github.com/python/cpython/pull/16495 ___ Python tracker <https://bugs.python.org/issue38

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-29 Thread Neil Schemenauer
Neil Schemenauer added the comment: > We can have finalizer code running during delete_garbage(). That > code should not have access to non-valid objects. Weakrefs seem be > a way to violate that. handle_weakrefs() take care of some of them > but it seems there are other is

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-29 Thread Neil Schemenauer
Neil Schemenauer added the comment: > Fleshing out something I left implicit: if there's a trash object T > with a finalizer but we don't KNOW it's trash, we won't force-run its > finalizer before delete_garbage starts either. Then, really the same > thing: we may tp_clear some pi

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-29 Thread Neil Schemenauer
Neil Schemenauer added the comment: Since W is in the unreachable set, we should not be executing its callback. Would the attached rough patch (gc_disable_wr_callback.txt) be a possible fix? When we find W inside handle_weakrefs(), we mark it as trash and will not execute the callback

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-29 Thread Neil Schemenauer
Neil Schemenauer added the comment: I hacked up my copy of CPython to add flags to PyObject to see the GC state of objects. That makes it easier to know if an object is in the 'unreachable' list or not. > We must know that F is trash, else we never would have called tp_clear(F).

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-29 Thread Neil Schemenauer
Neil Schemenauer added the comment: Victor: > I'm surprised that the function would be seen as "unreachable" if > it's reference counter was equal to 135: If all those references are contained within that garbage set, it is possible. Pablo: > For the weakref to b

[issue38009] Handle weakreference callbacks invoked indirectly in the middle of a gc collection

2019-09-27 Thread Neil Schemenauer
Neil Schemenauer added the comment: I think the problem with your logic is that the weakref to F is part of the garbage set. So, handle_finalizers() should detect that and clear the finalizer rather than call it. Once we get to delete_garbage() and start calling tp_clear(), we can't

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-09-27 Thread Neil Schemenauer
Neil Schemenauer added the comment: A few comments from the "peanut gallery". Thanks to Victor and Pablo for doing the hard work of investigating this bug. First, it has been a long time since Tim and I first developed gcmodule.c. So, some of my understanding may be inaccurate d

[issue37725] "make clean" should remove PGO task data

2019-07-30 Thread Neil Schemenauer
Change by Neil Schemenauer : -- keywords: +patch pull_requests: +14790 pull_request: https://github.com/python/cpython/pull/15033 ___ Python tracker <https://bugs.python.org/issue37

[issue37725] "make clean" should remove PGO task data

2019-07-30 Thread Neil Schemenauer
New submission from Neil Schemenauer : I find it annoying and surprising that "make clean" does not remove the PGO data. If you change a source file, running "make clean" and "make" should result in a newly built executable, IMHO. As it is now, you usually

[issue37707] Skip individual unit tests that are expensive for the PGO task

2019-07-30 Thread Neil Schemenauer
Neil Schemenauer added the comment: New changeset 382cb85401bb010ead411c0532499ffe16c3cf27 by Neil Schemenauer (Miss Islington (bot)) in branch '3.8': bpo-37707: Exclude expensive unit tests from PGO task (GH-15009) (#15024) https://github.com/python/cpython/commit

[issue37707] Skip individual unit tests that are expensive for the PGO task

2019-07-30 Thread Neil Schemenauer
Change by Neil Schemenauer : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.or

[issue36044] PROFILE_TASK for PGO build is not a good workload

2019-07-30 Thread Neil Schemenauer
Neil Schemenauer added the comment: Closing as I think PR 14702 mostly resolves this. The gc_collect() issue has not been resolved but based on some investigation, I think it is not worth the effort to try to reduce the time spend in that function. Reducing the number of unit tests being

[issue37707] Skip individual unit tests that are expensive for the PGO task

2019-07-30 Thread Neil Schemenauer
Neil Schemenauer added the comment: New changeset 52a48e62c6a94577152f9301bbe5f3bc806cfcf1 by Neil Schemenauer in branch 'master': bpo-37707: Exclude expensive unit tests from PGO task (GH-15009) https://github.com/python/cpython/commit/52a48e62c6a94577152f9301bbe5f3bc806cfcf1

[issue36044] PROFILE_TASK for PGO build is not a good workload

2019-07-29 Thread Neil Schemenauer
Neil Schemenauer added the comment: I think expanding the list of tests used by --pgo is fine, as long as we put a little thought into each one we add. The ones added by Steve look fine to me. It seems useful to run a profiler and see if there are any unusually expensive tests being added

[issue37707] Skip individual unit tests that are expensive for the PGO task

2019-07-29 Thread Neil Schemenauer
Change by Neil Schemenauer : -- keywords: +patch pull_requests: +14772 pull_request: https://github.com/python/cpython/pull/15009 ___ Python tracker <https://bugs.python.org/issue37

[issue37707] Skip individual unit tests that are expensive for the PGO task

2019-07-29 Thread Neil Schemenauer
New submission from Neil Schemenauer : Add a new support decorator, @skip_if_pgo_task and then use it to mark test cases. I suspect the PGO task works well if it can exercise common code paths. Running long tests likely have rapidly diminishing benefits. The instrumented PGO executable

[issue36044] PROFILE_TASK for PGO build is not a good workload

2019-07-22 Thread Neil Schemenauer
Neil Schemenauer added the comment: New changeset 4e16a4a3112161a5c6981c0588142d4a4673a934 by Neil Schemenauer in branch 'master': bpo-36044: Reduce number of unit tests run for PGO build (GH-14702) https://github.com/python/cpython/commit/4e16a4a3112161a5c6981c0588142d4a4673a934

[issue36044] PROFILE_TASK for PGO build is not a good workload

2019-07-19 Thread Neil Schemenauer
Neil Schemenauer added the comment: I changed configure.in to use AC_ARG_VAR instead. So, you can override it as you suggest: ./configure [..] PROFILE_TASK="-m test --pgo-extended" -- ___ Python tracker <https://bugs.python.o

[issue36044] PROFILE_TASK for PGO build is not a good workload

2019-07-15 Thread Neil Schemenauer
Neil Schemenauer added the comment: Thanks for the feedback. I agree that putting the tests in regrtest is better. I've made the following changes: - the --pgo option now uses the shorter list of tests by default - I added --pgo-extended as a regrtest option that enables the old behavior

[issue36044] PROFILE_TASK for PGO build is not a good workload

2019-07-11 Thread Neil Schemenauer
Neil Schemenauer added the comment: +-+---+-+ | Benchmark | task-all2 | task-short7 | +=+===+=+ | 2to3| 304 ms| 305 ms

[issue36044] PROFILE_TASK for PGO build is not a good workload

2019-07-11 Thread Neil Schemenauer
Neil Schemenauer added the comment: I tweaked the list of unit tests a little more, trying to incorporate some from your Docker build settings. Not sure what's going on with the pickle results. Below are new pyperformance runs, comparing my PR to the "master" version it is ba

[issue36044] PROFILE_TASK for PGO build is not a good workload

2019-07-11 Thread Neil Schemenauer
Change by Neil Schemenauer : -- keywords: +patch pull_requests: +14501 stage: -> patch review pull_request: https://github.com/python/cpython/pull/14702 ___ Python tracker <https://bugs.python.org/issu

[issue36044] PROFILE_TASK for PGO build is not a good workload

2019-07-10 Thread Neil Schemenauer
Neil Schemenauer added the comment: > Decreasing the total wall time for a default --enable-optimizations build > would > be a good thing for everyone, provided the resulting interpreter remains > "effectively similar" in speed. If you somehow manage to find som

[issue37537] Compute allocated blocks in _Py_GetAllocatedBlocks()

2019-07-10 Thread Neil Schemenauer
Neil Schemenauer added the comment: New changeset 5d25f2b70351fc6a56ce5513ccf5f58556c18837 by Neil Schemenauer in branch 'master': bpo-37537: Compute allocated blocks in _Py_GetAllocatedBlocks() (#14680) https://github.com/python/cpython/commit/5d25f2b70351fc6a56ce5513ccf5f58556c18837

[issue37537] Compute allocated blocks in _Py_GetAllocatedBlocks()

2019-07-10 Thread Neil Schemenauer
Neil Schemenauer added the comment: PyPerformance result, using --enable-optimizations +-+-+--+ | Benchmark | base| no-allocated

[issue37537] Compute allocated blocks in _Py_GetAllocatedBlocks()

2019-07-10 Thread Neil Schemenauer
Change by Neil Schemenauer : -- keywords: +patch pull_requests: +14486 stage: -> patch review pull_request: https://github.com/python/cpython/pull/14680 ___ Python tracker <https://bugs.python.org/issu

[issue37537] Compute allocated blocks in _Py_GetAllocatedBlocks()

2019-07-10 Thread Neil Schemenauer
New submission from Neil Schemenauer : This is a small but measurable optimization for _PyObject_Malloc and _PyObject_Free. It slows down _Py_GetAllocatedBlocks a bit but I think that's a price worth paying. -- components: Interpreter Core messages: 347602 nosy: nascheme priority

[issue37448] obmalloc: radix tree for tracking arena address ranges

2019-06-29 Thread Neil Schemenauer
Change by Neil Schemenauer : -- keywords: +patch pull_requests: +14291 pull_request: https://github.com/python/cpython/pull/14474 ___ Python tracker <https://bugs.python.org/issue37

[issue37448] obmalloc: radix tree for tracking arena address ranges

2019-06-29 Thread Neil Schemenauer
Change by Neil Schemenauer : Added file: https://bugs.python.org/file48446/obmalloc_overhead.py ___ Python tracker <https://bugs.python.org/issue37448> ___ ___ Python-bug

[issue37448] obmalloc: radix tree for tracking arena address ranges

2019-06-29 Thread Neil Schemenauer
Change by Neil Schemenauer : Added file: https://bugs.python.org/file48445/pyperf_radix_compare.txt ___ Python tracker <https://bugs.python.org/issue37448> ___ ___ Pytho

[issue37448] obmalloc: radix tree for tracking arena address ranges

2019-06-29 Thread Neil Schemenauer
New submission from Neil Schemenauer : This patch implements an alternative version of obmalloc's address_in_range(). It uses a radix tree to map the areas of memory covered by obmalloc arenas. pymalloc_free() uses address_in_range() to determine if a block of memory is controlled

[issue37221] PyCode_New API change breaks backwards compatibility policy

2019-06-11 Thread Neil Schemenauer
Neil Schemenauer added the comment: I suggest we change PyCode_New() back in the next beta, despite that change breaking Cython again. We could work with them so that they have a new release with a "PY_VERSION_HEX > 0x030800b1" test in it. Try to get that Cython version re

[issue37200] PyType_GenericAlloc might over-allocate memory

2019-06-07 Thread Neil Schemenauer
Neil Schemenauer added the comment: Updated patch is attached. It appears that the extra item is only needed if Py_TPFLAGS_TYPE_SUBCLASS set. In all other cases, it seems we don't need the extra space for the sentinel. At least, the unit tests pass with this change. It could

[issue37200] PyType_GenericAlloc might over-allocate memory

2019-06-07 Thread Neil Schemenauer
New submission from Neil Schemenauer : In the process of working on some garbage collector/obmalloc experiments, I noticed what seems to be a quirk about PyType_GenericAlloc(). It calls: size = _PyObject_VAR_SIZE(type, nitems+1); Note the "+1" which is documented as "f

[issue27987] obmalloc's 8-byte alignment causes undefined behavior

2019-05-22 Thread Neil Schemenauer
Neil Schemenauer added the comment: We now have a concrete use case. ;-) My idea was that we can introduce a new, CPython internal API that aligns on 8-byte boundaries (or takes alignment as a parameter). The API would be a stop-gap measure. We can use the API to reduce the overhead

[issue27987] obmalloc's 8-byte alignment causes undefined behavior

2019-05-21 Thread Neil Schemenauer
Neil Schemenauer added the comment: > sys.getsizeof(3.14) is 24. And it becomes 32 byte in 16byte aligned > pymalloc. (+33%) I've been doing some reading and trying to understand this issue. My understanding is that malloc() needs to return pointers that are 16-byte aligned on

[issue36710] Pass _PyRuntimeState as an argument rather than using the _PyRuntime global variable

2019-05-02 Thread Neil Schemenauer
Neil Schemenauer added the comment: I think there are two questions to answer. First, do we want to support multiple runtimes per process? Second, if we do, what is the best way to do that? Some people would argue that multiple runtimes are not needed or are too hard to do. Maybe

[issue35810] Object Initialization does not incref Heap-allocated Types

2019-02-21 Thread Neil Schemenauer
Neil Schemenauer added the comment: Sorry, morning coffee didn't kick in yet I guess. ;-) My actual wish is to make all types heap allocated and eliminate the statically allocated ones. So, Py_TPFLAGS_HEAPTYPE would be set on all types in that world. That is a gigantic task, affecting

[issue35810] Object Initialization does not incref Heap-allocated Types

2019-02-21 Thread Neil Schemenauer
Neil Schemenauer added the comment: Hello Eddie, Thank you for putting what looks to be significant effort into this PR. It would be great if we can get this fixed. There is a real issue about breaking 3rd party extensions. So, we want to proceed with care. I wonder, if we are going

[issue36044] PROFILE_TASK for PGO build is not a good workload

2019-02-19 Thread Neil Schemenauer
New submission from Neil Schemenauer : I was doing some 'perf' profiling runs of Python. I decided to try running PROFILE_TASK to see what the profile looks like. I was surprised that gc_collect dominated runtime: Children Self Symbol + 93.93% 6.00

[issue36012] Investigate slow writes to class variables

2019-02-19 Thread Neil Schemenauer
Neil Schemenauer added the comment: BTW, 'perf report [...]' has a really neat annotated assembly view. Scroll to the function you are interested in and press 'a'. Press 't' to toggle the time units (left side numbers). I'm attaching a screenshot of the disassembly of the lookdict

[issue36012] Investigate slow writes to class variables

2019-02-19 Thread Neil Schemenauer
Neil Schemenauer added the comment: Some profiling using 'perf'. This is for cpython 63fa1cfece4912110ce3a0ff11fb3ade3ff5e756. children self [...] + 97.27% 0.00% run_mod (inlined) + 88.53% 6.33% PyObject_SetAttr + 79.34% 6.80% type_setattro + 43.92%43.92

[issue35813] shared memory construct to avoid need for serialization between processes

2019-02-17 Thread Neil Schemenauer
Neil Schemenauer added the comment: Some thoughts on this API. I think we need the "create with exclusive behavior" option, even though we don't know how to implement it on Windows right now. To me, there are two cases when calling SharedMemory: 1) You want to create a new sha

[issue35903] Build of posixshmem.c should probe for required OS functions

2019-02-08 Thread Neil Schemenauer
Change by Neil Schemenauer : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.or

[issue35903] Build of posixshmem.c should probe for required OS functions

2019-02-08 Thread Neil Schemenauer
Neil Schemenauer added the comment: New changeset 5741c45acf9b0ce22ff0dbf56322fe0ff16cfcfc by Neil Schemenauer in branch 'master': bpo-35903: Use autoconfig to probe for shm_open() and shm_unlink(). (#11765) https://github.com/python/cpython/commit/5741c45acf9b0ce22ff0dbf56322fe0ff16cfcfc

[issue35813] shared memory construct to avoid need for serialization between processes

2019-02-07 Thread Neil Schemenauer
Neil Schemenauer added the comment: I didn't finish reviewing completely yet but here are some comments. I think the random filename generation can be pulled out of _posixshmem. Make it require that a filename is passed into it. That moves a fair bit of complexity out of C and into Python

[issue35903] Build of posixshmem.c should probe for required OS functions

2019-02-06 Thread Neil Schemenauer
Neil Schemenauer added the comment: Hi Ronald, I tested with my MacBook but I'm not sure what version of the OS. Probably the latest but I can check when I get home. Maybe my comment was not clear. On my MacBook, you don't need librt for shm_open, etc. That sounds like it is the same

[issue35903] Build of posixshmem.c should probe for required OS functions

2019-02-05 Thread Neil Schemenauer
Neil Schemenauer added the comment: I didn't understand autoconfig well enough but I think I fixed it now. Apparently MacOS is a platform where it has shm_open but you don't need to link with -lrt. The revised 'configure' should handle that correctly now

[issue35903] Build of posixshmem.c should probe for required OS functions

2019-02-05 Thread Neil Schemenauer
Change by Neil Schemenauer : -- pull_requests: +11719, 11720 ___ Python tracker <https://bugs.python.org/issue35903> ___ ___ Python-bugs-list mailing list Unsub

[issue35903] Build of posixshmem.c should probe for required OS functions

2019-02-05 Thread Neil Schemenauer
Change by Neil Schemenauer : -- pull_requests: +11719, 11720, 11721 ___ Python tracker <https://bugs.python.org/issue35903> ___ ___ Python-bugs-list mailin

[issue35903] Build of posixshmem.c should probe for required OS functions

2019-02-05 Thread Neil Schemenauer
Change by Neil Schemenauer : -- pull_requests: +11719 ___ Python tracker <https://bugs.python.org/issue35903> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35903] Build of posixshmem.c should probe for required OS functions

2019-02-05 Thread Neil Schemenauer
New submission from Neil Schemenauer : The logic in setup.py that determines if _multiprocessing/posixshmem.c should get built is not very robust. I think it is better to use autoconfig to probe for the required functions and libraries. My autoconfig brain cells are a bit fuzzy but I think

[issue23903] Generate PC/python3.def by scraping headers

2019-01-23 Thread Neil Schemenauer
Change by Neil Schemenauer : -- nosy: +nascheme ___ Python tracker <https://bugs.python.org/issue23903> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22232] str.splitlines splitting on non-\r\n characters

2018-10-07 Thread Neil Schemenauer
Neil Schemenauer added the comment: I too would prefer a new method name rather than overloading splitlines() with more keyword args (passed as hardcoded constants, usually). Again, I think we want: list(open(..).read().()) == list(open(..)) readlines() returns a list but I think

[issue22232] str.splitlines splitting on non-\r\n characters

2018-10-05 Thread Neil Schemenauer
Neil Schemenauer added the comment: > Why not simply add a new parameter, to make people who want ASCII linebreaks > continue to use .splitlines() ? That could work but I think in nearly every case you don't want to use splitlines() without supplying the parameter. So, it seems like

[issue22232] str.splitlines splitting on non-\r\n characters

2018-10-05 Thread Neil Schemenauer
Neil Schemenauer added the comment: I've created a topic on this inside the "Ideas" area of discuss.python.org. Sorry if that wasn't appropriate, not sure if I should have keep the discussion here. Inada Naoki suggests creating a new method str.iterlines{[keepends]). Given t

[issue22232] str.splitlines splitting on non-\r\n characters

2018-10-05 Thread Neil Schemenauer
Neil Schemenauer added the comment: If we introduce a keyword parameter, I think the default of str.splitlines() should be changed to match bytes.splitlines (and match Python 2 str.splitlines()). I.e. split on \r and \n by default. I looked through the stdline and I can't find any calls

[issue18291] codecs.open interprets FS, RS, GS as line ends

2018-10-05 Thread Neil Schemenauer
Neil Schemenauer added the comment: I just found bug #22232 myself but thanks for pointing it out. > changing the behavior unconditionally is not an option At this point, I disagree. If I do a search on the web, lots of pages referring to str.splitlines() seem it imply that is splits o

[issue18291] codecs.open interprets FS, RS, GS as line ends

2018-10-04 Thread Neil Schemenauer
Neil Schemenauer added the comment: New patch that changes str.splitlines to work like Python 2 str.splitlines and like Python 3 bytes.splitlines. Surprisingly, only a few cases in the unit test suite fail. I've fixed them in my patch. -- Added file: https://bugs.python.org

[issue18291] codecs.open interprets FS, RS, GS as line ends

2018-10-04 Thread Neil Schemenauer
Neil Schemenauer added the comment: Some further progress on this. My patch slows down reading files with the codecs module very significantly. So, I think it could never be merged as is. Maybe we would need to implement an alternative str.splitlines that behaves as we want, implemented

[issue18291] codecs.open interprets FS, RS, GS as line ends

2018-10-04 Thread Neil Schemenauer
Neil Schemenauer added the comment: Attached is a rough patch that tries to fix this problem. I changed the behavior in that unicode char 0x2028 is no longer treated as a line separator. It would be trival to change the regex to support that too, if we want to preserve backwards

[issue18291] codecs.open interprets FS, RS, GS as line ends

2018-10-04 Thread Neil Schemenauer
Neil Schemenauer added the comment: I think one bug here is that codecs readers use str.splitlines() internally. The splitlines method treats a bunch of different characters as line separators, unlike io..readlines(). So, you end up with different behavior between doing iter

[issue30825] csv.Sniffer does not detect lineterminator

2018-10-04 Thread Neil Schemenauer
Neil Schemenauer added the comment: There is another issue related to this. If you use codecs to get a reader, it uses str.splitlines() internally, which treats a bunch of different characters as line terminators. See issue #18291 and: https://docs.python.org/3.8/library/stdtypes.html

[issue34801] codecs.getreader() splits lines containing control characters

2018-10-04 Thread Neil Schemenauer
Neil Schemenauer added the comment: Perhaps the 'csv' module should do some sanity checking on the file passed to the reader. The docs recommend that newline='' be used to open the file. Maybe 'csv' could check that and warn if its not the case. I poked around but it seems like io files

[issue34801] codecs.getreader() splits lines containing control characters

2018-10-04 Thread Neil Schemenauer
Neil Schemenauer added the comment: Thank you for the research. The problem is indeed that \v is getting treated as a line separator. That is an intentional design choice, see: https://bugs.python.org/issue12855 It would seem to have some surprising implications for CSV parsing. E.g

[issue34850] Emit a syntax warning for "is" with a literal

2018-10-04 Thread Neil Schemenauer
Neil Schemenauer added the comment: > The problem with a SyntaxWarning is that the wrong people will see it. It > gets in the way of users of applications that happen to be written in Python. Turn the check on only when PYTHONDEVMODE is set? Seems like it solves the issue with the

[issue34867] Add mode to disable small integer and interned string caches

2018-10-04 Thread Neil Schemenauer
Neil Schemenauer added the comment: Woudn't turning these off hurt performance a lot? If so, I don't know if people would actually use such a mode. Then it becomes pretty useless. Could we combine this idea with the PYTHONDEVMODE flag? If PYTHONDEVMODE is turned on, we could do a check

[issue34822] Simplify AST for slices

2018-10-03 Thread Neil Schemenauer
Neil Schemenauer added the comment: Hello Serhiy, I've not reviewed the patch but I trust that if you say it simplifies things, that's so. Perhaps the important question is if it is okay to change the AST in backwards incompatible ways within 3.x releases. As a library author who could

[issue34880] About the "assert" bytecode

2018-10-03 Thread Neil Schemenauer
Neil Schemenauer added the comment: Assuming it is not crazy complicated to fix, I would like to to be changed. It should work like the TypeError example. That fact that it has worked the current way since Python 1.5 isn't a strong argument. I think no one should be surprised

[issue34801] codecs.getreader() splits lines containing control characters

2018-09-25 Thread Neil Schemenauer
New submission from Neil Schemenauer : This seems to be a bug in codecs.getreader(). io.TextIOWrapper(fp, encoding) works correctly. -- files: codecs_bug.py messages: 326382 nosy: nascheme priority: low severity: normal status: open title: codecs.getreader() splits lines containing

[issue32892] Remove specific constant AST types in favor of ast.Constant

2018-09-24 Thread Neil Schemenauer
Neil Schemenauer added the comment: I've checked Quixote PTL module support. I will have to make some changes to support the removal of Str and other node types. E.g. I have to change visit_Str to visit_Constant. The fixes are pretty easy though and I think it is reasonable

[issue32892] Remove specific constant AST types in favor of ast.Constant

2018-09-21 Thread Neil Schemenauer
Neil Schemenauer added the comment: As someone who does AST manipulation (Quixote PTL template support), I'm interested in this patch. I think what is proposed sounds like a good change. I.e. having the extra node types is not useful and makes the compiler harder to understand. Providing

[issue34704] Do not access ob_type directly, introduce Py_TP

2018-09-16 Thread Neil Schemenauer
Change by Neil Schemenauer : -- pull_requests: +8767 ___ Python tracker <https://bugs.python.org/issue34704> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue34704] Do not access ob_type directly, introduce Py_TP

2018-09-16 Thread Neil Schemenauer
New submission from Neil Schemenauer : My long term goal is to make it possible to make PyObject an opaque structure. Obviously that would break nearly every 3rd party extension at this point. However, to allow freedom on certain interpreter implementation strategies, it is helpful if code

[issue26979] The danger of PyType_FromSpec()

2018-09-16 Thread Neil Schemenauer
Change by Neil Schemenauer : -- nosy: +nascheme ___ Python tracker <https://bugs.python.org/issue26979> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33608] [subinterpreters] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

2018-09-16 Thread Neil Schemenauer
Neil Schemenauer added the comment: I would suggest that sharing of objects between interpreters should be stamped out. Could we have some #ifdef debug checking that would warn or assert so this doesn't happen? I know currently we share a lot of objects. However, in the long term

[issue34690] Store startup modules as C structures for 20%+ startup speed improvement

2018-09-14 Thread Neil Schemenauer
Neil Schemenauer added the comment: I commented on python-dev but maybe it is better to keep discussion here. Could we make the frozenmodules thing into a dynamically loaded module? Then you could have support for end users making their own. E.g. a command-line param that lists a set

[issue31342] test.bisect module causes tests to fail

2018-07-16 Thread Neil Schemenauer
Neil Schemenauer added the comment: Yes, it looks like the same issue as bpo-29512. Renaming test.bisect is the simplest solution. I have trained myself to run "python -m test.regrtest " so this issue doesn't affect me any more. However, I think it was a trap that will catch s

[issue31342] test.bisect module causes tests to fail

2018-07-16 Thread Neil Schemenauer
Neil Schemenauer added the comment: This is still broken, IMHO. Either we should rename test.bisect or we should remove all of the 'if __name__ == "__main__"' part of the test scripts. You can't run the tests reliably as scripts anymore. Doing so puts Lib/test into

[issue28643] Broken makefile depends for profile-opt target

2018-07-16 Thread Neil Schemenauer
Neil Schemenauer added the comment: Closing this as the current makefile works much better in this regard, at least based on my experience. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracke

[issue34106] Add --with-module-config= to 'configure' script

2018-07-12 Thread Neil Schemenauer
Change by Neil Schemenauer : -- keywords: +patch pull_requests: +7804 ___ Python tracker <https://bugs.python.org/issue34106> ___ ___ Python-bugs-list mailin

[issue34106] Add --with-module-config= to 'configure' script

2018-07-12 Thread Neil Schemenauer
New submission from Neil Schemenauer : This patch is based on top of GH-8229 but could be changed to work without it. -- components: Build messages: 321580 nosy: nascheme priority: normal severity: normal stage: patch review status: open title: Add --with-module-config= to 'configure

[issue32430] Simplify Modules/Setup{,.dist,.local}

2018-07-12 Thread Neil Schemenauer
Neil Schemenauer added the comment: I removed the "makesetup_fallback_dist.txt". PR-8260 is a more polished implementation of the same idea. -- ___ Python tracker <https://bugs.python.o

[issue32430] Simplify Modules/Setup{,.dist,.local}

2018-07-12 Thread Neil Schemenauer
Change by Neil Schemenauer : Removed file: https://bugs.python.org/file47685/makesetup_fallback_dist.txt ___ Python tracker <https://bugs.python.org/issue32430> ___ ___

[issue32430] Simplify Modules/Setup{,.dist,.local}

2018-07-11 Thread Neil Schemenauer
Change by Neil Schemenauer : -- pull_requests: +7793 ___ Python tracker <https://bugs.python.org/issue32430> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue32430] Simplify Modules/Setup{,.dist,.local}

2018-07-11 Thread Neil Schemenauer
Change by Neil Schemenauer : Added file: https://bugs.python.org/file47685/makesetup_fallback_dist.txt ___ Python tracker <https://bugs.python.org/issue32430> ___ ___

[issue32430] Simplify Modules/Setup{,.dist,.local}

2018-07-11 Thread Neil Schemenauer
Neil Schemenauer added the comment: PR 8229 doesn't work if the build dir is separate from the src dir (mentioned in the PR comments). The attached patch (makesetup_fallback_dist.txt) changes makesetup to fallback to using $srcdir/Modules/Setup.dist if Modules/Setup doesn't exist. I

[issue34089] Remove required (non-optional) modules from Modules/Setup.dist

2018-07-10 Thread Neil Schemenauer
Change by Neil Schemenauer : -- keywords: +patch pull_requests: +7770 ___ Python tracker <https://bugs.python.org/issue34089> ___ ___ Python-bugs-list mailin

[issue34089] Remove required (non-optional) modules from Modules/Setup.dist

2018-07-10 Thread Neil Schemenauer
New submission from Neil Schemenauer : This is related to Issue32430. The behavior of Modules/Setup.dist and Modules/Setup is quirky and problematic. Issue32430 links to some of the related issues. The problem discussed in Issue24575 is also related (Py_BUILD_CORE not being defined when

[issue24575] timemodule build fail - missing definitions for _Py_BEGIN_SUPPRESS_IPH and _Py_END_SUPPRESS_IPH

2018-06-28 Thread Neil Schemenauer
Neil Schemenauer added the comment: I ran into an error like this and perhaps I have a clue was why some people run into it. Dynamic modules built from Modules/Setup will need to have -DPy_BUILD_CORE defined within the Setup file in the case they need to use the _Py_BEGIN_SUPPRESS_IPH

[issue33609] Document that dicts preserve insertion order

2018-06-07 Thread Neil Schemenauer
Neil Schemenauer added the comment: New changeset d3ed67d14ed401dfe2b5d07b6941adc3ecacb268 by Neil Schemenauer in branch 'master': bpo-33609: small wording fixes to dict ordering docs https://github.com/python/cpython/commit/d3ed67d14ed401dfe2b5d07b6941adc3ecacb268

[issue33609] Document that dicts preserve insertion order

2018-06-07 Thread Neil Schemenauer
Change by Neil Schemenauer : -- pull_requests: +7110 ___ Python tracker <https://bugs.python.org/issue33609> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33609] Document that dicts preserve insertion order

2018-06-04 Thread Neil Schemenauer
Neil Schemenauer added the comment: The wording sounds strange to me. Currently it is: > Note that updating key doesn't affects the order ... I would say: > Note that updating a key does not affect the order ... -- nosy: +nascheme ___

[issue33124] Lazy execution of module bytecode

2018-03-22 Thread Neil Schemenauer
Change by Neil Schemenauer <nas-pyt...@arctrix.com>: -- keywords: +patch pull_requests: +5941 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python

<    1   2   3   4   >