[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

Nick Coghlan added the comment: Yep, my current goal is to see if I can come up with a surgical fix that solves the established problem with the bad interaction between cells and trace functions without any unintended consequences for either CPython or other interpreters.

[issue31757] Tutorial: Fibonacci numbers start with 1, 1

Change by Raymond Hettinger : -- keywords: +patch pull_requests: +3967 stage: -> patch review ___ Python tracker ___

[issue31672] string.Template should use re.ASCII flag

Change by INADA Naoki : -- pull_requests: +3966 ___ Python tracker ___ ___

[issue31672] string.Template should use re.ASCII flag

INADA Naoki added the comment: New changeset 7060380d577690a40ebc201c0725076349e977cd by INADA Naoki in branch '3.6': bpo-31672: Fix string.Template accidentally matched non-ASCII identifiers (GH-3872)

[issue31757] Tutorial: Fibonacci numbers start with 1, 1

Raymond Hettinger added the comment: I update the example in "First Steps Towards Programming" to match the one in "Defining Functions" (which is also shown on the home page at www.python.org. -- ___ Python tracker

[issue31757] Tutorial: Fibonacci numbers start with 1, 1

Change by Raymond Hettinger : -- assignee: docs@python -> rhettinger nosy: +rhettinger versions: +Python 3.7 -Python 3.6 ___ Python tracker

[issue31778] ast.literal_eval supports non-literals in Python 3

Nick Coghlan added the comment: I'm marking this as documentation issue for now, as the operators that literal_eval allows are solely those where constant folding support is needed to correctly handle complex and negative numbers (as noted in the original post): ``` >>>

[issue31772] SourceLoader uses stale bytecode in case of equal mtime seconds

Nick Coghlan added the comment: Aye, I think that check would make the most sense, since the bytecode invalidation check is "_r_long(raw_timestamp) != source_mtime" (to allow for things like version control operations that send source timestamps backwards). A test for

[issue31742] Default to emitting FutureWarning for provisional APIs

Nick Coghlan added the comment: OK, I'll head down the path of creating a new procedural PEP to supersede PEP 411 (I'll try to get the locals() semantic clarification PEP out of the way first, though). I'll make "Where to put the feature flags?" an open question, as my

[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

Armin Rigo added the comment: Guido: you must be tired and forgot that locals() is a regular function :-) The compiler cannot recognize it reliably. Moreover, if f_locals can be modified outside a tracing hook, then we have the same problem in a cross-function

[issue31785] Move instruction code from ceval.c to a separate file

Change by pdox : -- title: Move instruction code blocks to separate file -> Move instruction code from ceval.c to a separate file ___ Python tracker

[issue31785] Move instruction code blocks to separate file

New submission from pdox : I'd like to move all instruction code (the code inside a TARGET(NAME) block) from Python/ceval.c to a new file, Python/instructions.h. The new file will contain only instruction bodies (prefixed by related helper functions/macros). Eval-context

[issue31622] Make threading.get_ident() return an opaque type

pdox added the comment: I don't see much enthusiasm or agreement here, so I'm closing for now. -- resolution: -> postponed stage: -> resolved status: open -> closed ___ Python tracker

[issue31780] Using format spec ',x' displays incorrect error message

Terry J. Reedy added the comment: The formatting part of PEP 515 was implemented in #27080. Chris Angelico's initial patch https://bugs.python.org/file44152/underscores_decimal_only.patch was, as the name says, for decimal only, and added "or '_' " to the error message.

[issue31778] ast.literal_eval supports non-literals in Python 3

Terry J. Reedy added the comment: It has been some time since literal_eval literally only evaluated literals. 'constant_eval' might be a better name now, with the proviso of 'safely, in reasonable time'. >>> from ast import literal_eval as le >>> le('(1,2,3)') (1, 2, 3)

[issue31784] Add time.time_ns(): get time with nanosecond resolution

Change by STINNER Victor : -- keywords: +patch pull_requests: +3965 stage: -> patch review ___ Python tracker ___

[issue31784] Add time.time_ns(): get time with nanosecond resolution

New submission from STINNER Victor : time.time() returns time as a float, but the conversion to float lose precision at the nanosecond resolution. I propose to add a new time.time_ns() function which returns time as an integer number of nanoseconds since epoch. It's

[issue31765] BUG: System deadlocks performing big loop operations in python 3.5.4, windows 10

Terry J. Reedy added the comment: Nikhil, I am closing this for now because I am at least 85% sure David is right. To get any help, you need to reduce your code to the minimum needed to produce the symptoms and then include that with any question. You would be told the

[issue31783] Race condition in ThreadPoolExecutor when scheduling new jobs while the interpreter shuts down

New submission from Steven Barker : While investigating a Stack Overflow question (here: https://stackoverflow.com/q/46529767/1405065), I found that there may be a race condition in the cleanup code for concurrent.futures.ThreadPoolIterator. The behavior in normal

[issue31757] Tutorial: Fibonacci numbers start with 1, 1

Terry J. Reedy added the comment: I agree that we should be consistent -- with the current standard definition -- with the changes suggested above. Heinrich, can you, and do you want to, submit a patch? If so, please also sign the contributor agreement. See

[issue14465] xml.etree.ElementTree: add feature to prettify XML output

STINNER Victor added the comment: For the record, at 2015-04-02, the bpo-23847 has been marked as a duplicate of this issue. -- nosy: +haypo ___ Python tracker

[issue31759] re wont recover nor fail on runaway regular expression

Matthew Barnett added the comment: @Tim: the regex module includes some extra checks to reduce the chance of excessive backtracking. In the case of the OP's example, they seem to be working. However, it's difficult to know when adding such checks will help, and

[issue31754] Documented type of parameter 'itemsize' to PyBuffer_FillContiguousStrides is incorrect.

Mariatta Wijaya added the comment: Welcome Aniket. Yes those two links are good starting points. Please propose a PR with the fix. -- nosy: +Mariatta ___ Python tracker

[issue31676] test.test_imp.ImportTests.test_load_source has side effects

Change by STINNER Victor : -- versions: +Python 3.6 ___ Python tracker ___ ___

[issue30807] setitimer() can disable timer by mistake

STINNER Victor added the comment: New changeset ef611c96eab0ab667ebb43fdf429b319f6d99890 by Victor Stinner in branch 'master': bpo-30807: signal.setitimer() now uses _PyTime API (GH-3865)

[issue31676] test.test_imp.ImportTests.test_load_source has side effects

Change by Roundup Robot : -- pull_requests: +3964 ___ Python tracker ___

[issue31676] test.test_imp.ImportTests.test_load_source has side effects

STINNER Victor added the comment: New changeset a505ecdc5013cd8f930aacc1ec4fb2afa62d3853 by Victor Stinner in branch 'master': bpo-31676: Fix test_imp.test_load_source() side effect (#3871)

[issue25588] Run test suite from IDLE idlelib.run subprocess

STINNER Victor added the comment: New changeset 6234e9068332f61f935cf13fa5b1a924a99c28b2 by Victor Stinner (Miss Islington (bot)) in branch '3.6': [3.6] bpo-25588: Fix regrtest when run inside IDLE (GH-3962) (#3987)

[issue31753] Unnecessary closure in ast.literal_eval

Terry J. Reedy added the comment: On win10, installed 3.7.0a1, speedup is 7-8% (It is 'only' 5% on repository debug build that takes 5-6 times longer.) -- nosy: +terry.reedy ___ Python tracker

[issue31754] Documented type of parameter 'itemsize' to PyBuffer_FillContiguousStrides is incorrect.

Aniket Vyas added the comment: Hello ! I am new to the community and would love to start my contribution here. Can I take up this bug ? In order to do so I am going through the following links: https://docs.python.org/devguide/docquality.html

[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

Change by STINNER Victor : -- nosy: -haypo ___ Python tracker ___ ___

[issue25588] Run test suite from IDLE idlelib.run subprocess

Change by Roundup Robot : -- pull_requests: +3963 ___ Python tracker ___

[issue25588] Run test suite from IDLE idlelib.run subprocess

STINNER Victor added the comment: New changeset ccef823939d4ef602f2d8d13d0bfec29eda597a5 by Victor Stinner in branch 'master': bpo-25588: Fix regrtest when run inside IDLE (#3962) https://github.com/python/cpython/commit/ccef823939d4ef602f2d8d13d0bfec29eda597a5

[issue31705] test_sha256 from test_socket fails on ppc64le arch

Ryan C. Decker added the comment: I seem to be having this issue on CentOS 7.4 but running on x86_64 instead of ppc64le. I have attached an strace using version 4.17 (the lastest version from scl) created as follows: strace -s 128 -e trace=%network -o trace ./python -m

[issue31782] Add a timeout to multiprocessing's Pool.join

Will Starms added the comment: A timeout alternative that raises TimeoutError -- Added file: https://bugs.python.org/file47220/cpython_raise_timeout.patch ___ Python tracker

[issue31752] Assertion failure in timedelta() in case of bad __divmod__

Terry J. Reedy added the comment: On Win10, installed 3.5.4, 3.6.3, 3.7.1a1 all raise SystemError. 3.6 and 3.7 repository debug builds raise AssertionError and Windows crash box. After the patch, a silent crash. -- nosy: +terry.reedy

[issue31781] crashes when calling methods of an uninitialized zipimport.zipimporter object

Change by Oren Milman : -- keywords: +patch pull_requests: +3962 stage: -> patch review ___ Python tracker ___

[issue31782] Add a timeout to multiprocessing's Pool.join

New submission from Will Starms : Pool's join function currently (3.6.3) lacks a timeout, which can cause the managing thread to sleep indefinitely when a pool worker hangs or starts misbehaving. Adding a timeout allows the owning thread to attempt a join and, after the

[issue31727] FTP_TLS errors when use certain subcommands

Change by Terry J. Reedy : -- stage: -> test needed title: FTP_TLS errors when -> FTP_TLS errors when use certain subcommands type: -> behavior versions: +Python 3.7 ___ Python tracker

[issue31772] SourceLoader uses stale bytecode in case of equal mtime seconds

Brett Cannon added the comment: To make the proposal concrete, would you then change https://github.com/python/cpython/blob/master/Lib/importlib/_bootstrap_external.py#L785 to include a `source_mtime != int(time.time())` guard? I think as long as that's the last check in

[issue31742] Default to emitting FutureWarning for provisional APIs

Aaron Gallagher <_...@habnab.it> added the comment: >Storing the marker attribute in __main__ [...] Can I request please not using __main__ for this? setuptools console_scripts are very common, which is a case where __main__ will be a generated (i.e. not user-controllable) file. Making

[issue31742] Default to emitting FutureWarning for provisional APIs

Guido van Rossum added the comment: I think this ought to be a new PEP which supersedes PEP 411. I think it should simply *offer* the option of using a feature flag rather than prescribing it. It could also explain that feature flags could cover an entire module or only the

[issue31781] crashes when calling methods of an uninitialized zipimport.zipimporter object

New submission from Oren Milman : The following code crashes: import zipimport zi = zipimport.zipimporter.__new__(zipimport.zipimporter) zi.find_module('foo') This is because get_module_info() (in Modules/zipimport.c) assumes that the zipimporter object is initialized, so it

[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

Guido van Rossum added the comment: Hm I may just be completely off here, but I thought that compilers could be allowed to recognize the use of locals() in a particular function and then disable JIT optimizations for that function. (In fact I thought we already had a

[issue28507] Regenerate ./configure on the default branch

Дилян Палаузов added the comment: For the record, on master runstatedir was added on 7th September 2017, removed on 5th September, added on 29 June, removed on 9th June, added on 14th April, removed on 6th December 2016 and 10th October in two branches, added on

[issue31780] Using format spec ',x' displays incorrect error message

Change by Ned Deily : -- nosy: +eric.smith ___ Python tracker ___ ___ Python-bugs-list

[issue30156] PYTHONDUMPREFS segfaults on exit

STINNER Victor added the comment: Note: if you care of namedtuple performance, Raymond Hettinger wrote that he would be interested to reuse the C structseq sequence. I measured that getting an attribute by name is faster in structseq than with the current property

[issue30156] PYTHONDUMPREFS segfaults on exit

STINNER Victor added the comment: > Removing this micro-optimization makes attribute access in namedtuple more > than 1.5 times slower: > Mean +- std dev: [python.default] 126 ns +- 4 ns -> [python] 200 ns +- 7 ns: > 1.58x slower (+58%) I wrote the PR 3985, it's

[issue30156] PYTHONDUMPREFS segfaults on exit

Change by STINNER Victor : -- keywords: +patch pull_requests: +3961 ___ Python tracker ___

[issue30767] logging must check exc_info correctly

Vinay Sajip added the comment: > I've triggered it which is why I looked for the problem and offered the > defensive patch. That's why I asked for a small example which used logging as documented and demonstrated a problem. You haven't done that. > As API writers

[issue31779] assertion failures and a crash when using an uninitialized struct.Struct object

Change by Oren Milman : -- keywords: +patch pull_requests: +3960 stage: -> patch review ___ Python tracker ___

[issue30767] logging must check exc_info correctly

Matthew Patton added the comment: I've triggered it which is why I looked for the problem and offered the defensive patch. As API writers you can NEVER assume your parameters are what you think they should be and just blindly proceed. --

[issue31509] test_subprocess hangs randomly on AMD64 Windows10 3.x

STINNER Victor added the comment: Ned: "I also have seen test_subprocess hangs on both macOS and on Debian Linux on both 3.6 and master, as recently as 3.6.3 and 3.7.0a1 but not with current heads. After some experimenting and bisecting, I tracked the fix down to

[issue31455] ElementTree.XMLParser() mishandles exceptions

Serhiy Storchaka added the comment: It would be nice. But I see you already have opened issue31758 for reference leaks. I think that other problems can be solved in the same issue. Do you mind to backport your patch to 2.7 Stefan? If this makes sense. Otherwise

[issue31732] Add TRACE level to the logging module

STINNER Victor added the comment: Vinay: > I feel that there is no need for a TRACE level in the stdlib Ok, that's fine. I just pushed the idea of someone on IRC. Since I had the same idea once, I tried, but I lost :-) I can easily survive without TRACE in the

[issue28647] python --help: -u is misdocumented as binary mode

Berker Peksag added the comment: Modules/main.c and Python.man is same in 3.6 branch. We could backport the change in Doc/library/sys.rst from 7f580970836b0f6bc9c5db868d95bea81a3e1558 but I didn't do it yet since it needs be manually backported. --

[issue22674] RFE: Add signal.strsignal(): string describing a signal

STINNER Victor added the comment: > 3. For the unknown signal, what is the description should be? "Unknown > signal" like c function returns or None? Hum, Linux returns "Unknown signal 12345". I propose to use this behaviour on all platforms (which provide

[issue28647] python --help: -u is misdocumented as binary mode

STINNER Victor added the comment: Thanks Berker for this nice documentation enhancement! It was required. Do we need to update Python 3.6 documentation using the commit 5f908005ce16b06d5af7b413264009c4b062f33c, or are we good? (sorry, I didn't check) --

[issue28647] python --help: -u is misdocumented as binary mode

Berker Peksag added the comment: Thank you for reviews, Serhiy and Victor. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue30404] Make stdout and stderr truly unbuffered when using -u option

Berker Peksag added the comment: New changeset 7f580970836b0f6bc9c5db868d95bea81a3e1558 by Berker Peksag in branch 'master': bpo-28647: Update -u documentation after bpo-30404 (GH-3961) https://github.com/python/cpython/commit/7f580970836b0f6bc9c5db868d95bea81a3e1558

[issue28647] python --help: -u is misdocumented as binary mode

Berker Peksag added the comment: New changeset 7f580970836b0f6bc9c5db868d95bea81a3e1558 by Berker Peksag in branch 'master': bpo-28647: Update -u documentation after bpo-30404 (GH-3961) https://github.com/python/cpython/commit/7f580970836b0f6bc9c5db868d95bea81a3e1558

[issue31780] Using format spec ',x' displays incorrect error message

New submission from FHTMitchell : Minor issue. Using the ',b', ',o' or ',x' raises the error ValueError("Cannot specify ',' or '_' with 'x'.",) (or equivalently for 'b' and 'o'). However, it is possible to use the format specs '_b', '_o' and '_x' in Python 3.6 due to PEP

[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

Nick Coghlan added the comment: I'll also note there's a simpler reason the namespace object exposed at the function level can't just be a write-through proxy for the underlying frame: references to frame.f_locals may outlive the frame backing it, at which point we really

[issue31779] assertion failures and a crash when using an uninitialized struct.Struct object

New submission from Oren Milman : The following code causes an assertion failure: import _struct struct_obj = _struct.Struct.__new__(_struct.Struct) struct_obj.iter_unpack(b'foo') This is because Struct_iter_unpack() (in Modules/_struct.c) assumes that

[issue31773] Rewrite _PyTime_GetWinPerfCounter() for _PyTime_t

STINNER Victor added the comment: I reopen the issue since I found a solution to only use integer in pytime.c for QueryPerformanceCounter() / QueryPerformanceFrequency() *and* prevent integer overflow. -- resolution: fixed -> status: closed -> open

[issue31773] Rewrite _PyTime_GetWinPerfCounter() for _PyTime_t

Change by STINNER Victor : -- pull_requests: +3959 ___ Python tracker ___ ___

[issue31761] regrtest: faulthandler.enable() fails with io.UnsupportedOperation: fileno when run from IDLE

Terry J. Reedy added the comment: After starting Python from a command line instead of an icon, importing/running autotest results in the same three failures. -- ___ Python tracker

[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

Armin Rigo added the comment: Thanks Nick for the clarification. Yes, that's what I meant: supporting such code in simple JITs is a nightmare. Perhaps more importantly, I am sure that if Python starts supporting random mutation of locals outside tracing hooks,

[issue31455] ElementTree.XMLParser() mishandles exceptions

Oren Milman added the comment: Serhiy, in addition to the problems you mentioned with not calling __init__(), it seems that calling every method of an uninitialized XMLParser object would crash. If you don't mind, i would be happy to open an issue to fix these crashes.

[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

Nick Coghlan added the comment: We're OK with the idea that installing a trace function might automatically turn off various compiler and interpreter managed optimisations (it's similar to how requesting or implying reliance on full frame support in other implementations

[issue31758] various refleaks in _elementtree

Oren Milman added the comment: Shame on me. I only now found out that Serhiy already mentioned most of the refleaks in https://bugs.python.org/issue31455#msg302103. -- ___ Python tracker

[issue30767] logging must check exc_info correctly

Vinay Sajip added the comment: Matthew Patton: you don't appear to have read the documentation correctly. The formatException() method's exc_info positional parameter is expected to be a normal exception tuple, not just any truthy value. This is clearly stated in the

[issue31772] SourceLoader uses stale bytecode in case of equal mtime seconds

Nick Coghlan added the comment: I wasn't clear on what you meant by "potentially in the future". Now that I realise you meant "Defer refreshing the bytecode cache to the next import attempt if `int(source_mtime) == int(time.time())`, but still bypass it for the current

[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

Nathaniel Smith added the comment: @arigo: But CPython is already committed to supporting writes to locals() at any moment, because at any moment you can set a trace function and in every proposal trace functions can reliably write to locals. So I don't think this is a

[issue31732] Add TRACE level to the logging module

Vinay Sajip added the comment: As Raymond has said: though it might appear reasonable to add a TRACE level from the numerous examples that Victor has given, in practice it is hard enough to know when a particular level should be applied. Victor says "we need to a 6th

[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

Armin Rigo added the comment: FWIW, a Psyco-level JIT compiler can support reads from locals() or f_locals, but writes are harder. The need to support writes would likely become another hard step on the way towards adding some simple JIT support to CPython in the

[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

Nathaniel Smith added the comment: I guess I should say that I'm still confused about why we're coming up with such elaborate schemes here, instead of declaring that f_locals and locals() shall return a dict proxy so that from the user's point of view, they Always Just Work

[issue31772] SourceLoader uses stale bytecode in case of equal mtime seconds

Devin Bayer added the comment: You can't demand a hot loader to react instantly and there are other use cases, like generating modules programatically. What is your objection to my proposed solution, which behaves correctly in all cases? If you are not importing modules

[issue31672] string.Template should use re.ASCII flag

Change by INADA Naoki : -- pull_requests: +3958 ___ Python tracker ___ ___

[issue31772] SourceLoader uses stale bytecode in case of equal mtime seconds

Nick Coghlan added the comment: If there's no hot reloader forcing a reimport for every saved edit, it's sufficiently unlikely to encounter this problem in the first place that I'm not worried about that scenario. (The time required for a human to context switch from code

[issue31672] string.Template should use re.ASCII flag

INADA Naoki added the comment: New changeset b22273ec5d1992b0cbe078b887427ae9977dfb78 by INADA Naoki in branch 'master': bpo-31672: Fix string.Template accidentally matched non-ASCII identifiers (GH-3872)

[issue31772] SourceLoader uses stale bytecode in case of equal mtime seconds

Devin Bayer added the comment: That wouldn't always work either. If the source file is imported, then edited, then not reimported until the next second (or far in the future) the stale bytecode would still be used. -- ___ Python

[issue31772] SourceLoader uses stale bytecode in case of equal mtime seconds

Nick Coghlan added the comment: The problem with changing the bytecode format is that code other than the import machinery reads the bytecode headers, so when we change the format, we need to consider the impact on that code. (Even my multiplication proposal above suffers

[issue31772] SourceLoader uses stale bytecode in case of equal mtime seconds

Devin Bayer added the comment: Thanks for the support Nick. I think your proposed idea would still result in rare but confusing behavior, which is the type of surprise Python should avoid. The hash-based pyc files doesn't seem like a solution to me, because it won't be