[issue43172] Fix test_readline when compiled using --with-readline=edit
Change by Gregory P. Smith : -- keywords: +patch pull_requests: +23288 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/24499 ___ Python tracker <https://bugs.python.org/issue43172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43172] Fix test_readline when compiled using --with-readline=edit
Gregory P. Smith added the comment: examining the behaviors being tested, it seems there are differences in 0 and 1 based indexing behaviors between libreadline and libedit. libedit frankly seems more consistent. but the Python readline module docs document the 0 and 1 based quirks as part of the module API. We may need to adjust for these based on direct libedit use vs libreadline within the module code. it appears code doing such adjustments already exists on macOS (i haven't yet looked in the module code or if Darwin as a platform does it in their readline shim around libedit). -- ___ Python tracker <https://bugs.python.org/issue43172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43172] Fix test_readline when compiled using --with-readline=edit
Gregory P. Smith added the comment: https://github.com/python/buildmaster-config/pull/229 tracks my buildbot config update -- ___ Python tracker <https://bugs.python.org/issue43172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13501] Make libedit support more generic; port readline / libedit to FreeBSD
Gregory P. Smith added the comment:
That seems like a good idea to prevent regressions if anyone knows how to do
that.
For python.org/dev/'s buildbot fleet, the configuration of what configure and
make flags happens via
https://github.com/python/buildmaster-config/tree/master/master/custom - note
the 'def setup' overriding that needs to be done in factories.py so that older
branches {'3.7', '3.8', '3.9'} are not configured using the flag.
Coordinate with the owner of a particular buildbot to make sure the system has
the requisite installed libraries and headers.
I manually tested a Linux build against libedit, it builds and works. But
test_readline has failures. tracking those in
https://bugs.python.org/issue43172
--
___
Python tracker
<https://bugs.python.org/issue13501>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43172] Fix test_readline when compiled using --with-readline=edit
Gregory P. Smith added the comment: Motivation: I want to add --with-readline=edit to the configure flags of one of my buildbot configs via an edit to https://github.com/python/buildmaster-config/tree/master/master/custom -- ___ Python tracker <https://bugs.python.org/issue43172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43172] Fix test_readline when compiled using --with-readline=edit
New submission from Gregory P. Smith : https://bugs.python.org/issue13501 added configure --with-readline=edit support so that the readline module can build and link against libedit on any platform instead of only using libreadline. Building Python that way on Debian Linux and running test_readline results in a variety of test failure for me: ``` == CPython 3.10.0a5+ (heads/master-dirty:e1f7769513, Feb 9 2021, 01:20:59) [GCC 10.2.1 20210110] == cwd: /home/gps/oss/cpython/b/build/test_python_1681021æ == CPU count: 16 == encodings: locale=UTF-8, FS=utf-8 0:00:00 load avg: 0.47 Run tests sequentially 0:00:00 load avg: 0.47 [1/1] test_readline readline version: 0x402 readline runtime version: 0x402 readline library version: 'EditLine wrapper' use libedit emulation? True testHistoryUpdates (test.test_readline.TestHistoryManipulation) ... ERROR test_nonascii_history (test.test_readline.TestHistoryManipulation) ... FAIL test_write_read_append (test.test_readline.TestHistoryManipulation) ... FAIL test_auto_history_disabled (test.test_readline.TestReadline) ... ok test_auto_history_enabled (test.test_readline.TestReadline) ... ok test_history_size (test.test_readline.TestReadline) ... skipped 'this readline version does not support history-size' test_init (test.test_readline.TestReadline) ... ok test_nonascii (test.test_readline.TestReadline) ... FAIL == ERROR: testHistoryUpdates (test.test_readline.TestHistoryManipulation) -- Traceback (most recent call last): File "/home/gps/oss/cpython/gpshead/Lib/test/test_readline.py", line 59, in testHistoryUpdates readline.replace_history_item(0, "replaced line") ValueError: No history item at position 0 == FAIL: test_nonascii_history (test.test_readline.TestHistoryManipulation) -- Traceback (most recent call last): File "/home/gps/oss/cpython/gpshead/Lib/test/test_readline.py", line 127, in test_nonascii_history self.assertEqual(readline.get_history_item(1), "entrée 1") AssertionError: 'entrée 22' != 'entrée 1' - entrée 22 ?^^ + entrée 1 ?^ == FAIL: test_write_read_append (test.test_readline.TestHistoryManipulation) -- Traceback (most recent call last): File "/home/gps/oss/cpython/gpshead/Lib/test/test_readline.py", line 98, in test_write_read_append self.assertEqual(readline.get_current_history_length(), 3) AssertionError: 4 != 3 == FAIL: test_nonascii (test.test_readline.TestReadline) -- Traceback (most recent call last): File "/home/gps/oss/cpython/gpshead/Lib/test/test_readline.py", line 231, in test_nonascii self.assertIn(b"indexes 11 13\r\n", output) AssertionError: b'indexes 11 13\r\n' not found in bytearray(b"^A^B^B^B^B^B^B^B\t\tx\t\r\n[\xc3\xafnserted]|t\xc3\xab[after]\x08\x08\x08\x08\x08\x08\x08text \'t\\xeb\'\r\nline \'[\\xefnserted]|t\\xeb[after]\'\r\nindexes 10 12\r\n\x07\r\x1b[14Gtext \'t\\xeb\'\r\nline \'[\\xefnserted]|t\\xeb[after]\'\r\nindexes 10 12\r\n\r\nt\xc3\xabnt t\xc3\xabxt\r\n\r\x1b[K[\xc3\xafnserted]|t\xc3\xab[after]\x1b[14G\r\x1b[14G\x1b[1@x[\x08\x07\r\x1b[15G\x1b[1@t[\x08\r\nresult \'[\\xefnserted]|t\\xebxt[after]\'\r\nhistory \'[\\xefnserted]|t\\xebxt[after]\'\r\n") -- Ran 8 tests in 0.123s FAILED (failures=3, errors=1, skipped=1) ``` I suspect these are just behavior differences in the library and some tests need to be skipped or test alternate behavior in this situation? -- assignee: gregory.p.smith components: Library (Lib), Tests messages: 386681 nosy: gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: Fix test_readline when compiled using --with-readline=edit type: behavior versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13501] Make libedit support more generic; port readline / libedit to FreeBSD
Gregory P. Smith added the comment: I'm closing this as I believe everything we need done is done at this point. Open new issues if there are remaining libedit vs libreadline things to take care of. Thanks everyone! -- resolution: -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker <https://bugs.python.org/issue13501> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38634] Symbol resolution conflict when embeding python in an application using libedit
Gregory P. Smith added the comment: New changeset e1f77695132e814728cda982f11342a2e3c7272c by Roland Hieber in branch 'master': bpo-13501: allow choosing between readline and libedit (GH-24189) https://github.com/python/cpython/commit/e1f77695132e814728cda982f11342a2e3c7272c -- ___ Python tracker <https://bugs.python.org/issue38634> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13501] Make libedit support more generic; port readline / libedit to FreeBSD
Gregory P. Smith added the comment: New changeset e1f77695132e814728cda982f11342a2e3c7272c by Roland Hieber in branch 'master': bpo-13501: allow choosing between readline and libedit (GH-24189) https://github.com/python/cpython/commit/e1f77695132e814728cda982f11342a2e3c7272c -- ___ Python tracker <https://bugs.python.org/issue13501> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43113] os.posix_spawn errors with wrong information when shebang points to not-existing file
Gregory P. Smith added the comment: That bash produces a nicer error message is because bash happens to implement its own special logic to try and figure out why an exec failed with an error other than ENOEXEC. The OS kernel & libc do not give it that information, there is no such errno. Bash inspects the executable itself for a #! in some error paths, extracts an interpreter from that and constructs an error message involving it. (bash's execute_cmd.c and HAVE_HASH_BANG_EXEC logic) I agree with Eric. I don't think we should do anything here. This isn't posix_spawn, subprocess, or os.exec* specific. It's just how posix-y OSes that have the concept of a #! line interpreter for executable files work. The errno that comes back from an exec failure is not super informative. If someone disagrees, the way to move forward on this is to implement equivalent logic in a central place and have it called from all of the relevant places within the posixmodule (os) and the _posixsubprocess module. With tests of all possible errno cases and code paths for each. And make a PR out of that. If you're going to take that on; do _not_ look at the bash source code. That's GPL, we cannot copy it. Just go by this description. To me, this seems over complicated to get right and maintain. I'd rather not do this within Python. But if someone is going to make a PR for it, I'll at least look at it to see if it seems like something we could accept maintenance of. I cannot guarantee we'd accept it. -- resolution: -> not a bug stage: -> needs patch status: open -> closed type: behavior -> enhancement ___ Python tracker <https://bugs.python.org/issue43113> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36379] nb_inplace_pow is always called with an invalid argument
Gregory P. Smith added the comment: I'm pretty sure this is fixed for 3.8+. whether or not it should be considered a bugfix and backported to 3.7.x is probably too late at this point in release lifecycles anyways. thanks for raising this and the fixing PR! -- nosy: +gregory.p.smith resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue36379> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42987] HTTP header injection in urllib on windows
Gregory P. Smith added the comment:
Have you tried this on a more recent Python? works for me on 3.7.8 on macos.
Python 3.7.8 (v3.7.8:4b47a5b6ba, Jun 27 2020, 04:47:50)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.request import urlopen
>>> remote_image = urlopen('http://127.0.0.1:6379/\r\nset ce test\r\n/1.jpg')
Traceback (most recent call last):
File "", line 1, in
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py",
line 222, in urlopen
return opener.open(url, data, timeout)
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py",
line 525, in open
response = self._open(req, data)
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py",
line 543, in _open
'_open', req)
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py",
line 503, in _call_chain
result = func(*args)
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py",
line 1378, in http_open
return self.do_open(http.client.HTTPConnection, req)
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py",
line 1350, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py",
line 1262, in request
self._send_request(method, url, body, headers, encode_chunked)
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py",
line 1273, in _send_request
self.putrequest(method, url, **skips)
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py",
line 1116, in putrequest
self._validate_path(url)
File
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py",
line 1207, in _validate_path
raise InvalidURL(f"URL can't contain control characters. {url!r} "
http.client.InvalidURL: URL can't contain control characters. '/\r\nset ce
test\r\n/1.jpg' (found at least '\r')
If this is somehow Windows specific (that'd be surprising), I don't have
windows and someone else will need to confirm.
--
nosy: +gregory.p.smith
___
Python tracker
<https://bugs.python.org/issue42987>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42969] pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful
Change by Gregory P. Smith : -- title: pthread_exit & PyThread_exit_thread are harmful -> pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful ___ Python tracker <https://bugs.python.org/issue42969> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42969] pthread_exit & PyThread_exit_thread are harmful
Gregory P. Smith added the comment: C-APIs such as `PyEval_RestoreThreads()` are insufficient for the task they are asked to do. They return void, yet have a failure mode. They call pthread_exit() on failure today. Instead, they need to return an error to the calling application to indicate that "The Python runtime is no longer available." Callers need to act on that in whatever way is most appropriate to them. -- ___ Python tracker <https://bugs.python.org/issue42969> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42969] pthread_exit & PyThread_exit_thread are harmful
New submission from Gregory P. Smith : ## BACKGROUND `PyThread_exit_thread()` calls `pthread_exit(`) and is in turn called from a variety of APIs as documented in the C-API doc update from https://bugs.python.org/issue36427. The `pthread_exit()` call was originally introduced as a way "resolve" a crashes from daemon threads during shutdown https://bugs.python.org/issue1856. It did that. That fix to that even landed in 2.7.8 but was rolled back before 2.7.9 due to a bug in an existing application it exposed at the time (did we miss the implications of that? maybe). It remained in 3.x. ## PROBLEM `pthread_exit()` cannot be used blindly by any application. All code in the threaded application needs to be on board with it and always prepared for any API call they make to potentially lead to thread termination. Quoting a colleague: "pthread_exit() does not result in stack unwind or local variable destruction". This means that any code up the stack from the ultimate `pthread_exit()` call that has process-wide state implications that did not go out of its way to register cleanup with `pthread_cleanup_push()` could lead to deadlocks or lost resources. Something implausible to assume that code does. We're seeing this happen with C/C++ code. Our C++ builds with `-fno-exceptions` so uncatchable exception based stack unwinding as some pthread_exit implementations may trigger does not happen (and cannot be guaranteed anyways, see bpo-42888). Said C/C++ code is calling back into Python from a thread and thus must use `PyEval_RestoreThread()` or similar APIs before performing Python C API calls. If the interpreter is being finalized from another thread... these enter a codepath that ultimately calls `pthread_exit()` leaving corrupt state in the process. In this case that unexpected thread simply disappearing can lead to a deadlock in our process. Fundamentally I do not believe the CPython VM should ever call `pthread_exit()` when non-CPython frames are anywhere in the C stack. This may mean we should never call `pthread_exit()` at all (unsure; but it'd be ideal). The documentation suggests that all callers in user code of the four C-APIs with the documented `pthread_exit()` caveats need auditing and pre-call `_Py_IsFinalizing()` API checks. But... I do not believe that would fix anything even if it were done. `_Py_IsFinalizing()` called without the GIL held means that it could change by the time the `PyEval_RestoreThreads()` API calls it internally do determine if it should exit the thread. Thus the race condition window would merely be narrowed, not eliminated. Not good enough. ## CURRENT WORKAROUND (Big Hammer) Change CPython to call abort() instead of pthread_exit() as that situation is unresolvable and the process dying is better than hanging, partially alive. That solution isn't friendly, but is better than being silent and allowing deadlock. A failing process is always better than a hung process, especially a partially hung process. ## SEMI RELATED WORK https://bugs.python.org/issue42888 - appears to be avoiding some `PyThread_exit_thread()` calls to stop some crashes due to `libgcc_s` being loaded on demand upon thread exit. -- components: Interpreter Core keywords: 3.2regression messages: 385288 nosy: gregory.p.smith priority: normal severity: normal status: open title: pthread_exit & PyThread_exit_thread are harmful type: behavior versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue42969> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42942] Feature request: Add decdigest() to hashlib
Gregory P. Smith added the comment: Agreed, using a dict or set hash table lookup is more appropriate for such an algorithm. Also agreed: comparing python integers (30-bit digit bignums internally) cannot be faster than comparing a binary bytes object. -- ___ Python tracker <https://bugs.python.org/issue42942> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42899] Is it legal to eliminate tests of a value, when that test has no effect on control flow?
Gregory P. Smith added the comment: If the body of a conditional does nothing, it seems fine to optimize the condition out to me. But I see code from a low level compiled language perspective where that is clearly what would happen. In reality, who ever meaningfully writes code where the body of a conditional does nothing? * Placeholder code with a # TODO perhaps. [fine to optimize out] * Unit tests attempting to test the behavior of __bool__(). [an annoying behavior change] Are there others? Are we expecting this odd "not quite a no-op because we're so high level" pattern to ever appear in a performance critical situation? The workaround for the latter would be to explicitly `if bool(x):` instead of `if x:` when the body is a no-op. Or not make the body a no-op. I expect unittest of __bool__() code owners would be fine with that so long as we call it out clearly in What's New docs, it's just that it could be an annoying change for them to make. Ideally we'd also provide a lib2to3 fixer to detect and fixup code exhibiting that pattern. The easiest answer is just not to optimize this out if it isn't actually providing us anything deemed important. -- ___ Python tracker <https://bugs.python.org/issue42899> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42899] Possible regression introduced by bpo-42615
Change by Gregory P. Smith : -- priority: normal -> high ___ Python tracker <https://bugs.python.org/issue42899> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42899] Possible regression introduced by bpo-42615
Change by Gregory P. Smith : -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue42899> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7946] Convoy effect with I/O bound threads and New GIL
Change by Gregory P. Smith : -- resolution: -> wont fix ___ Python tracker <https://bugs.python.org/issue7946> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42736] Add support for making Linux prctl(...) calls to subprocess
Gregory P. Smith added the comment: Felt and understood. The plethora of things to do between (v)fork+exec really makes me wish for a "little" eBPF interpreter rather than needing so much specific plumbing. But that'd have the same problem as preexec_fn: in absence of something that declares an operation safe or not, it'd open the door to unsafe and leave us in no better state than preexec_fn. -- ___ Python tracker <https://bugs.python.org/issue42736> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42736] Add support for making Linux prctl(...) calls to subprocess
Gregory P. Smith added the comment: Thanks for the golang SysProcAttr reference. The internals of _posixsubprocess have already becoming unwieldy with the abundance of args. Such a struct/object would also fit in well there and avoid excessive C stack use. (as izbyshev noted during the vfork work) Most prctl uses I noticed were PDEATHSIG but I'd need to explicitly audit those. Users don't seem to care about it's documented main thread caveat (which matches what I've seen; most programs don't use non-daemon threads and exit the main thread). I want what we do for this to be futureproof for the syscall so that we don't wind up merely picking one feature such as PDEATHSIG to pass a flags through to and needing to add logic to support others later on, delaying the ability to use new system features. -- ___ Python tracker <https://bugs.python.org/issue42736> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38435] Start the deprecation cycle for subprocess preexec_fn
Gregory P. Smith added the comment: > "using Python is more portable than relying on a shell." Not in environments I use. :) There isn't an installed python interpreter that can be executed when deploying Python as an embedded interpreter such as anyone using pyoxidizer or similar. Plus "using python" means adding a Python startup time delay to anything that triggered such an action. That added latency isn't acceptable in some situations. When I suggest a workaround for something as involving an intermediate shell script, read that to mean "the user needs an intermediate program to do this complicated work for them - how is up to them - we aren't going to provide it from the stdlib". A shell script is merely one easy pretty-fast solution - in environments where that is possible. TL;DR - there's no one size fits all solution here. But third party libraries could indeed implement any/all of these options including abstracting how and what gets used when if someone wanted to do that. -- ___ Python tracker <https://bugs.python.org/issue38435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42738] subprocess: don't close all file descriptors by default (close_fds=False)
Gregory P. Smith added the comment: Note that vfork() support has been merged for 3.10 via bpo-35823, so posix_spawn() is less of a performance carrot than it used to be on Linux. vfork() exists macOS, that code could likely be enabled there after some investigation+testing. Regardless, changing this default sounds difficult due to the variety of things depending on the existing behavior - potentially for security issues as you've noted - when running in a process with other file descriptors potentially not managed by Python (ie: extension modules) that don't explicitly use CLOEXEC. The subprocess APIs are effectively evolving to become lower level over time as we continually find warts in them that need addressing but find defaults that cannot change due to existing uses. A higher level "best practices for launching child processes module" with APIs reflecting explicit intents (performance vs security vs simplicity) rather than requiring users to understand subprocess platform specific details may be a good idea at this point (on PyPI I assume). We changed posix close_fds default to True in 3.2 when Jeffrey and I wrote _posixsubprocess to better match the behavior most users actually want - undoing that doesn't feel right. -- type: -> performance ___ Python tracker <https://bugs.python.org/issue42738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38435] Start the deprecation cycle for subprocess preexec_fn
Gregory P. Smith added the comment: Doing the code inspection of existing preexec_fn= users within our codebase at work is revealing. But those seem to be the bulk of uses. I expect this deprecation to take a while. Ex: if we mark it as PendingDeprecationWarning in 3.10, I'd still wait until _at least_ 3.13 to remove it. Code using it often has a long legacy and may be written to run on a wide variety of Python versions. It's painful to do so when features you need in order to stop using it are still only in modern versions. -- components: +Library (Lib) ___ Python tracker <https://bugs.python.org/issue38435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38435] Start the deprecation cycle for subprocess preexec_fn
Change by Gregory P. Smith : -- pull_requests: +22786 pull_request: https://github.com/python/cpython/pull/23936 ___ Python tracker <https://bugs.python.org/issue38435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38435] Start the deprecation cycle for subprocess preexec_fn
Gregory P. Smith added the comment: signal.signal use case: Calls to signal.signal(x, y) to sometimes to set a non SIG_DFL behavior on a signal. SIGINT -> SIG_IGN for example. I see a lot of legacy looking code calling signal.signal in prexec_fn that appears to set SIG_DFL for the signals that Python otherwise modifies. Which restore_signals=True should already be doing. -- ___ Python tracker <https://bugs.python.org/issue38435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38435] Start the deprecation cycle for subprocess preexec_fn
Gregory P. Smith added the comment: I'm also seeing a lot of os.setpgrp() calls, though those are more likely able to use start_new_session to do setsid() as a dropin replacement. -- ___ Python tracker <https://bugs.python.org/issue38435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38435] Start the deprecation cycle for subprocess preexec_fn
Gregory P. Smith added the comment: Another preexec_fn use to consider: resource.setrlimit(resource.RLIMIT_CORE, (XXX, XXX)) Using an intermediate shell script wrapper that changes the rlimit and exec's the actual process is also an alternative. -- ___ Python tracker <https://bugs.python.org/issue38435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38435] Start the deprecation cycle for subprocess preexec_fn
Gregory P. Smith added the comment: https://bugs.python.org/issue42736 filed to track adding Linux prctl() support. -- ___ Python tracker <https://bugs.python.org/issue38435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42736] Add support for making Linux prctl(...) calls to subprocess
New submission from Gregory P. Smith :
Another use of `subprocess preexec_fn=` that I've come across has been to call
Linux's `prctl` in the child process before the `exec`.
`_libc.prctl(_PR_SET_PDEATHSIG, value)` for example.
Adding a linux_prctl_calls= parameter listing information about which prctl
call(s) to make in the child before exec would satisfy this needed.
No need to go overboard here, this is a _very_ low level syscall. users need
to know what they're doing and will likely abstract use away from actual users
via a wrapper library. i.e: Lets not attempt to reinvent the
https://pythonhosted.org/python-prctl/ interface.
Proposal:
linux_prctl_calls: Sequence[tuple]
Where every tuple indicates one prctl call. the tuple [0] contains a bool
indicating if an error returned by that prctl call should be ignored or not.
the tuple[1:6] contain the five int arguments to be passed to prctl. If the
tuple is shorter than 2 elements, the remaining values are assumed 0.
At most, a namedtuple type could be created for this purpose to allow the user
to use the https://man7.org/linux/man-pages/man2/prctl.2.html argument names
rather than just blindly listing a tuple.
```
namedtuple('LinuxPrctlDescription',
field_names='check_error, option, arg2, arg3, arg4, arg5',
defaults=(0,0,0,0))
```
This existing helps https://bugs.python.org/issue38435 deprecate preexec_fn.
--
components: Extension Modules, Library (Lib)
messages: 383723
nosy: gregory.p.smith
priority: normal
severity: normal
stage: needs patch
status: open
title: Add support for making Linux prctl(...) calls to subprocess
type: enhancement
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue42736>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38435] Start the deprecation cycle for subprocess preexec_fn
Gregory P. Smith added the comment: PR up to add setpgid support. From what I've come across, some setpgid() users can use setsid() already available via start_new_session= instead. But rather than getting into the differences between those, making both available makes sense to allow for anyone's case where setsid() isn't desired. -- ___ Python tracker <https://bugs.python.org/issue38435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42388] subprocess.check_output(['echo', 'test'], text=True, input=None) fails
Gregory P. Smith added the comment: Meta issue behind this one: The input= behavior on check_output is yet another unfortunate wart in the subprocess collection of APIs. PR to the main branch is in, 3.9 and 3.8 will automerge after CI runs. -- resolution: -> fixed stage: patch review -> commit review ___ Python tracker <https://bugs.python.org/issue42388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42388] subprocess.check_output(['echo', 'test'], text=True, input=None) fails
Gregory P. Smith added the comment: New changeset 64abf373444944a240274a9b6d66d1cb01ecfcdd by Gregory P. Smith in branch 'master': bpo-42388: Fix subprocess.check_output input=None when text=True (GH-23467) https://github.com/python/cpython/commit/64abf373444944a240274a9b6d66d1cb01ecfcdd -- ___ Python tracker <https://bugs.python.org/issue42388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38435] Start the deprecation cycle for subprocess preexec_fn
Change by Gregory P. Smith : -- versions: +Python 3.10 -Python 3.9 ___ Python tracker <https://bugs.python.org/issue38435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42727] [Enum] EnumMeta.__prepare__ needs to accept **kwds
Gregory P. Smith added the comment: New changeset 8badadec53cbf9dc049c5b54198c5689481e3f3f by Gregory P. Smith in branch 'master': bpo-42727: Fix the NEWS entry .rst (GH-23932) https://github.com/python/cpython/commit/8badadec53cbf9dc049c5b54198c5689481e3f3f -- ___ Python tracker <https://bugs.python.org/issue42727> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42727] [Enum] EnumMeta.__prepare__ needs to accept **kwds
Change by Gregory P. Smith : -- nosy: +gregory.p.smith nosy_count: 2.0 -> 3.0 pull_requests: +22782 pull_request: https://github.com/python/cpython/pull/23932 ___ Python tracker <https://bugs.python.org/issue42727> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38435] Start the deprecation cycle for subprocess preexec_fn
Change by Gregory P. Smith : -- keywords: +patch pull_requests: +22780 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/23930 ___ Python tracker <https://bugs.python.org/issue38435> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42648] subprocess: add a helper/parameter to catch exec() OSError exception
Gregory P. Smith added the comment: arguably run() with check=True could've turned the OSError into a CalledProcessError [can't do that now, someone surely depends on it]. So if both are supplied, perhaps it does that instead of returning a CompletedProcess with .oserror set? the API surface we've accumulated in subprocess over the decades is huge. :( -- ___ Python tracker <https://bugs.python.org/issue42648> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42648] subprocess: add a helper/parameter to catch exec() OSError exception
Gregory P. Smith added the comment: We could also be thinking too low level here. We don't have to re-use returncode for this or do it on POpen itself. This could be done at the `run()` API level and CompletedProcess could be given state indicating success or failure of the exec itself. ex: Add a `capture_oserror=` arg to `run()`. ``` >>> proc = subprocess.run(['foo'], capture_oserror=True) >>> proc CompletedProcess(args='foo', oserror=FileNotFoundError(2, 'No such file or directory')) >>> if proc.failure: ... ``` Add two properties to CompletedProcess: ``` @property def success(self): return bool(self.oserror is None and not self.returncode) @property def failure(self): return bool(self.oserror or self.returncode) ``` to make using that an easy one liner. Another thing that came up recently was someone wishing CompletedProcess had a `__bool__()` method. I rejected that as it could break existing code already testing None vs CompletedProcess via truthiness. But such a desire should be satisfied with the above .success and .failure properties. -- ___ Python tracker <https://bugs.python.org/issue42648> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42648] subprocess: add a helper/parameter to catch exec() OSError exception
Gregory P. Smith added the comment: I suggest just adding a couple options to getstatusoutput instead of wrangling new to-catch-or-not-to-catch APIs if code like your test_posix example is what you want to replace. -- ___ Python tracker <https://bugs.python.org/issue42648> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42648] subprocess: add a helper/parameter to catch exec() OSError exception
Gregory P. Smith added the comment:
The shell was isolating the user from the exec error by always existing and
turning the exec error into a status return.
>>> subprocess.run('foo', shell=True)
/bin/sh: line 1: foo: command not found
CompletedProcess(args='foo', returncode=127)
>>> subprocess.getstatusoutput('foo')
(127, '/bin/sh: line 1: foo: command not found')
For things that don't actually _need_ the output as a pipe file (such as your
.read() example), I recommend using .run() like that today and accessing the
.stdout and .returncode attributes of the CompletedProcess. Or use the old
getstatusoutput() API if you don't mind stderr being included.
As far as new APIs go, yet another keyword only argument to subprocess.POpen's
existing huge list that tells it to turn exec failures into a particular
returncode value could be added as a feature. Is this use case compelling
enough to do that?
...
As far as distinguishing failures go, I suggest making such an API be to allow
the user to specify a non-negative int to use as returncode - a unique int or
one that is out of range such as a negative number or large number could be
used to distinguish things if the user were so inclined or even an int subclass
like True or an IntEnum value. But lets keep it a non-negative int, not an
arbitrary object. Negative ints map to signals; using those would be confusing
for CalledProcessError when a check constructs one.
```
>>> subprocess.run('foo', exec_fails_via_returncode=256)
CompletedProcess(args='foo', returncode=256)
```
With a default of exec_fails_via_returncode=None being the existing exception
raising behavior.
That isn't joyful to use as an API. So I don't expect most people would use it
directly. They'd have a wrapper function. Ie: just another variation on
getstatusoutput() that allows for use of a shell or stderr inclusion to be
controlled.
The shell is fast. I'd just use the shell when replacing os.popen uses in
tests.
--
___
Python tracker
<https://bugs.python.org/issue42648>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41877] Check against misspellings of assert etc. in mock
Gregory P. Smith added the comment: New changeset fdb9efce6ac211f973088eef508740c3fa2bd182 by vabr-g in branch 'master': bpo-41877: Check for misspelled speccing arguments (GH-23737) https://github.com/python/cpython/commit/fdb9efce6ac211f973088eef508740c3fa2bd182 -- ___ Python tracker <https://bugs.python.org/issue41877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36541] Make lib2to3 grammar better match Python, support the := walrus
Change by Gregory P. Smith : -- resolution: out of date -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker <https://bugs.python.org/issue36541> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36541] Make lib2to3 grammar better match Python, support the := walrus
Gregory P. Smith added the comment: New changeset 42c9f0fd0a5e67d4ae0022bfd7370cb9725a5b01 by Gregory P. Smith in branch 'master': bpo-36541: Add lib2to3 grammar PEP-570 pos-only arg parsing (GH-23759) https://github.com/python/cpython/commit/42c9f0fd0a5e67d4ae0022bfd7370cb9725a5b01 -- ___ Python tracker <https://bugs.python.org/issue36541> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36541] Make lib2to3 grammar better match Python, support the := walrus
Gregory P. Smith added the comment: While I said i didn't care... and don't really want to... I found a reason to at least not omit pep-570 positional only arg parsing support give things like yapf still use it rather than forking their own copy. PR testing. -- assignee: -> gregory.p.smith status: closed -> open ___ Python tracker <https://bugs.python.org/issue36541> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36541] Make lib2to3 grammar better match Python, support the := walrus
Change by Gregory P. Smith : -- pull_requests: +22615 pull_request: https://github.com/python/cpython/pull/23759 ___ Python tracker <https://bugs.python.org/issue36541> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21627] Concurrently closing files and iterating over the open files directory is not well specified
Gregory P. Smith added the comment: The Linux kernel code is not sufficiently easy to follow to understand if it has this issue or if it pre-creates the dirent structures for all fds at opendir time for /proc/self/fd or if it is iterating through the list of fds in sorted order so an older closed fd will not interfere with its internal iteration. Regardless, I've yet to knowingly witness a problem from this come up in practice. knowingly and yet being key words. :) But I like the general theme of your patch to set CLOEXEC on all of the fd's rather than explicitly call close(fd) in the directory reading loop. -- assignee: -> gregory.p.smith components: +Library (Lib) resolution: out of date -> type: -> behavior versions: +Python 3.10, Python 3.9 -Python 3.5 ___ Python tracker <https://bugs.python.org/issue21627> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31904] Python should support VxWorks RTOS
Gregory P. Smith added the comment: New changeset 8d4f57dbd10846ffb4881b6509a511be0ab3b913 by pxinwr in branch 'master': bpo-31904: fix test_doctest.py failures for VxWorks (GH-23419) https://github.com/python/cpython/commit/8d4f57dbd10846ffb4881b6509a511be0ab3b913 -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue31904> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42558] waitpid/waitid race caused by change to Popen.send_signal in Python 3.9
Gregory P. Smith added the comment: I agree with Victor. When code launches a process with subprocess APIs, it is expected that the subprocess module manages the process. Calling os.waitpid directly instead of using subprocess's APIs breaks that expectation. Also, WNOWAIT and waitid() (not waitpid()) are not available on all platforms. Nor is pidfd_open(). Code using any of those for such a PR needs to be conditional on their runtime availability. -- ___ Python tracker <https://bugs.python.org/issue42558> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42406] Importing multiprocessing breaks pickle.whichmodule
Gregory P. Smith added the comment: thanks Renato! -- resolution: -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker <https://bugs.python.org/issue42406> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42406] Importing multiprocessing breaks pickle.whichmodule
Gregory P. Smith added the comment: New changeset 86684319d3dad8e1a7b0559727a48e0bc50afb01 by Renato Cunha in branch 'master': bpo-42406: Fix whichmodule() with multiprocessing (GH-23403) https://github.com/python/cpython/commit/86684319d3dad8e1a7b0559727a48e0bc50afb01 -- ___ Python tracker <https://bugs.python.org/issue42406> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42406] Importing multiprocessing breaks pickle.whichmodule
Gregory P. Smith added the comment: gross / nice find :) -- assignee: -> gregory.p.smith nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue42406> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13337] IGNORE_CASE doctest option flag
Gregory P. Smith added the comment: FWIW, while i'm never a fan of doctests, this would also help reduce the impact of other golden value tests where different platforms capitalize their error messages or not. -- nosy: +gregory.p.smith versions: +Python 3.10 -Python 3.3 ___ Python tracker <https://bugs.python.org/issue13337> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41200] Add pickle.loads fuzz test
Gregory P. Smith added the comment: Given that pickle is documented as: """ Warning The pickle module is not secure. Only unpickle data you trust. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling. """ https://docs.python.org/3/library/pickle.html What is fuzzing pickle.loads() expected to accomplish? -- assignee: -> gregory.p.smith nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue41200> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42468] subprocess.CompletedProcess: Add boolean value
Gregory P. Smith added the comment: Thanks for the suggestion though ThatXliner. It is good for us to keep these things in mind for future API designs. That __bool__ exists and could be meaningful in some contexts is often overlooked. -- resolution: -> rejected stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue42468> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42468] subprocess.CompletedProcess: Add boolean value
Gregory P. Smith added the comment: My concern with this as an API change is if anyone is returning or passing a CompletedProcess instance around it may well be used in a context where None is a potential value. Existing code in that situation would ordinarily be doing a bare `if cp:` test on it to check for None. I don't ordinarily see code that passes a CompletedProcess instance around... but it seems like an annoying potentially breaking change to make for a very small added value. `if cp.returncode:` is very explicit about its intent when read or written. `if cp:` is not, and could even be misread by someone unaware of the API to assume that cp is a number most likely the returncode itself and misunderstand that as "the returncode was non-zero" when it really means the opposite. If we had done this on day one of the run() -> CompletedProcess API this would've been a fine choice. But changing it now doesn't seem like a good idea to me. -- versions: -Python 3.9 ___ Python tracker <https://bugs.python.org/issue42468> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42468] subprocess.CompletedProcess: Add boolean value
Change by Gregory P. Smith : -- assignee: -> gregory.p.smith ___ Python tracker <https://bugs.python.org/issue42468> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42443] Provide Thread creation hook support
Change by Gregory P. Smith : -- components: +Library (Lib) ___ Python tracker <https://bugs.python.org/issue42443> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42443] Provide Thread creation hook support
Change by Gregory P. Smith : -- assignee: -> gregory.p.smith ___ Python tracker <https://bugs.python.org/issue42443> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42388] subprocess.check_output(['echo', 'test'], text=True, input=None) fails
Change by Gregory P. Smith : -- assignee: -> gregory.p.smith ___ Python tracker <https://bugs.python.org/issue42388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42388] subprocess.check_output(['echo', 'test'], text=True, input=None) fails
Gregory P. Smith added the comment: It was a mere oversight that this didn't handle text= the same as universal_newlines=. I made a PR to keep their behavior consistent and match the documentation. -- ___ Python tracker <https://bugs.python.org/issue42388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42388] subprocess.check_output(['echo', 'test'], text=True, input=None) fails
Change by Gregory P. Smith : -- keywords: +patch pull_requests: +22357 stage: -> patch review pull_request: https://github.com/python/cpython/pull/23467 ___ Python tracker <https://bugs.python.org/issue42388> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36906] Compile time textwrap.dedent() equivalent for str or bytes literals
Gregory P. Smith added the comment: I expect several phases here: (1) add a .dedent() method to str (and bytes?) - behaviors to consider mirroring are textwrap.dedent() and inspect.cleandoc(). Given their utility and similarities, it makes sense to offer both behaviors; behavior could be selected by a kwarg passed to the method. https://docs.python.org/3/library/textwrap.html#textwrap.dedent https://docs.python.org/3/library/inspect.html#inspect.cleandoc (2a) Ponder the d" prefix - but in general I expect sentiment to be against yet another letter prefix. They aren't pretty. This would need a PEP. Someone would need to champion it. (2b) Ponder making cleandoc dedenting automatic for docstrings. This would be enabled on a per-file basis via a `from __future__ import docstring_dedent` or similar as Serhiy suggested. No prefix letter required. Several releases later we could consider making it the default. This will also need needs a PEP. (3) Optimizations when .dedent() is called on a constant? Nice to have, But I suggest we land (1) first as its own base implementation PR. Then consider the follow-ons in parallel. I believe the current patch contains (1)+(3) right now. If so we should simplify it to (1) and make (3) am immediate followup as saving the runtime cost and data space is worthwhile. Ultimate end state: probably 1+2b+3, but even 1+3 or 1+2b is a nice win. -- ___ Python tracker <https://bugs.python.org/issue36906> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36906] Compile time textwrap.dedent() equivalent for str or bytes literals
Gregory P. Smith added the comment: (assigning to me as I want to help remi.lapeyre's .dedent() method PR move forward) -- assignee: -> gregory.p.smith ___ Python tracker <https://bugs.python.org/issue36906> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36906] Compile time textwrap.dedent() equivalent for str or bytes literals
Change by Gregory P. Smith : -- versions: +Python 3.10 -Python 3.9 ___ Python tracker <https://bugs.python.org/issue36906> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40550] Popen.terminate fails with ProcessLookupError under certain conditions
Gregory P. Smith added the comment: Thanks for the patch! PRs are in or on their way in for 3.10 and 3.9. The 3.8 auto-backport failed, if anyone wants it on a future 3.8.x please follow up with a manual cherry pick to make a PR for the 3.8 branch. -- resolution: -> fixed stage: patch review -> commit review status: open -> closed versions: -Python 3.8 ___ Python tracker <https://bugs.python.org/issue40550> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40550] Popen.terminate fails with ProcessLookupError under certain conditions
Gregory P. Smith added the comment: New changeset 01a202ab6b0ded546e47073db6498262086c52e9 by Filipe Laíns in branch 'master': bpo-40550: Fix time-of-check/time-of-action issue in subprocess.Popen.send_signal. (GH-20010) https://github.com/python/cpython/commit/01a202ab6b0ded546e47073db6498262086c52e9 -- ___ Python tracker <https://bugs.python.org/issue40550> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41655] New Node may not be visited in lib2to3.refactor.RefactoringTool.traverse_by
Change by Gregory P. Smith : -- resolution: -> wont fix stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue41655> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41655] New Node may not be visited in lib2to3.refactor.RefactoringTool.traverse_by
Gregory P. Smith added the comment: I expect your overall logic is sound in the PR. But this is a public API change, and while we've typically exempted lib2to3 from needing to concern itself about deprecations and API changes by leaving it undocumented and explicitly promising not to care about this module in that sense... Enough things use it today that we should probably not do that anymore. Especially given that we're considering it a deprecated library these days. It isn't likely to stand up well to 3.10+'s new grammar. Various projects have forked lib2to3 into their own maintained thing either directly on PyPI or as an internalized third_party fork within their own project (as I believe Black has done?). It is probably best to do that for changes of this nature rather than push for them to be included in a future stdlib lib2to3. See https://bugs.python.org/issue40360 for context. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue41655> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40791] hmac.compare_digest could try harder to be constant-time.
Gregory P. Smith added the comment: New changeset 31729366e2bc09632e78f3896dbce0ae64914f28 by Devin Jeanpierre in branch 'master': bpo-40791: Make compare_digest more constant-time. (GH-20444) https://github.com/python/cpython/commit/31729366e2bc09632e78f3896dbce0ae64914f28 -- ___ Python tracker <https://bugs.python.org/issue40791> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40550] Popen.terminate fails with ProcessLookupError under certain conditions
Change by Gregory P. Smith : -- assignee: -> gregory.p.smith nosy: +gregory.p.smith versions: +Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue40550> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42367] Restore os.makedirs ability to apply mode to all directories created
Gregory P. Smith added the comment: For consistency, pathlib.Path.mkdir should also gain this option. -- ___ Python tracker <https://bugs.python.org/issue42367> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42367] Restore os.makedirs ability to apply mode to all directories created
Change by Gregory P. Smith : -- type: behavior -> enhancement ___ Python tracker <https://bugs.python.org/issue42367> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19930] os.makedirs('dir1/dir2', 0) always fails
Gregory P. Smith added the comment: This API change was not strictly a bugfix, it removed a feature existing code was relying on. https://bugs.python.org/issue42367 opened to reconcile the two. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue19930> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42367] Restore os.makedirs ability to apply mode to all directories created
New submission from Gregory P. Smith : os.makedirs used to pass its mode argument on down recursively so that every level of directory it created would have the specified permissions. That was removed in Python 3.7 as https://bugs.python.org/issue19930 as if it were a mis-feature. Maybe it was in one situation, but it was also a desirable *feature*. It wasn't a bug. We've got 15 year old code depending on that and the only solution for it to work on Python 3.7-3.9 is to reimplement recursive directory creation. :( This feature needs to be brought back. Rather than flip flop on the API, adding an flag to indicate if the permissions should be applied recursively or not seems like the best way forward. The "workaround" documented in the above bug is invalid. umask cannot be used to control the intermediate directory creation permissions as that is a process wide global that would impact other threads or signal handlers. umask also cannot be used as umask does not allow setting of special bits such as stat.S_ISVTX. result: Our old os.makedirs() code that tried to set the sticky bit (ISVTX) on all directories now fails to set it at all levels. -- components: Library (Lib) keywords: 3.7regression messages: 381082 nosy: gregory.p.smith, serhiy.storchaka priority: normal severity: normal stage: needs patch status: open title: Restore os.makedirs ability to apply mode to all directories created type: behavior versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue42367> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42353] Proposal: re.prefixmatch method (alias for re.match)
Gregory P. Smith added the comment: My point is that re.match is a common bug when people really want re.search. re.prefixmatch makes it explicit and non-confusing and thus unlikely to be used wrong or misunderstood when read or reviewed. The term "match" when talking about regular expressions is not normally meant to imply any anchoring as anchors can be expressed within the regex. Python is relatively unique in bothering to have different methods for a prefix match and an anywhere match. (We'd have been better off without a match method entirely, only having search - too late now) -- ___ Python tracker <https://bugs.python.org/issue42353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42353] Proposal: re.prefixmatch method (alias for re.match)
New submission from Gregory P. Smith : A well known anti-pattern in Python is use of re.match when you meant to use re.search. re.fullmatch was added in 3.4 via https://bugs.python.org/issue16203 for similar reasons. re.prefixmatch would be similar: we want the re.match behavior, but want the code to be obvious about its intent. This documents the implicit ^ in the name. The goal would be to allow linters to ultimately flag re.match as the anti-pattern when in 3.10+ mode. Asking people to use re.prefixmatch or re.search instead. This would help avoid bugs where people mean re.search but write re.match. The implementation is trivial. This is **not** a decision to deprecate the widely used in 25 years worth of code's re.match name. That'd be painful and is unlikely to be worth doing without spreading it over a 8+ year timeline. -- components: Library (Lib) messages: 380928 nosy: gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: Proposal: re.prefixmatch method (alias for re.match) type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue42353> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42329] typing classes do not have __name__ attributes in 3.7+
Gregory P. Smith added the comment: working as intended, they aren't classes. -- resolution: -> rejected stage: needs patch -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue42329> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42329] typing classes do not have __name__ attributes in 3.7+
Gregory P. Smith added the comment: Not a big deal if we don't, I just found it odd so I figured I'd pose the question. That it's been in three releases and only just now come up is pretty telling that isn't critical. The code in question was trying to identify public functions in a module by inspecting names and I think they've got a more pedantic way to do that than their existing code that wouldn't be tripped up by a mere callable without a __name__. On Wed, Nov 11, 2020, 8:47 PM Guido van Rossum wrote: > > Guido van Rossum added the comment: > > Between 3.6 and 3.7 they stopped being types. > > IIRC this enabled optimizations. (Serhiy?) > > I don't think this is important, but I suppose you have some code that > this breaks? > > The name is passed to the constructor of _SpecialGenericAlias, so I'm fine > with fixing this, though the backports may get tricky when you get down to > 3.7. > > -- > nosy: +serhiy.storchaka > > ___ > Python tracker > <https://bugs.python.org/issue42329> > ___ > -- ___ Python tracker <https://bugs.python.org/issue42329> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42329] typing classes do not have __name__ attributes in 3.7+
New submission from Gregory P. Smith : Python 3.7-3.10a1: ``` >>> List.__name__ Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.8/typing.py", line 760, in __getattr__ raise AttributeError(attr) AttributeError: __name__ >>> type(List) >>> type(List[int]) ``` Python 3.6: ``` >>> from typing import List >>> List.__name__ 'List' >>> type(List) >>> type(List[int]) ``` Is this lack of common meta attributes intentional? It makes the typing types unusual. We just saw it trip up some code that was expecting everything to have a `__name__` attribute while moving beyond 3.6. Judging by that `__getattr__` implementation it should happen on other common `__` attributes as mentioned in https://docs.python.org/3/library/stdtypes.html?highlight=__name__#special-attributes as well. -- components: Library (Lib) messages: 380801 nosy: gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: typing classes do not have __name__ attributes in 3.7+ type: behavior versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue42329> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35560] format(float(123), "00") causes segfault in debug builds
Gregory P. Smith added the comment: This bug was introduced into the Python 3.6.8 "bugfix" release. https://github.com/python/cpython/pull/23231 will fix it if anyone needs that on modern 3.6. -- nosy: +gregory.p.smith versions: +Python 3.6 ___ Python tracker <https://bugs.python.org/issue35560> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39892] Enable DeprecationWarnings by default when not explicit in unittest.main()
Gregory P. Smith added the comment: my bad :) -- resolution: -> not a bug stage: needs patch -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue39892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14903] dictobject infinite loop in module set-up
Gregory P. Smith added the comment: I suspect so. If any modern supported python 3.x version runs into an issue like this I think opening a fresh bugreport is good. Closing as not reproducable / obsolete. -- resolution: -> out of date stage: test needed -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue14903> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41877] Check against misspellings of assert etc. in mock
Gregory P. Smith added the comment: Thanks for the patch! I'm leaving this open as dealing with the other aspect: * Wrong attribute names around asserts: autospect/auto_spec -> autospec, set_spec -> spec_set is still a possibility. (given you also found a number of those in our codebase leading to hidden testing bugs) Vedran: We're not claiming these are fixes for the fundamental problem. But they are useful practical bandaids on the existing APIs that a hundred of millions of lines of existing unittest code around the world make use of to prevent common unintended consequences of our existing API's now well know flaws. issue24651 looks like the better place to take up future design ideas. -- ___ Python tracker <https://bugs.python.org/issue41877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41877] Check against misspellings of assert etc. in mock
Gregory P. Smith added the comment: New changeset 4662fa9bfe4a849fe87bfb321d8ef0956c89a772 by vabr-g in branch 'master': bpo-41877 Check for asert, aseert, assrt in mocks (GH-23165) https://github.com/python/cpython/commit/4662fa9bfe4a849fe87bfb321d8ef0956c89a772 -- ___ Python tracker <https://bugs.python.org/issue41877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24651] Mock.assert* API is in user namespace
Change by Gregory P. Smith : -- versions: +Python 3.10 -Python 3.6 ___ Python tracker <https://bugs.python.org/issue24651> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24651] Mock.assert* API is in user namespace
Change by Gregory P. Smith : -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue24651> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41877] Check against misspellings of assert etc. in mock
Change by Gregory P. Smith : -- assignee: -> gregory.p.smith ___ Python tracker <https://bugs.python.org/issue41877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41877] Check against misspellings of assert etc. in mock
Change by Gregory P. Smith : -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue41877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13501] Make libedit support more generic; port readline / libedit to FreeBSD
Gregory P. Smith added the comment: at a glance, it looks like the PR needs updating. -- ___ Python tracker <https://bugs.python.org/issue13501> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42146] subprocess.Popen() leaks cwd in case of uid/gid overflow
Change by Gregory P. Smith : -- resolution: -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker <https://bugs.python.org/issue42146> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42146] subprocess.Popen() leaks cwd in case of uid/gid overflow
Gregory P. Smith added the comment: New changeset d3b4e068077dd26927ae7485bd0303e09d962c02 by Alexey Izbyshev in branch 'master': bpo-42146: Unify cleanup in subprocess_fork_exec() (GH-22970) https://github.com/python/cpython/commit/d3b4e068077dd26927ae7485bd0303e09d962c02 -- ___ Python tracker <https://bugs.python.org/issue42146> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42185] class body bytecode uses less efficient *_NAME opcodes
New submission from Gregory P. Smith :
The opcodes generated for the closure defining a class body looks like they
might be suboptimal. It seems stuck using the generic LOAD_NAME and STORE_NAME
opcodes rather than the LOAD_GLOBAL and STORE_FAST and friends as one would
expect and as happens within a normal function closure.
If true and the normal optimization pass (which I believe is done by
`compiler.c` `compiler_nameop()` if i'm understanding this area of the code I
don't know very well correctly...?) is not happening, it _appears_ maybe to be
due to the `PyST_GetScope` call not currently having a way to represent this
situation rather than being in a normal function?
I tried searching for an open issue on this but was at a loss of what to search
for. semi-related concepts might be found in:
https://bugs.python.org/issue34750 - "locals().update doesn't work in Enum
body, even though direct assignment to locals() does"
https://bugs.python.org/issue10399 - "AST Optimization: inlining of function
calls"
https://bugs.python.org/issue9226 - "errors creating classes inside a closure"
None of those is really the same though. if there are other filed issues
regarding this, feel free to link to them in comments.
If this can be improved as it has been for function bodies already, it should
be a measurable improvement to CPython startup time and/or import time.
Especially on larger programs and things with a lot of generated code full of
classes.
```
>>> import dis
>>> klass_def = '''
... class Klassy:
... pinky = 'brain'
... def Funktion(castle_argh):
... __module__ = __name__
... __qualname__ = 'not Klassy'
... pinky = 'brain'
... '''
>>> dis.dis(compile(klass_def, '', 'exec'))
2 0 LOAD_BUILD_CLASS
2 LOAD_CONST 0 (", line 2>)
4 LOAD_CONST 1 ('Klassy')
6 MAKE_FUNCTION0
8 LOAD_CONST 1 ('Klassy')
10 CALL_FUNCTION2
12 STORE_NAME 0 (Klassy)
4 14 LOAD_CONST 2 (", line 4>)
16 LOAD_CONST 3 ('Funktion')
18 MAKE_FUNCTION0
20 STORE_NAME 1 (Funktion)
22 LOAD_CONST 4 (None)
24 RETURN_VALUE
Disassembly of ", line 2>:
2 0 LOAD_NAME0 (__name__)
2 STORE_NAME 1 (__module__)
4 LOAD_CONST 0 ('Klassy')
6 STORE_NAME 2 (__qualname__)
3 8 LOAD_CONST 1 ('brain')
10 STORE_NAME 3 (pinky)
12 LOAD_CONST 2 (None)
14 RETURN_VALUE
Disassembly of ", line
4>:
5 0 LOAD_GLOBAL 0 (__name__)
2 STORE_FAST 1 (__module__)
6 4 LOAD_CONST 1 ('not Klassy')
6 STORE_FAST 2 (__qualname__)
7 8 LOAD_CONST 2 ('brain')
10 STORE_FAST 3 (pinky)
12 LOAD_CONST 0 (None)
14 RETURN_VALUE
```
--
components: Interpreter Core
messages: 379839
nosy: gregory.p.smith, twouters
priority: normal
severity: normal
stage: needs patch
status: open
title: class body bytecode uses less efficient *_NAME opcodes
type: performance
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue42185>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42096] zipfile.is_zipfile incorrectly identifying a gzipped file as a zip archive
Gregory P. Smith added the comment: The first four bytes of the file do not identify a zip file. A zip file is identified by the end of file central directory. Which you then must read entries of to determine where the start of the archive may be... often not at position zero. -- ___ Python tracker <https://bugs.python.org/issue42096> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42096] zipfile.is_zipfile incorrectly identifying a gzipped file as a zip archive
Gregory P. Smith added the comment: ZipFile.open() is not the code for opening a zip file. :) That's the code for opening a file embedded within an already constructed mode='r' archive as done the ZipFile.__init__() constructor. By the time you've gotten to the open() method, you've loaded the entire unbounded in size central directory into memory as ZipInfo objects [constructor] and are checking signature of an individual file header you are attempting to read out of the archive. Follow the ZipFile() constructor, it calls ZipFile._RealGetContents() which is the true start of parsing the archive. https://github.com/python/cpython/blob/master/Lib/zipfile.py#L1317 Sure, more and more steps can be done. But if you want to do that, you may as well just get rid of is_zipfile() entirely - a functions who's point is to be fast and not consume an amount of memory determined by the input data - and have people just call `zipfile.ZipFile(path_in_question, mode='r')` and live with the consequences of attempting to load and parse the whole thing. If that doesn't raise an exception, it is more likely to be a zip file. But that could still raise an exception when trying to open each of the files inside, so you'd need to iterate over this and open those and make sure they're valid. is_zipfile() isn't a verify_zipfile_integrity() routine. Just a quick best guess. is_zipfile() cannot be perfect and is not intended to be. There is always going to be yet another thing it _could_ try. It isn't worth chasing the impossible goal and making it not be fast. Just update the is_zipfile() docs. -- ___ Python tracker <https://bugs.python.org/issue42096> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42146] subprocess.Popen() leaks cwd in case of uid/gid overflow
Gregory P. Smith added the comment: New changeset c0590c0033e86f98cdf5f2ca6898656f98ab4053 by Alexey Izbyshev in branch 'master': bpo-42146: Fix memory leak in subprocess.Popen() in case of uid/gid overflow (GH-22966) https://github.com/python/cpython/commit/c0590c0033e86f98cdf5f2ca6898656f98ab4053 -- ___ Python tracker <https://bugs.python.org/issue42146> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35823] Use vfork() in subprocess on Linux
Gregory P. Smith added the comment:
Performance improvement measured on a 1.4Ghz quad aarch64 A57 (nvidia jetson
nano):
#define VFORK_USABLE 1
test_subprocess: 36.5 seconds
#undef VFORK_USABLE
test_subprocess: 45 seconds
Nice. I really didn't expect a 20% speedup on its testsuite alone!
Lets dive into that with a microbenchmark:
$ ./build-novfork/python ten_seconds_of_truth.py
Launching /bin/true for 10 seconds in a loop.
Launched 2713 subprocesses in 10.00194378796732 seconds.
271.247275281014 subprocesses/second.
Increased our mapped pages by 778240KiB.
Launching /bin/true for 10 seconds in a loop.
Launched 212 subprocesses in 10.006392606999725 seconds.
21.186456331095847 subprocesses/second.
$ ./build/python ten_seconds_of_truth.py
Launching /bin/true for 10 seconds in a loop.
Launched 3310 subprocesses in 10.001623224001378 seconds.
330.94628000551285 subprocesses/second.
Increased our mapped pages by 778240KiB.
Launching /bin/true for 10 seconds in a loop.
Launched 3312 subprocesses in 10.001519071985967 seconds.
331.1496959773679 subprocesses/second.
Demonstrating perfectly the benefit of vfork(). The more mapped pages, the
slower regular fork() becomes. With vfork() the size of the process's memory
map does not matter.
ten_seconds_of_truth.py:
```python
from time import monotonic as now
import subprocess
def benchmark_for_ten():
print('Launching /bin/true for 10 seconds in a loop.')
count = 0
start = now()
while now() - start < 10.:
subprocess.run(['/bin/true'])
count += 1
end = now()
duration = end-start
print(f'Launched {count} subprocesses in {duration} seconds.')
print(f'{count/duration} subprocesses/second.')
benchmark_for_ten()
map_a_bunch_of_pages = '4agkahglahaa^#\0ag3\3'*1024*1024*40
print(f'Increased our mapped pages by {len(map_a_bunch_of_pages)//1024}KiB.')
benchmark_for_ten()
```
--
___
Python tracker
<https://bugs.python.org/issue35823>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35823] Use vfork() in subprocess on Linux
Gregory P. Smith added the comment: Nice links. LOL, yes, musl source was going to be my next stop if the larger libc sources proved impossible for a mere mortal to reason about. :) regarding macOS, agreed. If someone needs vfork() to work there, I believe it could be made to. Options like selecting the architecture of the child process could be higher level options to the subprocess API. Hiding the platform specific details of how that happens and deciding which underlying approach to use based on the flags. multi-arch systems are a thing. It is conceivable that we may even see non-esoteric multi-arch hardware at some point. -- ___ Python tracker <https://bugs.python.org/issue35823> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35823] Use vfork() in subprocess on Linux
Gregory P. Smith added the comment: New changeset be3c3a0e468237430ad7d19a33c60d306199a7f2 by Gregory P. Smith in branch 'master': bpo-35823: Allow setsid() after vfork() on Linux. (GH-22945) https://github.com/python/cpython/commit/be3c3a0e468237430ad7d19a33c60d306199a7f2 -- ___ Python tracker <https://bugs.python.org/issue35823> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
