[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: New changeset 0cecc22842dcc4090eb9cb99e7dababea7034a87 by Victor Stinner in branch '3.6': bpo-20891: Remove test_capi.test_bpo20891() (#5425) https://github.com/python/cpython/commit/0cecc22842dcc4090eb9cb99e7dababea7034a87

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by STINNER Victor : -- pull_requests: +5260 ___ Python tracker ___ ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: Antoine Pitrou: > @vstinner I don't think so. People can already call PyEval_InitThreads. Since only two users complained about https://bugs.python.org/issue20891 in 3 years, I agree that it's ok to not fix Python 2.7 and 3.6. The

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: New changeset d951157268b2122109098c792562b71ccc41920b by Victor Stinner in branch 'master': bpo-20891: Reenable test_embed.test_bpo20891() (GH-5420) https://github.com/python/cpython/commit/d951157268b2122109098c792562b71ccc41920b

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: Antoine Pitrou considers that my PR 5421 for Python 3.6 should not be merged: "@vstinner I don't think so. People can already call PyEval_InitThreads." https://github.com/python/cpython/pull/5421#issuecomment-361214537 --

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: I proposed PR 5421 to fix Python 3.6: modify Py_Initialize() to call PyEval_InitThreads(). I'm not sure of Python 2.7. Not only the backport is not straighforward, but I'm not sure that I want to touch the super-stable 2.7 branch.

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by STINNER Victor : -- pull_requests: +5256 ___ Python tracker ___ ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: > I ran pyperformance on my PR 4700. Differences of at least 5%: (...) I tested again these 5 benchmarks were Python was slower with my PR. I ran these benchmarks manually on my laptop using CPU isolation. Result: vstinner@apu$

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by STINNER Victor : -- pull_requests: +5255 ___ Python tracker ___ ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: New changeset 2914bb32e2adf8dff77c0ca58b33201bc94e398c by Victor Stinner in branch 'master': bpo-20891: Py_Initialize() now creates the GIL (#4700) https://github.com/python/cpython/commit/2914bb32e2adf8dff77c0ca58b33201bc94e398c

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: New changeset 2e1ef00171179a8b906631b175cde2e68a804522 by Victor Stinner in branch '3.6': bpo-20891: Skip test_embed.test_bpo20891() (#4967) (#4969) https://github.com/python/cpython/commit/2e1ef00171179a8b906631b175cde2e68a804522

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by STINNER Victor : -- pull_requests: +4861 ___ Python tracker ___ ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: New changeset 550ee051d605b909dd75ef686d8e1244a0994394 by Victor Stinner in branch 'master': bpo-20891: Skip test_embed.test_bpo20891() (#4967) https://github.com/python/cpython/commit/550ee051d605b909dd75ef686d8e1244a0994394

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by STINNER Victor : -- pull_requests: +4859 ___ Python tracker ___ ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: I ran pyperformance on my PR 4700. Differences of at least 5%: haypo@speed-python$ python3 -m perf compare_to ~/json/uploaded/2017-12-18_12-29-master-bd6ec4d79e85.json.gz

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by Barry A. Warsaw : -- nosy: +barry ___ Python tracker ___ ___ Python-bugs-list

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by Kevin Chen : -- nosy: +kchen ___ Python tracker ___ ___ Python-bugs-list mailing

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Nick Coghlan added the comment: +1 for making this change 3.6+ only. Victor, could you run your patch through the performance benchmarks? While I suspect Antoine is right that our current GIL management heuristics will mean we don't need the lazy initialisation

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Guido van Rossum added the comment: For 2.7 I hesitate to OK this, who knows what skeletons are still in that closet. It doesn't sound like a security fix to me. For 3.6 I'm fine with it as a bugfix. -- ___ Python tracker

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: I wrote the PR 4700 to create the GIL in Py_Initialize(). Would you be ok to backport this fix to Python 2.7 and 3.6 as well? -- ___ Python tracker

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by STINNER Victor : -- pull_requests: +4612 stage: resolved -> patch review ___ Python tracker ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Guido van Rossum added the comment: Yeah, the original reasoning was that threads were something esoteric and not used by most code, and at the time we definitely felt that always using the GIL would cause a (tiny) slowdown and increase the risk of crashes due to bugs in the

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Antoine Pitrou added the comment: > I guess that the intent of dynamically created GIL is to reduce the > "overhead" of the GIL when 100% of the code is run in single thread. I'm not sure it is ok to guess here. The original code was written in 1992, when system threading

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: The "if (gil_created())" check can be found in the Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS macros which call PyEval_SaveThread() / PyEval_RestoreThread(): PyThreadState * PyEval_SaveThread(void) { PyThreadState *tstate =

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: > Why not *always* call PyEval_InitThreads() at interpreter initialization? > Are there any downsides? The code is quite old: commit 1984f1e1c6306d4e8073c28d2395638f80ea509b Author: Guido van Rossum Date: Tue

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Antoine Pitrou added the comment: Why not *always* call PyEval_InitThreads() at interpreter initialization? Are there any downsides? -- ___ Python tracker

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: "./Program/_testembed.exe bpo20891" fails randomly on macOS: --- macbook:master haypo$ while true; do ./Programs/_testembed bpo20891 ||break; date; done Lun 4 déc 2017 12:46:34 CET Lun 4 déc 2017 12:46:34 CET Lun 4 déc 2017

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: Ok, the bug is now fixed in Python 2.7, 3.6 and master (future 3.7). On 3.6 and master, the fix comes with an unit test. Thanks Steve Dower for the bug report, sorry for the long delay, I completely forgot this old bug! Marcin

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: New changeset e10c9de9d74fd4c26b32e6719d96f04a5be6987d by Victor Stinner in branch '3.6': bpo-20891: Fix PyGILState_Ensure() (#4650) (#4655) https://github.com/python/cpython/commit/e10c9de9d74fd4c26b32e6719d96f04a5be6987d

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: New changeset be6b74c0795b709c7a04e2187a7e32d08f5155f6 by Victor Stinner in branch '2.7': bpo-20891: Fix PyGILState_Ensure() (#4650) (#4657) https://github.com/python/cpython/commit/be6b74c0795b709c7a04e2187a7e32d08f5155f6

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by STINNER Victor : -- pull_requests: +4569 ___ Python tracker ___ ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by STINNER Victor : -- pull_requests: +4567 ___ Python tracker ___ ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: New changeset b4d1e1f7c1af6ae33f0e371576c8bcafedb099db by Victor Stinner in branch 'master': bpo-20891: Fix PyGILState_Ensure() (#4650) https://github.com/python/cpython/commit/b4d1e1f7c1af6ae33f0e371576c8bcafedb099db --

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: I have to check if Python 2.7 is impacted as well. -- versions: +Python 3.6, Python 3.7 -Python 3.4 ___ Python tracker

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: > Is this fix released? I can't find it in the changelog… Oops, I lost track of this issue, but it wasn't fixed, no. I just proposed my old fix as a pull requets: PR 4650. > (I faced this bug on 3.5.2, released a couple of months

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Change by STINNER Victor : -- pull_requests: +4562 stage: -> patch review ___ Python tracker ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Marcin Kasperski added the comment: Is this fix released? I can't find it in the changelog… (I faced this bug on 3.5.2, released a couple of months after this bug was closed…) -- nosy: +Mekk ___ Python tracker

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: Attached patch should fix the issue. -- keywords: +patch Added file: http://bugs.python.org/file42174/PyGILState_Ensure.patch ___ Python tracker

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: ptest.c: Portable example using Python API. -- Added file: http://bugs.python.org/file42173/ptest.c ___ Python tracker

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Changes by Xavier Combelle xavier.combe...@gmail.com: -- nosy: +xcombelle ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20891 ___ ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

STINNER Victor added the comment: IMO it's a bug in PyEval_InitThreads(). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20891 ___ ___

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

New submission from Steve Dower: In Python 3.4rc3, calling PyGILState_Ensure() from a thread that was not created by Python and without any calls to PyEval_InitThreads() will cause a fatal exit: Fatal Python error: take_gil: NULL tstate I believe this was added in

[issue20891] PyGILState_Ensure on non-Python thread causes fatal error

Steve Dower added the comment: Should have linked to #19576 as well, which is the issue associated with that changeset. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20891 ___