[issue32604] Expose the subinterpreters C-API in Python for testing use.

[STINNER Victor]

STINNER Victor  added the comment:

The revert fixed the test:

$ ./python -m test -R 3:3 test__xxsubinterpreters 
(...)
Tests result: SUCCESS

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[STINNER Victor]

STINNER Victor  added the comment:


New changeset f2c3b6823bc4777d4a14eb0c3615b719521f763a by Victor Stinner in 
branch 'master':
Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)" 
(GH-20089)
https://github.com/python/cpython/commit/f2c3b6823bc4777d4a14eb0c3615b719521f763a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[STINNER Victor]

STINNER Victor  added the comment:

The buildbots are broken for one whole week and nobody is available to 
investigate the reference leak regression. Following the CI policy, I wrote PR 
20089 to revert the change which broke the CI:
https://pythondev.readthedocs.io/ci.html#revert-on-fail

If someone wants to push again the change, make sure that "./python -m test -R 
3:3 test__xxsubinterpreters" does not leak.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19393
pull_request: https://github.com/python/cpython/pull/20089

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[STINNER Victor]

STINNER Victor  added the comment:

The commit a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57 introduced reference leaks:

$ ./python -m test -R 3:3 test__xxsubinterpreters -m 
test.test__xxsubinterpreters.RunFailedTests.test_traceback_propagated
(...)
test__xxsubinterpreters leaked [1863, 1861, 1863] references, sum=5587
test__xxsubinterpreters leaked [559, 558, 559] memory blocks, sum=1676
(...)

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57 by Eric Snow in branch 
'master':
bpo-32604: [_xxsubinterpreters] Propagate exceptions. (GH-19768)
https://github.com/python/cpython/commit/a1d9e0accd33af1d8e90fc48b34c13d7b07dcf57


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +19149
pull_request: https://github.com/python/cpython/pull/19829

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[miss-islington]

miss-islington  added the comment:


New changeset 5e8c691594d68925213d36296ce7c4b3e90bcb1d by Eric Snow in branch 
'master':
bpo-32604: Add support for a "default" arg in channel_recv(). (GH-19770)
https://github.com/python/cpython/commit/5e8c691594d68925213d36296ce7c4b3e90bcb1d


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +19090
pull_request: https://github.com/python/cpython/pull/19770

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +19088
pull_request: https://github.com/python/cpython/pull/19768

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Jamie Stribling]

Change by Jamie Stribling :


--
components: +Documentation -Interpreter Core

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

Also see #24553.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset 63799136e6c0491bb5d6f4a234d5a775db3458db by Eric Snow in branch 
'master':
bpo-33615: Re-enable a subinterpreter test. (gh-7251)
https://github.com/python/cpython/commit/63799136e6c0491bb5d6f4a234d5a775db3458db


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +6960

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset 110bc01407ac8c75545d0386577c6e17254d97d9 by Eric Snow in branch 
'master':
bpo-33615: Temporarily disable a test that is triggering crashes on a few 
buildbots. (gh-7288)
https://github.com/python/cpython/commit/110bc01407ac8c75545d0386577c6e17254d97d9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +6915

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +6877

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Emily Morehouse]

Change by Emily Morehouse :


--
nosy: +emilyemorehouse

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Yury Selivanov]

Change by Yury Selivanov :


--
nosy:  -yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset 3ab0136ac5d6059ce96d4debca89c5f5ab0356f5 by Eric Snow in branch 
'master':
bpo-32604: Implement force-closing channels. (gh-6937)
https://github.com/python/cpython/commit/3ab0136ac5d6059ce96d4debca89c5f5ab0356f5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +6607

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Alexander Belopolsky]

Change by Alexander Belopolsky :


--
nosy: +belopolsky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset 6d2cd9036c0ab78a83de43d1511befb7a7fc0ade by Eric Snow in branch 
'master':
bpo-32604: Improve subinterpreter tests. (#6914)
https://github.com/python/cpython/commit/6d2cd9036c0ab78a83de43d1511befb7a7fc0ade


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +6582

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Ned Deily]

Ned Deily  added the comment:

As agreed upon at the PyCon dev sprints, 
6bd0c476c58ca0046969f70dc2a4e4dfb3268062 removes _xxsubinterpreters from the 
3.7 branch and 3.7.0 and retargets for 3.8.  Since it was intended for internal 
use, no user documentation or APIs should be affected.

--
priority: deferred blocker -> 
versions:  -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset 6bd0c476c58ca0046969f70dc2a4e4dfb3268062 by Eric Snow in branch 
'3.7':
bpo-32604: Remove xid registry. (#6813)
https://github.com/python/cpython/commit/6bd0c476c58ca0046969f70dc2a4e4dfb3268062


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +6499
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

Sounds good, Ned.  Sorry for my confusion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Ned Deily]

Change by Ned Deily :


--
priority: normal -> deferred blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Ned Deily]

Ned Deily  added the comment:

> Still planned for 3.7: [...]

Eric, please try to get everything you want done and into master as soon as 
possible, then produce a PR for 3.7, and then we can review it and make a 
decision if it is appropriate for 3.7.  As I've mentioned in msg310448 and 
stated elsewhere, the expectation is that the few post-b1 feature freeze 
extensions granted were only for b2.  Even changes limited to Lib/test can 
break buildbots and downstream users' testing and take multiple releases to 
stabilize.  Thanks for your work on this!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

Still planned for 3.7:

1. add more tests in Lib/test/test__xxsubinterpreters.py
2. add test.support.interpreters (high-level module)
3. add more tests in Lib/test/test_interpreters.py

At that point I'll start using the high-level module in tests for 
subinterpreter functionality and for other open issues (e.g. #10915, #15751), 
aiming to improve stability of the feature.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

As far as I know, everything related to this issue is in a stable state.  I 
have not been able to determine any way in which the reported memory leak (in 
test_multiprocessing_fork) could be related.  Sorry, I didn't notice the 
release blocker priority.

I do not plan on landing any further changes here for beta2.  However, I do 
plan on a few more additions under Lib/test before we go final.

--
priority: release blocker -> normal
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Nick Coghlan]

Nick Coghlan  added the comment:

_xxsubinterpreters has been added, and we'll use PEP 554 as the interim 
documentation while it's only exposed for testing purposes.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Ned Deily]

Ned Deily  added the comment:

What's the status of this issue?  3.7.0b2 is tagging in 48 hours or so.

--
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset eed3c7af4ec520d324a55e0357f41589a66906f7 by Eric Snow (Miss 
Islington (bot)) in branch '3.7':
bpo-32604: Swap threads only if the interpreter is different. (gh-5783)
https://github.com/python/cpython/commit/eed3c7af4ec520d324a55e0357f41589a66906f7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset f53d9f2778a87bdd48eb9030f782a4ebf9e7622f by Eric Snow in branch 
'master':
bpo-32604: Swap threads only if the interpreter is different. (gh-5778)
https://github.com/python/cpython/commit/f53d9f2778a87bdd48eb9030f782a4ebf9e7622f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[miss-islington]

Change by miss-islington :


--
pull_requests: +5564

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +5559

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[pmpp]

Change by pmpp :


--
nosy: +pmpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Zachary Ware]

Zachary Ware  added the comment:

I'm pretty sure that report is still running on the last changeset on 
hg.python.org; can you confirm that Antoine?

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Yury Selivanov]

Yury Selivanov  added the comment:

FYI I found out about this refleak from 
https://mail.python.org/pipermail/python-checkins/2018-February/153907.html

So it's definitely not Mac OS X specific thing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Zachary Ware]

Zachary Ware  added the comment:

Looks like macOS rather than Windows, and I can't reproduce it locally with 
current master:

14:28 $ ./python.exe -m test -R3:3 test_multiprocessing_fork
Run tests sequentially
0:00:00 load avg: 2.68 [1/1] test_multiprocessing_fork
beginning 6 repetitions
123456
..
test_multiprocessing_fork passed in 9 min 20 sec
1 test OK.

Total duration: 9 min 20 sec
Tests result: SUCCESS



The Windows and Linux refleak buildbots [1] are currently unhappy about other 
things that appear to be unrelated.

[1] http://buildbot.python.org/all/#/builders?tags=%2Brefleak

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

Davin, any thoughts on how my most recent commit (for this issue) might be 
causing the leaks Yury found?

--
nosy: +davin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

I'm not seeing any refleak (on linux/clang).  I'm guessing this is 
Windows-specific (based on use of "./python.exe").  How does 
test_multiprocessing_fork even run on Windows?  I thought "fork" is an 
unsupported start method on Windows (see 
https://docs.python.org/3.7/library/multiprocessing.html#contexts-and-start-methods).

Also, I'm not sure how my change might cause a refleak outside of code using 
the _xxsubinterpreters module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

@Yury, thanks!  I thought I had checked before I made the PR, but apparently 
not.  I'll git it fixed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Yury Selivanov]

Yury Selivanov  added the comment:

Eric, it looks like your recent commit introduced a refleak.  We need to fix it 
before beta2.

~/d/p/cpython (master $) » ./python.exe -m test -R3:3 test_multiprocessing_fork
Run tests sequentially
0:00:00 load avg: 2.52 [1/1] test_multiprocessing_fork
beginning 6 repetitions
123456
..
test_multiprocessing_fork leaked [21, 2, 1] memory blocks, sum=24
test_multiprocessing_fork leaked [2, 0, 0] file descriptors, sum=2
test_multiprocessing_fork failed in 9 min 48 sec

1 test failed:
test_multiprocessing_fork


And just before it:


~/d/p/cpython ((bd093355…) $) » ./python.exe -m test -R3:3 
test_multiprocessing_fork
Run tests sequentially
0:00:00 load avg: 3.70 [1/1] test_multiprocessing_fork
beginning 6 repetitions
123456
..
test_multiprocessing_fork passed in 9 min 12 sec
1 test OK.

Total duration: 9 min 12 sec
Tests result: SUCCESS

--
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset 3db05a3a9c109f49d54b4965852e41c657e9b117 by Eric Snow (Miss 
Islington (bot)) in branch '3.7':
bpo-32604: Clean up created subinterpreters before runtime finalization. 
(gh-5710)
https://github.com/python/cpython/commit/3db05a3a9c109f49d54b4965852e41c657e9b117


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[miss-islington]

Change by miss-islington :


--
pull_requests: +5499

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset 4c6955e2b0ccf88c705f8d1fac685a8e65f9699e by Eric Snow in branch 
'master':
bpo-32604: Clean up created subinterpreters before runtime finalization. 
(gh-5709)
https://github.com/python/cpython/commit/4c6955e2b0ccf88c705f8d1fac685a8e65f9699e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +5498

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Zachary Ware]

Zachary Ware  added the comment:


New changeset fe61e8d8c7cd1f6fb0ce7e9b8f0460b47b5f29a5 by Zachary Ware (Miss 
Islington (bot)) in branch '3.7':
bpo-32604: Make _xxsubinterpreters build on Windows (GH-5624)
https://github.com/python/cpython/commit/fe61e8d8c7cd1f6fb0ce7e9b8f0460b47b5f29a5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[miss-islington]

Change by miss-islington :


--
pull_requests: +5432

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Zachary Ware]

Zachary Ware  added the comment:


New changeset 310b05289b5d9550040f469e60b5e8e77f1022b6 by Zachary Ware in 
branch 'master':
bpo-32604: Make _xxsubinterpreters build on Windows (GH-5516)
https://github.com/python/cpython/commit/310b05289b5d9550040f469e60b5e8e77f1022b6


--
nosy: +zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Zachary Ware]

Change by Zachary Ware :


--
pull_requests: +5348

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

FYI, out of 2389 source lines in the C extension, 1563 are the channel-related 
code.  That means the non-channel code is 826 lines (about a third).  That 
non-channel code does not depend on the channel code at all and I considered 
splitting the source out, but figured there wasn't enough benefit.  However, I 
might revisit the matter later when I circle back to PEP 554. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset f33ecedcad5a001735fa4ded5d21caa2cbf27911 by Eric Snow (Miss 
Islington (bot)) in branch '3.7':
bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (GH-5507)
https://github.com/python/cpython/commit/f33ecedcad5a001735fa4ded5d21caa2cbf27911


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[miss-islington]

Change by miss-islington :


--
pull_requests: +5340

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

I've landed a PR that fixes all the memory leaks in the module.  It also fixes 
the 2 defects reported by coverity.  The only thing left here is to get the 
module building under Windows.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset 4e9da0d163731caa79811c723c703ee416c31826 by Eric Snow in branch 
'master':
bpo-32604: Fix memory leaks in the new _xxsubinterpreters module. (#5507)
https://github.com/python/cpython/commit/4e9da0d163731caa79811c723c703ee416c31826


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +5338

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Yury Selivanov]

Yury Selivanov  added the comment:

A couple defects reported by coverity:



** CID 1428758: Integer handling issues (CONSTANT_EXPRESSION_RESULT) 
/Modules/_xxsubinterpretersmodule.c: 45 in _coerce_id()

 *** CID 1428758: Integer handling issues (CONSTANT_EXPRESSION_RESULT) 
/Modules/_xxsubinterpretersmodule.c: 45 in _coerce_id()
39 }
40 if (cid < 0) {
41 PyErr_SetString(PyExc_ValueError, 42 "'id' must be a non-negative int"); 43 
return -1;
44 }
45 if (cid > INT64_MAX) {
46 PyErr_SetString(PyExc_ValueError, 47 "'id' too large (must be 64-bit int)"); 
48 return -1;
49 }
50 return cid;
** CID 1428757: Integer handling issues (CONSTANT_EXPRESSION_RESULT) 
/Modules/_xxsubinterpretersmodule.c: 1215 in channelid_richcompare()

 *** CID 1428757: Integer handling issues (CONSTANT_EXPRESSION_RESULT) 
/Modules/_xxsubinterpretersmodule.c: 1215 in channelid_richcompare()
1209 }
1210 int64_t othercid = PyLong_AsLongLong(other); 1211 // XXX decref other here?
1212 if (othercid == -1 && PyErr_Occurred() != NULL) { 1213 return NULL;
1214 }
1215 if (othercid < 0 || othercid > INT64_MAX) { 1216 equal = 0;
 
1217 }
1218 else {
1219 equal = (cid->id == othercid); 1220 }
** CID 1428756: Error handling issues (CHECKED_RETURN) /Modules/gcmodule.c: 
1071 in gc_enable_impl()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

The buildbots should be happier now.  I'll keep an eye on them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset 83e64c8a544028ae677af2a0bc268dbe1c11cc3a by Eric Snow in branch 
'master':
bpo-32604: NULL-terminate kwlist in channel_drop_interpreter(). (gh-5437)
https://github.com/python/cpython/commit/83e64c8a544028ae677af2a0bc268dbe1c11cc3a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

I just put up a PR that should fix the 4 buildbots.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +5270

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

On the PPC64 AIX 3.x buildbot:

==
FAIL: test_repr (test.test__xxsubinterpreters.ChannelIDTests)
--
Traceback (most recent call last):
  File 
"/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test__xxsubinterpreters.py",
 line 784, in test_repr
self.assertEqual(repr(cid), 'ChannelID(10)')
AssertionError: 'ChannelID(0)' != 'ChannelID(10)'
- ChannelID(0)
+ ChannelID(10)
?   +

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

On 4 of the buildbots:


==
ERROR: test_drop_multiple_times (test.test__xxsubinterpreters.ChannelTests)
--
Traceback (most recent call last):
  File 
"/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test__xxsubinterpreters.py",
 line 890, in test_drop_multiple_times
interpreters.channel_drop_interpreter(cid, send=True, recv=True)
SystemError: More keyword list entries (7) than format specifiers (3)
==
ERROR: test_drop_single_user (test.test__xxsubinterpreters.ChannelTests)
--
Traceback (most recent call last):
  File 
"/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test__xxsubinterpreters.py",
 line 848, in test_drop_single_user
interpreters.channel_drop_interpreter(cid, send=True, recv=True)
SystemError: More keyword list entries (7) than format specifiers (3)
==
ERROR: test_drop_used_multiple_times_by_single_user 
(test.test__xxsubinterpreters.ChannelTests)
--
Traceback (most recent call last):
  File 
"/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test__xxsubinterpreters.py",
 line 957, in test_drop_used_multiple_times_by_single_user
interpreters.channel_drop_interpreter(cid, send=True, recv=True)
SystemError: More keyword list entries (7) than format specifiers (3)
==
ERROR: test_drop_with_unused_items (test.test__xxsubinterpreters.ChannelTests)
--
Traceback (most recent call last):
  File 
"/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test__xxsubinterpreters.py",
 line 899, in test_drop_with_unused_items
interpreters.channel_drop_interpreter(cid, send=True, recv=True)
SystemError: More keyword list entries (7) than format specifiers (3)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

Yeah, I'm looking into it.  Also, I noticed some refleaks that I'll be sorting 
out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Ned Deily]

Ned Deily  added the comment:

Eric, looks like some buildbots are unhappy, for instance:

http://buildbot.python.org/all/#builders/13/builds/648

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

I've merged the patch without Windows support, which shouldn't be a problem 
given the purpose of the extension module.  I've also added a PR for get the 
module building under Windows.  I'd like to get that resolved ASAP.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +5269

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:


New changeset 7f8bfc9b9a8381ddb768421b5dd5cbd970266190 by Eric Snow in branch 
'master':
bpo-32604: Expose the subinterpreters C-API in a "private" stdlib module. 
(gh-1748)
https://github.com/python/cpython/commit/7f8bfc9b9a8381ddb768421b5dd5cbd970266190


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

Sounds good, Ned.  Thanks for taking a look.  I should have everything finished 
up by Friday, so I'm hopeful for landing the change before the deadline.  I may 
have a few minor tweaks to make after that, but I'll discuss that with you 
before making any changes if that happens.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Ned Deily]

Ned Deily  added the comment:

@Eric, given the breadth of change introduced in the PR (including adding a new 
extension), I think it would be best if at all possible to get it in for beta 1 
if we can resolve the review comments in time.  If necessary and if there are 
no objections from other core developers, I would be willing to consider making 
an exception and allowing it into beta 2 as long as it remains a private 
interface. If it looks like it won't be in releasable shape by then, I think 
you should hold off for 3.8; doing otherwise would be unfair to others and to 
our downstream beta users / testers, for example, even if it is private, adding 
a new extension and setup.py changes potentially affect downstream packagers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

FYI, there are a few things I need to clean up in the PR.  However, I expect 
that those changes will be minor relative to the the whole patch, so I wanted 
to get the ball rolling on a review. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Change by Eric Snow :


--
keywords: +patch
pull_requests: +5094
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

@Nick, I may make the name change you suggested in issue #30439 
("_subinterpreters").

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

Eric Snow  added the comment:

@Ned, it may be a little tight to land this given the time left before beta 1.  
However, this is meant as a tool for us to use in the test suite (particularly 
to test the subinterpreter C-API).  So I'm arguing that, if necessary, it would 
still be okay to land this after the feature freeze.  (I'm still hoping to get 
this in before the cutoff.)  What do you think?

--
nosy: +ned.deily, steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32604] Expose the subinterpreters C-API in Python for testing use.

[Eric Snow]

New submission from Eric Snow :

In the interest of getting something landed for 3.7, so we can start using it 
in tests, I'm putting up a patch for a low-level interpreters module.  In some 
ways this is a precursor for issue #30439, which will add a proper public 
stdlib module in 3.8.

The module I'm adding draws from the ideas in PEP 554 (particularly for 
channels).  Consequently, this will also give us an opportunity to try out some 
of the semantics from the PEP to give us better ideas for 3.8.

I expect to have some follow-on patches to facilitate simpler use in tests.  
This patch is big enough already. :)

--
assignee: eric.snow
components: Interpreter Core
messages: 310314
nosy: eric.snow, ncoghlan, yselivanov
priority: normal
severity: normal
stage: needs patch
status: open
title: Expose the subinterpreters C-API in Python for testing use.
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com