[issue34200] importlib: python -m test test_pkg fails semi-randomly

[STINNER Victor]

STINNER Victor  added the comment:

Thank you for the fix Gregory!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[Gregory P. Smith]

Gregory P. Smith  added the comment:

I still feel like there is an underlying issue within the import system that 
use of test.support.modules_cleanup somehow triggers that I never managed to 
really understand.

but the particular issue in this bug that people were seeing frequently is 
fixed.

--
resolution:  -> fixed
stage: patch review -> commit review
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[miss-islington]

miss-islington  added the comment:


New changeset 3e3d4a4b55cbef481486aaa68ee255f4928b9c3f by Miss Islington (bot) 
in branch '3.6':
bpo-34200: Fix non-determinism of test_pkg (GH-9248)
https://github.com/python/cpython/commit/3e3d4a4b55cbef481486aaa68ee255f4928b9c3f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[miss-islington]

miss-islington  added the comment:


New changeset 90f7d455b7f4bd53d3af11cb951347c9c8230399 by Miss Islington (bot) 
in branch '3.7':
bpo-34200: Fix non-determinism of test_pkg (GH-9248)
https://github.com/python/cpython/commit/90f7d455b7f4bd53d3af11cb951347c9c8230399


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[miss-islington]

miss-islington  added the comment:


New changeset 4ae8ece5cd4c5853b625381db13429f25512108d by Miss Islington (bot) 
(Gregory P. Smith) in branch 'master':
bpo-34200: Fix non-determinism of test_pkg (GH-9248)
https://github.com/python/cpython/commit/4ae8ece5cd4c5853b625381db13429f25512108d


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[miss-islington]

Change by miss-islington :


--
pull_requests: +8684

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[miss-islington]

Change by miss-islington :


--
pull_requests: +8683

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[Gregory P. Smith]

Gregory P. Smith  added the comment:

this is also present in 3.5, but it'll be up to the 3.5 release manager to 
cherry pick the test_pkg.py reliability fix into the 3.5 branch if it impacts 
them during the release process.

--
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[Jeremy Kloth]

Jeremy Kloth  added the comment:

Did you attempt to use the 3-line change I posted earlier?  I stepped through 
to test line-by-line to find the offending piece of code.  And it was indeed 
the open() call causing the test-tree to be processed prior to it being 
completed.  Thus making the .py file seen before the directory of same name 
would exist.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[Gregory P. Smith]

Change by Gregory P. Smith :


--
keywords: +patch
pull_requests: +8679
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[Gregory P. Smith]

Gregory P. Smith  added the comment:

I can make the test reliable... but I wouldn't say I fully understand the 
ultimate cause of the problem.

The reliability fix for test_pkg is to stop using test.support.modules_setup() 
and test.support.modules_cleanup() in the setUp() and tearDown() methods.

these test support functions are semi scary.  they attempt to backup and 
replace sys.modules contents with special case code in the cleanup function to 
try and avoid doing that to things that are still necessary.

running python -vvv when I could get test_pkg to fail led me looking at code 
paths that were being executed mid-test that i'd expect to be executed only on 
process startup.  locale.getpreferredencoding triggering a _bootlocale import, 
etc.  I don't understand why it cause the problem though.

PR to at least make the test reliable coming.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[Gregory P. Smith]

Gregory P. Smith  added the comment:

kinda poking at ideas here, but from a lunch conversation could this be related 
to the filesystem iteration order within the temp directories.

assigned to me while i investigate possibilities.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[Gregory P. Smith]

Change by Gregory P. Smith :


--
assignee:  -> gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails semi-randomly

[Gregory P. Smith]

Gregory P. Smith  added the comment:

and I do agree that it is somewhat random.  while i have some situations where 
I can reproduce this, i have others where I can rerun exactly the same binary 
and single process single thread process running just test_pkg.py and nothing 
else where the behavior differs between runs in terms of which of test_4 and 
test_7 fail or not.

--
title: importlib: python -m test test_pkg fails -> importlib: python -m test 
test_pkg fails semi-randomly

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails

[Gregory P. Smith]

Gregory P. Smith  added the comment:

this also happens in 3.6.  whatever the problem is, it has been around a long 
time.

--
priority: release blocker -> deferred blocker
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34200] importlib: python -m test test_pkg fails

[Gregory P. Smith]

Gregory P. Smith  added the comment:

marking release blocker solely because this prevents CI runs from succeeding.

I'm surprised how long this issue has been here though, i don't know what 
caused it to start happening consistently but as with many such heisenbugs 
changes at a distance could be related. :/

--
title: importlib: python -m test test_pkg -m test_7 fails randomly -> 
importlib: python -m test test_pkg fails

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com