On Sat, Aug 20, 2016 at 2:25 AM, Steve Dower <steve.do...@python.org> wrote:
> On 19Aug2016 0910, Chris Angelico wrote:
>>
>> On Sat, Aug 20, 2016 at 1:26 AM, Steve Dower <steve.do...@python.org>
>> wrote:
>>>
>>> Check any .pth files you can find. I suspect mpl_toolkits has some magic
>>> in
>>> it to make the namespace package work on 2.7.
>>
>>
>> $ cat
>> /usr/local/lib/python3.6/site-packages/matplotlib-1.5.1-py3.6-nspkg.pth
>> import sys, types, os;p =
>> os.path.join(sys._getframe(1).f_locals['sitedir'],
>> *('mpl_toolkits',));ie =
>> os.path.exists(os.path.join(p,'__init__.py'));m = not ie and
>> sys.modules.setdefault('mpl_toolkits',
>> types.ModuleType('mpl_toolkits'));mp = (m or []) and
>> m.__dict__.setdefault('__path__',[]);(p not in mp) and mp.append(p)
>>
>> Is it possible that that's the cause? And if so, should "python -I"
>> remove /usr/local/lib/python3.6/site-packages from sys.path?
>
>
> That explains the extra modules - types.py imports collections and
> collections.abc, which would seem to account for the rest.
>
> I'm not sure whether that's causing the problem or not, but temporarily
> renaming that pth file so it isn't picked up may help you track down the
> test issue.
Sure enough, that did fix the problem - that is to say, renaming it
did allow test_site to pass. Thanks!
> "python -S" is the only way to avoid .pth files, but that's a pretty heavy
> hammer.
Hmmmmmmm. So the question is, what is this test testing? For
reference, here's the code:
class StartupImportTests(unittest.TestCase):
def test_startup_imports(self):
# This tests checks which modules are loaded by Python when it
# initially starts upon startup.
popen = subprocess.Popen([sys.executable, '-I', '-v', '-c',
'import sys; print(set(sys.modules))'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
stdout, stderr = popen.communicate()
stdout = stdout.decode('utf-8')
stderr = stderr.decode('utf-8')
modules = eval(stdout)
self.assertIn('site', modules)
# http://bugs.python.org/issue19205
re_mods = {'re', '_sre', 'sre_compile', 'sre_constants', 'sre_parse'}
# _osx_support uses the re module in many placs
if sys.platform != 'darwin':
self.assertFalse(modules.intersection(re_mods), stderr)
# http://bugs.python.org/issue9548
self.assertNotIn('locale', modules, stderr)
if sys.platform != 'darwin':
# http://bugs.python.org/issue19209
self.assertNotIn('copyreg', modules, stderr)
# http://bugs.python.org/issue19218>
collection_mods = {'_collections', 'collections', 'functools',
'heapq', 'itertools', 'keyword', 'operator',
'reprlib', 'types', 'weakref'
}.difference(sys.builtin_module_names)
self.assertFalse(modules.intersection(collection_mods), stderr)
It asserts that 'site' is in the list of loaded modules, so simply
adding '-S' to the subpython parameters doesn't work. But this test,
as written, depends on the behaviour of arbitrary third-party modules
installed with pip, which seems wrong for a standard library test.
AIUI this is trying to catch changes that accidentally cause large
modules to be loaded on startup, correct? Which is all well and good,
but Python shouldn't have test failures due to matplotlib being
installed.
I see two (or three) ways to deal with this.
1) Declare it a bug in matplotlib, and demand that .pth files not
import anything that wouldn't otherwise be imported.
2) Change the test to add '-S' and remove the assertion about 'site'
being present; this means that site.py itself is no longer covered by
the test.
3) Accept the status quo: this test can fail if .pth files are present.
Incidentally, where's it documented that .pth files can be executed?
It's not in the site.py docs [1] or docstring, only in the addpackage
function within that (which I found by browsing the source). Maybe the
solution is to disable that one specific feature when running this
test?
ChrisA
[1] https://docs.python.org/3.6/library/site.html
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
