[issue15314] Use importlib instead of pkgutil in runpy

[Nick Coghlan]

Nick Coghlan  added the comment:

Buildbots are green (at least as far as this issue goes), so closing again.

I created #15384 for the surprising behaviour of FileFinder on Windows.

#15385 records my investigation into the odd symlink behaviour on 3.2 (it was 
just a discrepancy between the emulation in pkgutil and the real import system)

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Nick Coghlan]

Nick Coghlan  added the comment:

OK, I just pushed a tweak to the test that will hopefully resolve the problem. 
If the tweak works, though, it does suggest we have a bit of weirdness in 
importlib: it means pkgutil.get_importer("") is returning an importer on POSIX 
systems, but None on Windows.

The test tweak to get OS X passing for #15230 on 3.2 (where it was already 
passing on trunk) suggests we may also have a case where 3.2 called 
os.path.realpath() before setting __file__, while 3.3 does not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 16e50e85d684 by Nick Coghlan in branch 'default':
Issue #15314: Tweak a pkgutil test to hopefully be more Windows friendly
http://hg.python.org/cpython/rev/16e50e85d684

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Antoine Pitrou]

Antoine Pitrou  added the comment:

A test fails on the XP buildbots:



==
FAIL: test_get_importer_avoids_emulation 
(test.test_pkgutil.ImportlibMigrationTests)
--
Traceback (most recent call last):
  File "D:\Buildslave\3.x.moore-windows\build\lib\test\test_pkgutil.py", line 
289, in test_get_importer_avoids_emulation
self.assertIsNotNone(pkgutil.get_importer(sys.path[0]))
AssertionError: unexpectedly None

--
nosy: +pitrou
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Nick Coghlan]

Nick Coghlan  added the comment:

OK, I think runpy and __main__.__loader__ are all sorted now. The relevant 
runpy and command line script tests all check for the expected values, and I 
added a couple of new tests to cover the "-c" and "reading from stdin" cases.

--
resolution:  -> fixed
stage: needs patch -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 3987667bf98f by Nick Coghlan in branch 'default':
Take the first step in resolving the messy pkgutil vs importlib edge cases by 
basing pkgutil explicitly on importlib, deprecating its internal import 
emulation and setting __main__.__loader__ correctly so that runpy still works 
(Affects #15343, #15314, #15357)
http://hg.python.org/cpython/rev/3987667bf98f

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Nick Coghlan]

Nick Coghlan  added the comment:

Ah, __main__, thou art such a special beast.

Here's my current plan:

1. __main__ will be initialised with __loader__ set to BuiltinImporter. This 
covers cases like the interactive prompt or execution of stdin where there is 
no source code available for __main__

2. When executing a script (either directly or via runpy), __main__.__loader__ 
will be *changed* at a later point to refer to the appropriate kind of loader 
object.

--
priority: high -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Nick Coghlan]

Nick Coghlan  added the comment:

Oh, special - importlib.find_loader() currently expects every module to have a 
__loader__ attribute, but there's at least one that doesn't: __main__.

Guess I'll be tweaking it to handle that case more gracefully after all :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Nick Coghlan]

Nick Coghlan  added the comment:

OK, the way I plan to tackle this is to actually redirect the pkgutil APIs to 
importlib (rather than using the private import emulation).

The reason I'm going to tackle it like this is that there are some assumptions 
pkgutil deals with that importlib doesn't. Specifically, runpy expects 
find_loader() to take care of importing the parent package. In addition, 
importlib is prone to throwing AttributeError or TypeError for problems where 
pkgutil consistently throws ImportError. By using pkgutil as a compatibility 
shim, most code (including runpy itself) will be able to transition to 
importlib without actually needing any changes.

This approach also brings test_runpy and test_cmd_line_script more fully to 
bear on the importlib implementation, which is a nice bonus :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Éric Araujo]

Changes by Éric Araujo :


--
nosy: +eric.araujo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Brett Cannon]

Brett Cannon  added the comment:

Ooh, first "customer" of importlib.find_loader()! Obviously if the API doesn't 
work for you let me know since it's a 3.3 addition and can be tweaked to meet 
your needs.

--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15314] Use importlib instead of pkgutil in runpy

[Nick Coghlan]

New submission from Nick Coghlan :

There's some lingering weirdness in runpy due to its reliance on pkgutil's 
import emulation (see #15272).

For 3.3, I'd like to explicitly switch runpy over to using importlib. This was 
supposed to happen automatically, but the relevant API is 
"importlib.find_loader" rather than the "imp.get_loader" that runpy is 
currently looking for.

--
assignee: ncoghlan
messages: 165153
nosy: georg.brandl, ncoghlan
priority: high
severity: normal
stage: needs patch
status: open
title: Use importlib instead of pkgutil in runpy
type: behavior
versions: Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com