[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Change by Dino Viehland : -- pull_requests: +17722 pull_request: https://github.com/python/cpython/pull/18347 ___ Python tracker ___

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Roundup Robot added the comment: New changeset 3a35638bce66 by Ned Deily in branch 'default': Issue #17636: Install new test directories. https://hg.python.org/cpython/rev/3a35638bce66 -- ___ Python tracker rep...@bugs.python.org

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Antoine Pitrou added the comment: Actually, looking up __package__ would be wrong. Say I have: from pack.module import foo and foo doesn't exist in pack.module but exists in pack. Since pack.module.__package__ == pack, using __package__ would wrongly find the foo in pack. --

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Roundup Robot added the comment: New changeset fded07a2d616 by Antoine Pitrou in branch 'default': Issue #17636: Circular imports involving relative imports are now supported. https://hg.python.org/cpython/rev/fded07a2d616 -- nosy: +python-dev ___

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Changes by Antoine Pitrou pit...@free.fr: -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Nick Coghlan added the comment: The language reference - at least the section on the import statement, and potentially the section on the overall operation of the import system. I don't *think* it would affect anywhere in the tutorial or the importlib docs, but they may be worth skimming to

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Antoine Pitrou added the comment: By the way, my patch is currently using the package's __name__ attribute to build the fully-qualified name. Should it use the package's __package__ attribute instead? -- ___ Python tracker rep...@bugs.python.org

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Nick Coghlan added the comment: If I understand your question correctly, then yes, __package__ should be used when converting explicit relative imports to absolute ones. To write a test, run a submodule that needs the new fallback feature via the -m switch (it will fail if using __name__)

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Antoine Pitrou added the comment: Le 10/10/2014 00:11, Nick Coghlan a écrit : Nick Coghlan added the comment: If I understand your question correctly, then yes, __package__ should be used when converting explicit relative imports to absolute ones. To write a test, run a submodule that

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Antoine Pitrou added the comment: By the way, is there any documentation that would need to be updated for this? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Changes by Antoine Pitrou pit...@free.fr: -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___ ___ Python-bugs-list

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Changes by Antoine Pitrou pit...@free.fr: -- stage: test needed - needs patch versions: +Python 3.5 -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Antoine Pitrou added the comment: This is Brett's tests patch updated against default branch. -- nosy: +pitrou Added file: http://bugs.python.org/file36803/import_from_tests2.diff ___ Python tracker rep...@bugs.python.org

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Antoine Pitrou added the comment: And this is a full patch. -- stage: needs patch - patch review Added file: http://bugs.python.org/file36804/import_from_tests2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Changes by Antoine Pitrou pit...@free.fr: Added file: http://bugs.python.org/file36805/circrelimports.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Antoine Pitrou added the comment: (full patch is actually the latest upload: circrelimports.patch. Sorry for the glitch) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Nick Coghlan added the comment: With PEP 451 implemented, note that I have reopened issue 992389 - the patch to set the parent module attribute at the same time as setting the sys.module attribute is actually pretty trivial for the PEP 451 loader case, and that now includes all the standard

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

PJ Eby added the comment: Unfortunately, this is not quite true: the weird edge cases for that approach are simply *different*, and harder to diagnose. ;-) The approach here can only affect execution paths that would currently raise ImportError; that one can break execution paths that are

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Kristján Valur Jónsson added the comment: sorry, the last example really needs to be: #module foo.B A = __import__(.A, fromlist=[dummy]) to invoke the return final module behaviour. Hm, maybe this simply works... I didn't test Nope, I get ValueError: Empty module name (in 2.7) --

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Kristján Valur Jónsson added the comment: One question. when doing Programmatic import, i.e. using the __import__ function, how do we do the same? #module foo.B A = __import__(., fromlist=[A]).A #module foo.A B = __import__(., fromlist=[B]).B Clearly the above won't work. Can we extend

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Brett Cannon added the comment: You use importlib.import_module to do programmatic imports (and FYI dots never go into the first argument of __import__). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Kristján Valur Jónsson added the comment: Interesting. And I can report that it works, in 2.7, with code such as b = importlib.import_module(..b, __name__) and a = importlib.import_module(..a, __name__) Still, it seems odd that a whole importlib is requried ot resolve the relative import, and

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Brett Cannon added the comment: If you look at the importlib source code in 2.7 it's actually really small and simply translates the call into an __import__ call under the hood. So __import__ can completely handle relative imports, but you were not using the level argument to __import__ nor

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Brett Cannon added the comment: Don't be distracted when trying to write tests is the lesson learned. Fixed basic and rebinding and just deleted indirect. -- Added file: http://bugs.python.org/file30132/import_from_tests.diff ___ Python tracker

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Phillip J. Eby added the comment: It looks like maybe basic2 should be importing basic, not basic2. Otherwise, you might as well just import basic2 directly and be done with it. ;-) Likewise, I think indirect should be importing from indirect2, not from itself? i.e., I'd expect that test to

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Brett Cannon added the comment: I have uploaded a patch with failing tests that should work after this is all said and done. Philip, please make sure I covered your tests as expected (I tweaked one because it already was working the way I did it). This way we at least know what we are aiming

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Changes by Pascal Chambon chambon.pas...@gmail.com: -- nosy: +Pascal.Chambon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___ ___

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Nick Coghlan added the comment: Thanks for working through that Phillip - falling back to sys.modules when the expected attribute was missing is actually something I suggested as a possibility years ago, and Guido's response at the time was If it was that easy, someone would have done it

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Phillip J. Eby added the comment: On Sun, Apr 14, 2013 at 3:51 AM, Nick Coghlan rep...@bugs.python.org wrote: Your analysis is one of the pieces that was missing, Unfortunately, I just noticed it's actually incorrect in a pretty important part In my original example, I said, because of the

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Changes by Martin Morrison m...@ensoft.co.uk: -- nosy: +isoschiz ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___ ___ Python-bugs-list

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Changes by Phil Connell pconn...@gmail.com: -- nosy: +pconnell ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___ ___ Python-bugs-list

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Kristján Valur Jónsson added the comment: Like I mentioned on python-dev, it worries me a bit that this could be considered unexpected, i.e. when there is a conflict between a legitimage attribute member of a module, and a submodule of the same name. Also, I wonder if this isn't a bigger

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Brett Cannon added the comment: It won't conflict with attributes. Only if the attribute does not exist on the module already will it fall back to sys.modules. If the import finished then any attribute created from the import will already be there and thus not be an issue. But to make sure

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Kristján Valur Jónsson added the comment: While I'm happy that this is being ackowledged as a problem, I'm not sure changing the import x from y semantics is necessarily a good idea. I mean, it is obvious to me that it means import x, then getattr(x, y). I presume that is the meaning most

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Brett Cannon added the comment: By declaring what the semantics will be to make the case even possible we are not breaking anything but making something possible. IOW it isn't even an edge case to me since there is no working case to compare against. =) --

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Changes by Barry A. Warsaw ba...@python.org: -- nosy: +barry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___ ___ Python-bugs-list mailing

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Phillip J. Eby added the comment: On Fri, Apr 5, 2013 at 9:07 AM, Kristján Valur Jónsson rep...@bugs.python.org wrote: But I can think of contrived examples where this could break things: #a.py: from .b import util #b.py from . import a from .util import util Because of the circular

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Phillip J. Eby added the comment: Actually, after a little reflection, I can see that there are more complex conditions to analyze, if 'b' doesn't import 'b.util', but some other module imports b and sets b.util. But that's just freaking insane and whoever does that probably deserves whatever

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Phillip J. Eby added the comment: ...and I thought of *one* more way to trigger the changed behavior, which looks like: #b.py from .b import util import .a util = util.util #b/util.py def util(): pass (with the other files the same as before). I'm including it only for completeness' sake,

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Kristján Valur Jónsson added the comment: Wow, Good analysis Phillip. So, we agree that the fallback still is a sensible fallback? Then everything is fine and I'll put my official +1 stamp of approval on this. Now... should we consider the current behavious to be a bug? 2.7 sure could do

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Phillip J. Eby added the comment: I don't care much one way or the other about it being considered a bug in 2.x, but it might be worth considering a bug in 3.x. Either way, though, the language reference for from import should reflect the change, and alternative implementations should be

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Brett Cannon added the comment: It is not a bug but a change in semantics to accommodate a use-case so this will only be going into Python 3.4. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

New submission from Brett Cannon: To help with circular imports, IMPORT_FROM should be modified to check sys.modules if a getattr() check fails. http://mail.python.org/pipermail/python-dev/2013-April/125123.html -- components: Interpreter Core messages: 186059 nosy: brett.cannon, pje

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Changes by INADA Naoki songofaca...@gmail.com: -- nosy: +naoki ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___ ___ Python-bugs-list

[issue17636] Modify IMPORT_FROM to fallback on sys.modules

Changes by Eric Snow ericsnowcurren...@gmail.com: -- nosy: +eric.snow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17636 ___ ___ Python-bugs-list