[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Nick Coghlan]

Nick Coghlan added the comment:

Just noting explicitly that this has been forward merged to the default branch 
by Steve Dower after Larry merged in the latest 3.5.0 rc changes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Roundup Robot]

Roundup Robot added the comment:

New changeset 087464c9f982 by Nick Coghlan in branch '3.5':
Close #24748: Restore imp.load_dynamic compatibility
https://hg.python.org/cpython/rev/087464c9f982

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Nick Coghlan]

Nick Coghlan added the comment:

3.5.0 PR for the change is at 
https://bitbucket.org/larry/cpython350/pull-requests/16

This *is* a regression (since it's an unintended behavioural change in 
imp.load_dynamic), but it's one with a relatively straightforward workaround, 
so it would also be reasonable to defer fixing it to 3.5.1.

On the other hand, the risk is low, since the only functional change is in 
imp.load_dynamic itself, rather than in the core import machinery (that calls 
the underlying loader methods directly rather than going through the imp module 
API).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Larry Hastings]

Changes by Larry Hastings :


--
assignee:  -> ncoghlan
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Larry Hastings]

Larry Hastings added the comment:

Pull request accepted.  I had to do it manually, as I got a Misc/NEWS merge 
conflict.  But from where I'm sitting it looks like Bitbucket understands the 
pull request was accepted and merged.

Please forward-merge.  Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Petr Viktorin]

Petr Viktorin added the comment:

So, if I understand correctly, the problem is that the new imp.load_dynamic in 
3.5.0b checks sys.modules, so if a module of the same name was loaded 
previously, it's only reloaded, skipping the create_module step.

This patch bypasses that check, so a new module is always loaded.

This brings this aspect of imp.load_dynamic to how it was in 3.4. However, I 
have no way to test if it fixes pywin32 or py2exe.

--
keywords: +patch
Added file: http://bugs.python.org/file40352/issue24748.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Brett Cannon]

Brett Cannon added the comment:

Assuming I didn't screw this up -- I'm new to the whole Windows development 
thing -- the new test along with the rest of the test suite pass under Windows 
10 on the default branch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Nick Coghlan]

Nick Coghlan added the comment:

Huh, for some reason the Rietveld diff missed the changes to _testmultiphase.c 
that are in the patch file.

The change looks good to me. The test adds a new "test.imp_dummy" module, 
imports that, and then ensures it can be replaced by using imp.load_dynamic to 
import "test.imp_dummy" from the _testmultiphase module instead.

The functional change itself is isolated to imp.load_dynamic, so no other code 
paths will be affected.

--
stage: test needed -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Larry Hastings]

Larry Hastings added the comment:

Anything happening with this?  We tag 3.5.0rc3 in about 36 hours.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Petr Viktorin]

Petr Viktorin added the comment:

Indeed. I don't have access to a Windows machine, but I will try reproducing 
the problem on another system.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[James Salter]

James Salter added the comment:

This also affects py2exe, which dynamically generates stub .pyc loaders inside 
a ZIP which then load .pyd modules outside the ZIP with the same name.

Windows 8, x64, visual studio 2015 enterprise.

It is simple enough to work around by doing a del sys.modules[modname] prior to 
the load_dynamic call, but obviously this goes against the documented behaviour.

--
nosy: +James Salter

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Nick Coghlan]

Nick Coghlan added the comment:

Given that both pywin32 *and* py2exe have hit this, even if it can't be 
resolved for 3.5.0, it would be highly desirable to address it for 3.5.1.

It's certainly intended that load_module() keep working as it has previously, 
rather than forcing people to switch to exec_module() in order to support 
extensions modules properly on Windows. Aside from general backwards 
compatibility expectations, there are still some things that load_module() can 
do that the new PEP 451/489 mechanisms can't handle yet.

--
nosy: +larry
priority: normal - deferred blocker

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Etienne Fortin]

Etienne Fortin added the comment:

I replaced:
import importlib.machinery
loader = importlib.machinery.ExtensionFileLoader(name, path)
return loader.load_module()

With:
import importlib.machinery
loader = importlib.machinery.ExtensionFileLoader(modname, filename)
spec = importlib.machinery.ModuleSpec(
name = modname,
loader = loader,
origin = filename,
loader_state = 1234,
is_package = False,
)
mod = loader.create_module(spec)
loader.exec_module(mod)

And it now works as advertised. Since load_module() is flagged as Deprecated, I 
believe no correction is necessary as the preffered way to load a module, with 
exec_module(), is working. 

I will do some more tests to be sure it's the case.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Steve Dower]

Steve Dower added the comment:

Did the behavior change for 3.4 after the updates? It seems unlikely.

Zach's right: we need a test to formally state the expectations here.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Etienne Fortin]

Etienne Fortin added the comment:

It is also possible that the root cause is related to Microsoft Windows Update 
2999226 and/or 3065987. The behavior was the same between 3.4 and 3.5 on a 
machine without these updates and is believed to have changed after the install 
of these updates.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Etienne Fortin]

Etienne Fortin added the comment:

No the behavior only changed for 3.5. 3.4 works just fine.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Zachary Ware]

Changes by Zachary Ware zachary.w...@gmail.com:


--
stage:  - test needed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Petr Viktorin]

Petr Viktorin added the comment:

Eric or me.

I'm not sure I understand the description clearly. Etienne, would it be 
possible to write a smaller reproducer, that wouldn't be tied to Windows?
Or is this Windows-only behavior?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Steve Dower]

Steve Dower added the comment:

3.5 won't work without 2999226 at all, so ignore that. It's the C runtime.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Etienne Fortin]

Etienne Fortin added the comment:

I suggest the test should use pywin32. The test script could be only:

import pywintypes
dir(pywintypes)

Testing for an attribute that is defined in the DLL would make it pass or fail.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Steve Dower]

Steve Dower added the comment:

I'd rather have our own test pyd that shows a minimal repro of the behavior. 
Requiring pywin32 for running one test is excessive, won't reveal the issue (if 
it exists) on other platforms, and is not necessarily going to help someone 
debug the issue.

I agree this is probably due to PEP 489, which means that Eric is almost 
certainly the best person to look into it. It may also be due to the list of 
suffixes changing for 3.5 (.cp35-win32.pyd is now a valid suffix as well as 
just .pyd, for example), but looking at pywintypes.py that seems unlikely.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Etienne Fortin]

Etienne Fortin added the comment:

The only dll / pyd files I have are all in pywin32. I don't have a build 
environment for extensions. Can anyone provide me with a very simple extension 
DLL with at least one exported attribute?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Etienne Fortin]

Etienne Fortin added the comment:

At this point I can't say if it's Windows only or if it affect all platform. I 
can't even guarantee that it will appear on all Windows platform. On my 
platform (see following), it doesn't work:

Windows 7 64 bits with updates 2999226 and 3065987 installed.

I can't confirm the update are related either. I just did a quick analysis and 
between the time 3.5 was working and wasn't anymore these updates were 
installed.

I will create a quick script to reproduce the behavior.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Etienne Fortin]

Etienne Fortin added the comment:

Is it possible that the C runtime introduced with 2999226, which I believe is 
the universal runtime Microsoft is trying to introduce, modified something 
that makes importlib break on 3.5???

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Steve Dower]

Steve Dower added the comment:

It's possible, but that isn't solvable or testable by removing the update. To 
solve that we need a test and then fix the bug.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Eric Snow]

Eric Snow added the comment:

This is most likely related to PEP 489, which changed extension module loading 
in what was meant to be a backward-compatible way.

--
nosy: +brett.cannon, encukou, eric.snow, ncoghlan
versions: +Python 3.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24748
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com