[issue13959] Re-implement parts of imp in pure Python

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 672b6baa71010f236ee8c8ce912e98cb542385c6 by Victor Stinner in 
branch 'master':
bpo-32030: pass interp to _PyImport_Init() (#4736)
https://github.com/python/cpython/commit/672b6baa71010f236ee8c8ce912e98cb542385c6


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +4640

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset efb5e6ab10f4 by Brett Cannon in branch 'default':
Issue #15167 (as part of #13959): imp.get_magic() is no implemented in
http://hg.python.org/cpython/rev/efb5e6ab10f4

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Since the final issue is just a nicety, I am considering this issue closed.

--
dependencies:  -Deprecate imp.find_module()/load_module()
resolution:  - fixed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Changes by Brett Cannon br...@python.org:


--
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

hurray!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Georg Brandl]

Georg Brandl ge...@python.org added the comment:

Moving back to blocker for beta2.

--
priority: deferred blocker - release blocker

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Sorry, been on vacation the past week.

Georg is right, nothing left is a b1 blocker, just stuff I want to get in 
before 3.3 ships.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Changes by Brett Cannon br...@python.org:


--
dependencies: +Implement imp.get_tag() using sys.implementation

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Changes by Brett Cannon br...@python.org:


--
dependencies: +Re-implement imp.get_magic() in pure Python

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Georg Brandl]

Georg Brandl ge...@python.org added the comment:

OK, sounds like none of it would block beta1.

--
priority: release blocker - deferred blocker

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Georg Brandl]

Georg Brandl ge...@python.org added the comment:

Which parts are still missing here?

--
nosy: +georg.brandl

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

Here are the things I'm aware of:

* implement imp.get_tag() using sys.implementation
* *maybe* implement imp.get_magic() in pure Python (patch already attached here)
* (low priority) find a way to rip out the bulk of 
PyImport_ExecCodeModuleObject() from imp.c, if practical

From my discussions with Brett, there was a laundry list he was aiming to get 
in for 3.3 (for import stuff in general).  I know much of it got done, but not 
all of it.  Of the remainder, I'm not sure how much of that he still wants to 
see get in.  Some of it pertains to this issue, though not all.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Changes by Eric Snow ericsnowcurren...@gmail.com:


Added file: http://bugs.python.org/file26058/issue13959_get_tag.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 034c814eb187 by Brett Cannon in branch 'default':
Issue #13959: Add to imp.find_module() and load_module's docstrings
http://hg.python.org/cpython/rev/034c814eb187

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[John Ehresman]

John Ehresman j...@wingware.com added the comment:

This may be a known problem, but imp.load_module fails when trying to load a C 
extension.  This is with the 3.3a4 release on Windows, though I suspect the 
problem is cross-platform.

--
nosy: +jpe

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Does it work in Python 3.2, John? If it does then it's just an oversight thanks 
to the lack of tests in test_imp and it shouldn't be too difficult to support.

But do realize I have deprecated the function. =)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[John Ehresman]

John Ehresman j...@wingware.com added the comment:

On 6/5/12 1:08 PM, Brett Cannon wrote:

 Brett Cannonbr...@python.org  added the comment:

 Does it work in Python 3.2, John? If it does then it's just an oversight 
 thanks to the lack of tests in test_imp and it shouldn't be too difficult to 
 support.

 But do realize I have deprecated the function. =)

Attached is a short test file to demonstrate this.  It assumes the 
standard win32 layout, but shouldn't be hard to modify.  It does work in 
Python 3.2, but the else: clause in 3.3's imp.load_module function 
raises an ImportError.  I think the fix is to add an elif C_EXTENSION: 
clause that loads the .so / .pyd.

I've already rewritten my code to use importlib when running in Python 3.3.

--
Added file: http://bugs.python.org/file25834/imptest.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___import imp
import importlib
import sys
dirname = r'c:\Python33\dlls'

name = '_bz2'
assert name not in sys.modules

use_imp = True
if use_imp:
fp, pathname, description = imp.find_module(name, [dirname])
mod = imp.load_module(name, fp, pathname, description)
else:
loader = importlib.machinery.PathFinder.find_module(name, [dirname])
mod = loader.load_module(name)
print (mod)
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

You're right about the solution, John. I (or someone else) just needs to code 
it up.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Need to update the docstrings for imp.find_module() and load_module() to 
mention the functions are deprecated since there is no specific raised 
deprecation, only documented deprecation.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Changes by Brett Cannon br...@python.org:


--
dependencies: +Deprecate imp.find_module()/load_module()

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 59870239813c by Brett Cannon in branch 'default':
Issue #13959: Document imp.find_module/load_module as deprecated.
http://hg.python.org/cpython/rev/59870239813c

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 7bf8ac742d2f by Brett Cannon in branch 'default':
Issue #13959: Introduce importlib.find_loader().
http://hg.python.org/cpython/rev/7bf8ac742d2f

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

I have importlib.find_loader() coded up, but getting rid of find_module() is 
going to be a pain thanks to pkgutil, multiprocessing.forking, modulefinder, 
and idlelib.EditorWindow.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset b81ddaf0db47 by Brett Cannon in branch 'default':
Issue #13959: Deprecate imp.get_suffixes() for new attributes on
http://hg.python.org/cpython/rev/b81ddaf0db47

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

Question on this one:

snip
@@ -126,7 +131,7 @@ def load_compiled(name, pathname, file=N
 # XXX deprecate
 def load_package(name, path):
 if os.path.isdir(path):
-extensions = _bootstrap._SOURCE_SUFFIXES + 
[_bootstrap._BYTECODE_SUFFIX]
+extensions = machinery.SOURCE_SUFFIXES[:] + 
[machinery.BYTECODE_SUFFIXES]
 for extension in extensions:
 path = os.path.join(path, '__init__'+extension)
 if os.path.exists(path):
/snip

Should that be the following?

  extensions = machinery.SOURCE_SUFFIXES[:] + machinery.BYTECODE_SUFFIXES[:]

Also, why the [:]?

Finally, in a couple spots you use the first element of the list (like the old 
case of machinery.SOURCE_SUFFIXES[0] in source_from_cache() and the new one 
in find_module()).  Will this be a problem where the source file has one of the 
other suffixes?  I'm not sure it's a big enough deal for the moment to worry 
about, but thought I'd ask.  :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Yes.

And the [:] copies the list so I don't accidentally mutate in-place (did that 
once already, so now I'm just being paranoid; I doubt I need it hardly anywhere 
I don't do +=).

As for using SOURCE_SUFFIXES[0], I'm done following the ported code. I really 
don't care if .pyw isn't supported in that case. But in general people 
shouldn't assume just .py (or .py at all).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 626d5c6fbd95 by Brett Cannon in branch 'default':
Issue #13959: Have
http://hg.python.org/cpython/rev/626d5c6fbd95

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 257cbd2fac38 by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.get_suffixes() in Lib/imp.py.
http://hg.python.org/cpython/rev/257cbd2fac38

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

OK, I'm waiting on issue #14657 (avoiding the _frozen_importlib/importlib 
dichotomy) is resolved before I document the new imp.extension_suffixes() as it 
would be good to know where I should pull in the source and bytecode suffixes.

I think this only leaves get_magic() and get_tag() as the only things to move + 
trying to figure out how to handle PyImport_ExecCodeModuleObject() and its grip 
on various bits of C code.

--
dependencies: +Avoid two importlib copies

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 22b0689346f9 by Brett Cannon in branch 'default':
Issue #13959: Move module type constants to Lib/imp.py.
http://hg.python.org/cpython/rev/22b0689346f9

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Benjamin Peterson]

Benjamin Peterson benja...@python.org added the comment:

Windows is currently failing test_imp:

http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/214/steps/test/logs/stdio

--
nosy: +benjamin.peterson

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

That test is going to stay intermittent until issue #14657 gets resolved else 
the exact reason for the failure is going to be hard to debug remotely.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset eb5c5c23ca9b by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.NullImporter in Lib/imp.py.
http://hg.python.org/cpython/rev/eb5c5c23ca9b

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Update time!

With NullImporter dealt with, that leaves get_magic(), get_tag(), reload(), and 
get_suffixes() as things to potentially move to Lib/imp.py. I would also like 
to re-implement PyImport_ExecCodeModuleObject() as it's keeping a lot of C code 
alive just for its personal use.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

1659 lines less than 3.2’s import.c so far!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset eb68502731dd by Brett Cannon in branch 'default':
Issues #13959, 14647: Re-implement imp.reload() in Lib/imp.py.
http://hg.python.org/cpython/rev/eb68502731dd

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

So here is the deal with PyImport_ExecCodeModuleObject(): bootstrapping and 
Barry has made this a little annoying. =)

First off, PyImport_ImportFrozenModuleObject() uses 
PyImport_ExecCodeModuleObject(), so that precludes just directly importing imp 
to handle this function entirely. OK, so that means just trying to chop out the 
path manipulation stuff since that is duplicating code found in imp/importlib.

The problem, though, is that PyImport_ExecCodeModuleWithPathnames() will take 
its pathname argument and try to get a source path from it if it points to some 
.pyc file (PEP 3147 or sourceless .pyc, and if that new path exists then it is 
used instead of the given path. Unfortunately that API was introduced in Python 
3.2, so there is a backwards-compatibility issue in that one can't just rip out 
the code w/o supporting it. But those semantics are the reason the equivalent 
of imp.source_from_cache() continues to exist in Python/import.c.

I see two options here. One is to simply leave the C code in, but that has the 
drawback of duplicated Python and C code. Two is to stick in a call to 
imp.source_from_cache() between PyImport_ExecCodeModuleWithPathnames() and 
PyImport_ExecCodeModuleObject() so the former retains the semantics and the 
latter doesn't pick up the bad habit before 3.3 is released.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

updated patch for magic number support in imp/importlib

--
Added file: http://bugs.python.org/file25335/issue13959_magic.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

updated patch for moving TAG to importlib/imp.py

--
Added file: http://bugs.python.org/file25336/issue13959_tag.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

Ported _imp.reload() (Python/import.c) to Lib/imp.py.  Included is the change 
to PyImport_ReloadModule() to make it simply a wrapper around the pure Python 
imp.reload().  There's a good chance I don't have this right or that I have 
some reference leak.  I haven't worked a ton on the C side of Python (sounds 
tropical).  This patch also removes 'modules_reloading' from the interpreter 
state, since it's no longer used anywhere (see issue14618).

--
Added file: http://bugs.python.org/file25302/issue13959_reload.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

A patch for magic and tag.  It's not quite finished, but I wanted to see if the 
approach was palatable.  FYI, I'm also trying to push forward the 
sys.implementation stuff, which would help on the pyc tag.

--
Added file: http://bugs.python.org/file25303/issue13959_magic.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

After thinking about it, is MAGIC an implementation detail?  It certainly 
reflects changes specific to the CPython interpreter.  I'm much more 
comfortable with leaving implementation details in Python/import.c.

On the other hand, there's already no small amount of rather CPython-specific 
stuff in Lib/importlib/_bootstrap.py, which belongs there.  The pyc magic bytes 
are tightly coupled with it.  Because of that, I realize, I'm still fine with 
the patch.

But it makes me wonder if it might be worth having a clear separation between 
the general and CPython-specific stuff in _bootstrap.py, for the sake of people 
who look at the code for the first (or tenth) time.  That's the same rationale 
I have for advocating moving as much over from import.c as relates to the 
importlib implementation.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 4e853913054c by Brett Cannon in branch 'default':
Issue #13959: Continue to try to accomodate altsep in importlib by not
http://hg.python.org/cpython/rev/4e853913054c

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

First off, you should separate the patches for get_magic() and get_tag(). 
Second, why is there _get_pyc_magic_int() when it is never called? Third, all 
of this would be greatly simplified if you just had a _RAW_MAGIC_NUMBER of 
3220, did the bytes object creation for _MAGIC_NUMBER in-place (i.e. no 
separate function), and then in the C code just got _RAW_MAGIC_NUMBER and did 
the MAGIC macro work there.

As for what is CPython-specific and what isn't, only the other VMs can state 
that officially, so I'm not going to worry about that yet (but I will ask 
before Python 3.3 goes out so as to minimize backporting patches in the 
future). But importlib needs to stabilize more before that can happen.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

Good feedback.  The some of that code was the result of directly translating 
the C.  I'll get a new, simpler patch up probably tomorrow night.  Thanks.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

How consistent do the semantics of reload() need to remain?  (The C version 
does more type checking than the Python version probably needs to worry about.  
reload() seems to be one of those bits that doesn't have much test coverage.)

Also, what's the best way to exercise the changes one makes to the C code?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Loosening the type restrictions is fine.

As for testing, that's what the test suite is supposed to do. So if need be 
just write tests in Python that exercise the C code.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

To try and narrow down the issue, I now have separate patches for an explicit 
sys.meta_path and a sys.path_hooks. Both fail on tests, but for different 
reasons.

The explicit sys.meta_path is still failing on the __main__ issue on three of 
the tests and for threaded imports on the other.

The explicit sys.path_hooks patch fails on two tests: one on threading and  
then test_cmd_line_script on apparently lacking an absolute file path thanks to 
issue #8202 (and why an explicit sys.path_hooks triggers that I don't know).

Both failures Nick might know about.

--
Added file: http://bugs.python.org/file25308/explicit_meta_path.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Changes by Brett Cannon br...@python.org:


Added file: http://bugs.python.org/file25309/explicit_path_hooks.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset b773a751c2e7 by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
http://hg.python.org/cpython/rev/b773a751c2e7

New changeset ea46ebba8a0f by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.source_from_cache() in Lib/imp.py.
http://hg.python.org/cpython/rev/ea46ebba8a0f

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Time for a recap!

I am personally not bothering with moving imp.get_tag() and imp.get_magic() to 
importlib._bootstrap as the amount of C code required to support the public C 
API is not worth it. If someone else once to do the work then by all means 
attach a patch.

I still need to port imp.find_module() (and the various constants) and will 
base it off of Eric's code.

NullImporter will get ported once issue #14605 (exposing the import machinery) 
lands.

get_suffixes() will also get ported, but that is a little bit more involved as 
I need to change how _DynLoadFiletab works by only storing the file extensions 
and not the other fluff (which is the same for all OSs).

After all of that is done then I will expose some API in importlib to replace 
find_module() and load_*() functions and then deprecate them (see issue #14551 
for an ongoing discussion the possible API).

--
assignee:  - brett.cannon
dependencies: +Make import machinery explicit
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

Yeah, I'm hoping to keep pressing those odds and ends forward.  I have one 
lingering, oddball bug in find_module, but that patch is pretty much standing 
on its own.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 085cf1480cfe by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.find_module() in Lib/imp.py.
http://hg.python.org/cpython/rev/085cf1480cfe

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset c820aa9c0c00 by Brett Cannon in branch 'default':
Issue #13959: Keep imp.get_magic() in C code, but cache in importlib
http://hg.python.org/cpython/rev/c820aa9c0c00

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

I'd still like to consider this a bit more.  When you're trying to understand 
imports, having one place to look (Lib/importlib/_bootstrap.py) is better than 
two, especially when the one is pure Python code.  So it still may be worth it 
to pull in the odds and ends that play into that.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

Looking it over, I'm confident that tokenizer.detect_encoding() does not raise 
a SyntaxError where PyTokenizer_FindEncodingFilename() does.  I've run out of 
time tonight, but I'll look at it more tomorrow.

Once find_module() is done, I'd like to move on to reload(), which I expect 
will be pretty straightforward at this point.  Then the feasibility of 
issue14618 should be clear.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

You could change Lib/imp.py to have ``import _frozen_importlib as _bootstrap`` 
if you want the same modules coming from imp, but I would argue against 
changing importlib itself being changed as that complicates development since 
if you screw up importlib._bootstrap when you compile it becomes a major pain 
to revert the importlib.h change, recompile, and continue to do that until you 
get it right. Plus you would only care about this if you are doing isinstance() 
checks on what kind of loader you have which you shouldn't care about since we 
have clearly defined ABCs to test against.

As for Lib/test/badsyntax_pep3120.py, it *does* have a source encoding of UTF-8 
since it does not explicitly specify an encoding. Based on the name I'm 
assuming the file itself has bad UTF-8 characters, but that doesn't mean that 
the file is not supposed to be interpreted as UTF-8.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

On the _frozen_importlib point, I'm fine with that.  It was just 
counter-intuitive that the importlib._bootstrap functions were returning 
loaders whose classes weren't also defined there.

I'll have another look at that test tonight and run the patch through the full 
test suite, but otherwise I think find_module() is basically done.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

I doubt I will beat you to it, Eric, but I did want to say that your overall 
design was what I had in my head when I was thinking about how to re-implement 
the function, so keep at it!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 66bd85bcf916 by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.load_compiled() in imp.py.
http://hg.python.org/cpython/rev/66bd85bcf916

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

2 problems with that last patch:

* the types of the loaders that get returned by _bootstrap._find_module() are 
not the classes in _bootstrap (e.g. _frozen_importlib._SourceFileLoader).  That 
doesn't smell right.
* tokenize.get_encoding? is saying that Lib/test/badsyntax_pep3120.py has an 
encoding of utf-8, when test_find_module_encoding is expecting it to not.  So 
does PyTokenizer_FindEncodingFilename (which the old import code used) behaving 
differently than tokenize.get_encoding()?  FYI, tokenize.get_encoding() is what 
importlib uses...

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

rather, tokenize.detect_encoding()

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Berker Peksag]

Changes by Berker Peksag berker.pek...@gmail.com:


--
nosy: +berker.peksag

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

From Eric Smith on python-dev:

 +suffix, mode, type_ = details
 +if mode and (not mode.startswith(('r', 'U'))) or '+' in mode:
 +raise ValueError('invalid file open mode {!r}'.format(mode))

Should this be:
if mode and (not mode.startswith(('r', 'U')) or '+' in mode):

to match:

 -if (*mode) {
...
 -if (!(*mode == 'r' || *mode == 'U') || strchr(mode, '+')) {
 -PyErr_Format(PyExc_ValueError,
 - invalid file open mode %.200s, mode);

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 3b5b4b4bb43c by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.load_source() in imp.py.
http://hg.python.org/cpython/rev/3b5b4b4bb43c

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

This is a mostly-working sketch of find_module() in imp.py.  I've run out of 
time tonight, but wanted to get it up in case it's useful to anyone else (a.k.a 
Brett).  If nobody's touched it before then, I'll finish it up tomorrow.

--
Added file: http://bugs.python.org/file25250/imp_find_module.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Just because I was thinking about it, I wonder if necessarily all the frozen 
stuff really needs to stay in import.c. I mean a frozen module is really just 
an entry in an array of structs that has a name of an char*[]. I don't see why 
one couldn't simply have a get_frozen_bytes() method to convert that char*[] 
into a bytes object and use that to construct a module in pure Python code.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset d777f854a66e by Brett Cannon in branch 'default':
Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
http://hg.python.org/cpython/rev/d777f854a66e

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

OK, so I have started to check this stuff in, but I think it's best to do it 
piecemeal. Going forward I would like to commit in units of functions being 
replaced, and prioritize stuff that cuts out C code (e.g. the load_*() methods, 
find_module(), etc.). That way it's clear that progress is being made. 
Obviously the best way to tell if code is hanging on just because of imp is to 
comment out the interface code and see what your compiler complains about in 
terms of dead code (or at least clang is good at this).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

It looks like in order to get a clear sign of what it will take to remove 
various parts of import.c, imp.load_module() needs to go along with 
imp.load_package() (since they call each other in the C code). You also have to 
take care of imp.reload(), but I am simplifying the C code greatly and will 
take care of referencing other code in the module.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 4dce3afc392c by Brett Cannon in branch 'default':
Issue #13959: Simplify imp.reload() by relying on a module's
http://hg.python.org/cpython/rev/4dce3afc392c

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

I am seeing how this is going to go down. the load_dynamic, load_source, etc. 
family of functions are simply dispatched to by load_module(). So to keep some 
semblance of backwards-compatibility, each of those modules need to be 
implemented and then have load_module() simply dispatch to them based on the 
type of module it is.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 2df37938b8e1 by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.load_module() in imp.py.
http://hg.python.org/cpython/rev/2df37938b8e1

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 4256df44023b by Brett Cannon in branch 'default':
Issue #13959: Re-implement imp.load_package() in imp.py.
http://hg.python.org/cpython/rev/4256df44023b

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

I think this should be a blocker for 3.3.

--
nosy: +pitrou
priority: normal - release blocker

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

(Éric, I'd reply in the review if reitveld recognized my current username -- 
see http://psf.upfronthosting.co.za/roundup/meta/issue402.  FYI, I also did not 
get an email for your review, which is likely related to that issue.)

Regarding __all__, I didn't realize that about being a tuple.  I'll fix it. 

As to the dm in the SOABI value, that is definitely something we'll have to 
factor in.  There are actually a few such places in the current patch that rely 
on problematic data.   It's either hidden in C or not available during 
bootstrapping.  That's something we'll have to iron out before the solution for 
this issue gets committed.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

Of note for my patch is the addition of SUFFIXES and MODE to the 3 main file 
loader classes.  I did this to reverse the dependence on imp.get_suffixes().

As well, a couple of extra functions are added to 
Python/importlib/_bootstrap.py for a similar reason, but should also be 
meaningful externally in their own right.

load_module_from_file(), is one of these, though the current incarnation relies 
on a mythical new method on loaders that may not pass muster.

(FWIW, I accidently included the use_of_imp.txt file in the patch and don't 
intend it to get included)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

 Once things all the import stuff currently on the table is wrapped up,
 I'll probably work on switching things over to using importlib directly.

Please leave distutils unchanged; we don’t clean it up or improve it in any 
way, it’s bugfixes only.  Changing packaging is fine; I’ll just keep using the 
imp module in the distutils2-python3 backport.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

Sounds good.  It will be a while before we get there, but at that point I was 
already planning on getting in touch with the maintainers of relevant modules 
before messing with them.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

Here's an incomplete (though essentially test-passing) patch that flips more of 
imp into importlib.  It builds on Brett's patch.  The gist is that imp.py is a 
minimal wrapper around importlib, which uses _imp directly.

Ultimately the things implemented in Lib/importlib/_bootstrap.py should be 
yanked out of Python/import.c (and Python/dynload_*.c).  I expect that this 
will wait for the bootstrap merge (#2377) to land.

--
Added file: http://bugs.python.org/file24979/imp.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

Here's a listing of the places where import-ish modules are used in CPython.  
Once things all the import stuff currently on the table is wrapped up, I'll 
probably work on switching things over to using importlib directly.

--
Added file: http://bugs.python.org/file24980/use_of_imp.txt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Attached is a start for re-implementing imp. Still need code for 
cache_from_source, source_from_cache, find_module, load_module, reload, 
load_compiled, load_source, and load_package.

--
keywords: +patch
Added file: http://bugs.python.org/file24846/imp.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Changes by Brett Cannon br...@python.org:


--
nosy: +eric.snow

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Meador Inge]

Changes by Meador Inge mead...@gmail.com:


--
nosy: +meador.inge

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
nosy: +eric.araujo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

New submission from Brett Cannon br...@python.org:

A bunch of code in Python/import.c exists purely for the imp module, but a 
large chunk of it does not need to be implemented in C but instead can be 
implemented in Python code.

Once importlib is bootstrapped in then an attempt to scale back the C code 
should be done by re-implementing parts of imp in pure Python (either in some 
_imp module or directly in _importlib itself).

--
components: Library (Lib)
messages: 152800
nosy: brett.cannon
priority: normal
severity: normal
status: open
title: Re-implement parts of imp in pure Python
versions: Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13959] Re-implement parts of imp in pure Python

[Brett Cannon]

Changes by Brett Cannon br...@python.org:


--
dependencies: +Replace __import__ w/ importlib.__import__

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13959
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com