[issue16027] pkgutil doesn't support frozen modules

[Irit Katriel]

Irit Katriel  added the comment:

Python 2 issue.

--
nosy: +iritkatriel
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Martin Panter]

Martin Panter added the comment:

I know I had a bit of trouble adding support to iter_modules() and related 
functions, see Issue 25533.

Also, Issue 21749 about ImpLoader came up in my search.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Side note, as already indicated by Nick: pkgutil may well still not support 
frozen modules in Python 3.4 and 3.5, since runpy, which pyrun uses, does not 
rely on pkgutil in Python 3.4 and 3.5.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

We're not supporting Python 3.3 with PyRun, but I can confirm that it works 
fine in Python 3.4 and 3.5.

In Python 2.7, it's still broken, as you can test with

pyrun2.7 -m wsgiref.simple_server

--
versions: +Python 2.7 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Martin Panter]

Martin Panter added the comment:

FWIW Python 3.3 is almost a distant memory, but I think it works properly:

$ wine c:/Python33/python.exe -m __hello__
Hello world!
$ wine c:/Python33/python.exe -m __phello__.spam
Hello world!
Hello world!

On Python 2, it does fail. Is this what the original problem was?

$ python2.7 -m __hello__
/sbin/python2.7: No code object available for __hello__
[Exit 1]
$ python2.7 -c 'import __hello__'
Hello world...

As I see it, this only needs to be fixed in Python 2. Or is it not applicable 
to 2?

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Eric Snow]

Changes by Eric Snow :


--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Nick Coghlan wrote:
> 
> Nick Coghlan added the comment:
> 
> Can you confirm this problem still exists on 3.3? The pkgutil emulation isn't 
> used by runpy any more - with the migration to importlib, the interface that 
> runpy invokes fails outright if no loader is found rather than falling back 
> to the emulation (we only retained the emulation for backwards compatibility 
> - it's a public API, so others may be using it directly).

That's difficult to test, since the Tools/freeze/ tool no longer works
in Python 3.3. I'll open a separate issue for that.

> I have a feeling that there may still be a couple of checks which are 
> restricted to PY_SOURCE and PY_COMPILED that really should be allowing 
> PY_FROZEN as well.

Same here.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 25 2012)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2012-10-29: PyCon DE 2012, Leipzig, Germany ...34 days to go
2012-10-23: Python Meeting Duesseldorf ... 28 days to go
2012-09-25: Released mxODBC 3.2.1 ... http://egenix.com/go31
2012-09-18: Released mxODBC Zope DA 2.1.0 ... http://egenix.com/go32

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Nick Coghlan]

Nick Coghlan added the comment:

Can you confirm this problem still exists on 3.3? The pkgutil emulation isn't 
used by runpy any more - with the migration to importlib, the interface that 
runpy invokes fails outright if no loader is found rather than falling back to 
the emulation (we only retained the emulation for backwards compatibility - 
it's a public API, so others may be using it directly).

I have a feeling that there may still be a couple of checks which are 
restricted to PY_SOURCE and PY_COMPILED that really should be allowing 
PY_FROZEN as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Here's the fix we're applying in pyrun to make -m imports work at least for 
top-level modules:

--- /home/lemburg/orig/Python-2.7.3/Lib/pkgutil.py  2012-04-10 
01:07:30.0 +0200
+++ pkgutil.py  2012-09-24 22:53:30.982526065 +0200
@@ -273,10 +273,21 @@ class ImpLoader:
 def is_package(self, fullname):
 fullname = self._fix_name(fullname)
 return self.etc[2]==imp.PKG_DIRECTORY

 def get_code(self, fullname=None):
+if self.code is not None:
+return self.code
+fullname = self._fix_name(fullname)
+mod_type = self.etc[2]
+if mod_type == imp.PY_FROZEN:
+self.code = imp.get_frozen_object(fullname)
+return self.code
+else:
+return self._get_code(fullname)
+
+def _get_code(self, fullname=None):
 fullname = self._fix_name(fullname)
 if self.code is None:
 mod_type = self.etc[2]
 if mod_type==imp.PY_SOURCE:
 source = self.get_source(fullname)

This makes runpy work for top-level frozen modules, but it's really only 
partial solution, since pkgutil would need to get such support in more places.

We also found that for some reason, runpy/pkgutil does not work for frozen 
package imports, e.g. wsgiref.util. The reasons for this appear to be deeper 
than just in the pkgutil module. We don't have a solution for this yet. It is 
also not clear whether the problem still exists in Python 3.x. The __path__ 
attribute of frozen modules was changed in 3.0 to be a list like for all other 
modules, however, applying that change to 2.x lets runpy/pkgutil fail 
altogether (not even the above fix works anymore).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Brett Cannon]

Changes by Brett Cannon :


--
versions: +Python 3.4 -Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Correction: the helper function is called imp.get_frozen_object().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16027] pkgutil doesn't support frozen modules

[Marc-Andre Lemburg]

New submission from Marc-Andre Lemburg:

pkgutil is used by runpy to run Python modules that are loaded via the -m 
command line switch.

Unfortunately, this doesn't work for frozen modules, since pkgutil doesn't know 
how to load their code object (this can be had via imp.get_code_object() for 
frozen modules).

We found the problem while working on eGenix PyRun (see 
http://www.egenix.com/products/python/PyRun/) which uses frozen modules 
extensively. We currently only target Python 2.x, so will have work around the 
problem with a patch, but Python 3.x still has the same problem.

--
components: Library (Lib)
messages: 171163
nosy: lemburg
priority: normal
severity: normal
status: open
title: pkgutil doesn't support frozen modules
versions: Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com