Brett Cannon added the comment:
Nick's suggestions done in changeset 792810303239 .
--
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
___
Brett Cannon added the comment:
Nick had some good suggestions on improvements:
http://mail.python.org/pipermail/python-dev/2013-January/123602.html
Re-opening to remind me to do them.
--
status: closed - open
___
Python tracker
Roundup Robot added the comment:
New changeset a4292889e942 by Brett Cannon in branch 'default':
Issue #15031: Refactor some code in importlib pertaining to validating
http://hg.python.org/cpython/rev/a4292889e942
--
nosy: +python-dev
___
Python
Brett Cannon added the comment:
OK, so I took Nick's suggestion of doing header verification separately from
the actual compilation, but I ended up just writing the patch from scratch to
see how a possible API might play out. ATM the API is as functions and private,
but if I find the
Brett Cannon added the comment:
So we have IronPython who has their own format, Jython who has their own format
*and* .pyc files (which would screw with __cached__ so I assume they won't try
to use both at the same time), PyPy which use .pyc files, and CPython which
uses .pyc files. My
Philip Jenvey added the comment:
From the perspective of Jython we'd want the easiest way to hook into this as
possible of course, but I think that overriding marshal to handle a $py.class
or whatever format would be a misappropriation of the marshal module
Jython actually has a slow,
Nick Coghlan added the comment:
The suppression flag rings alarm bells for me, as does the fact that all the
arguments are optional. Do you remember the rationale for allowing the
marshalling errors to propagate rather than falling back to loading from
source? It seems weird that a truncated
Brett Cannon added the comment:
The rationale is that was the way it already was prior to importlib.
As for the approach you are suggesting, I am understand it, it will just
not give the public method the same functionality which might not be that
important.
On Nov 18, 2012 5:18 AM, Nick
Nick Coghlan added the comment:
OK, rereading the whole issue and getting completely back up to speed with the
problem we're trying to solve, I think parse_bytecode_container is a better
name than any of my suggestions, since there is no cache involved for
SourcelessLoader and similar cases.
Brett Cannon added the comment:
It's a method so that it can be overridden. Otherwise I can't develop my
own format and have that be the only differing option from SourceLoader.
On Nov 18, 2012 7:50 AM, Nick Coghlan rep...@bugs.python.org wrote:
Nick Coghlan added the comment:
OK, rereading
Brett Cannon added the comment:
Or to put it another way, without making it a method other interpreters like
IronPython and Jython will never be able to reuse SourceLoader effectively if
they want their own cached file format.
--
___
Python tracker
Brett Cannon added the comment:
FYI I'm talking with Dino for IronPython and I just emailed some Jython folks
to try to get an opinion on whether they would prefer to have a method to
override, something in _imp that they can implement, or simply implement
marshal.loads() and dumps() such
Changes by Brett Cannon br...@python.org:
--
nosy: +dino.viehland
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
___
___
Python-bugs-list
Nick Coghlan added the comment:
I thought about that, but decided that being able to hook the compilation
step and the reading from the cache step without overriding get code
completely is a separate design problem from providing a public API for
reading the bytecode container format.
Brett Cannon added the comment:
Another name for the method I thought of last night was
parse_bytecode_container().
People like that more? That would lead to new terminology to distinguish
bytecode ala dis module compared to bytecode file format ala importlib that
makes it clear that what is
Nick Coghlan added the comment:
parse_cache_contents is my last proposed colour for the shed, but I can
live with parse_bytecode_container
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
Brett Cannon added the comment:
I can go with parse_cache_contents() as that aligns with
imp.cache_from_source().
Does the API look reasonable (other than the parameter names since those would
have to be renamed to be more in line with cache)? Only thing I'm worried
about.
--
Brett Cannon added the comment:
In order to have exceptions that have messages like bad magic number in
module there would need to be a technically unneeded fullname parameter.
People cool with that? I personally dislike having info passed in just for
error reporting, but in this case import
Eric Snow added the comment:
Don't underestimate the potential value of having the fullname when overriding
the method in a subclass.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
Brett Cannon added the comment:
Attached is a patch that introduces _LoaderBasics.parse_bytecode_file(). It
takes in basically what _bytes_from_bytecode() did plus a keyword-only boolean
flag for exception suppression as that's needed to ignore stale bytecode when
source is available.
I
Brett Cannon added the comment:
Here is my current plan::
parse_bytecode_file(data, source_stats, source_path) - code
This will take in the bytes from the bytecode file and a stats dict from
SourceLoader.path_stats(). The method will parse the bytecode file, verify the
magic number,
Brett Cannon added the comment:
Might name it compile_bytecode_file() instead.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
___
___
Nick Coghlan added the comment:
Perhaps go a bit more generic and call it something like load_cached_data?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
___
Brett Cannon added the comment:
You can't use data as there are connotations for that word thanks to
get_data(). Same goes for load since this is something that get_code() would
use more directly than load_module().
--
___
Python tracker
Changes by Eric Snow ericsnowcurren...@gmail.com:
--
nosy: +eric.snow
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
___
___
Python-bugs-list
Brett Cannon added the comment:
While thinking about terminology, source files/modules vs. bytecode
files/modules make the distinction clear enough to differentiate just straight
bytecode for an object (i.e. what dis outputs) vs. bytecode plsu the other
stuff that goes with .pyc files.
Changes by Brett Cannon br...@python.org:
--
assignee: - brett.cannon
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
___
___
Changes by Brett Cannon br...@python.org:
--
versions: +Python 3.4 -Python 3.3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
___
___
Brett Cannon added the comment:
So the more I think about this, the more I'm willing to do this in Python 3.4.
First, though, the docs would need to be updated in importlib to distinguish
between bytecode and byte-compiled files.
From there, I think
Brett Cannon br...@python.org added the comment:
Why is distribute reading bytecode to begin with? What's the use case,
especially considering that using bytecode screws over other Python VMs like
Jython and IronPython.
--
___
Python tracker
Marc Abramowitz msabr...@gmail.com added the comment:
Well, it may be a vestige from setuptools and I don't know if it's still
needed/appropriate, but distribute scans the pyc modules to try to see whether
stuff is zip_safe or not when you run `python setup.py bdist_egg`:
Brett Cannon br...@python.org added the comment:
This all goes back to my original point: I don't want to promote people
shipping around bytecode only as it hampers the use of other VMs.
Anyway, I'll continue to contemplate this. Any function would have to verify
the magic number and flat-out
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:
--
nosy: +Arfrever
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
___
Marc Abramowitz msabr...@gmail.com added the comment:
Another package that inspects pyc files and which also ran into trouble because
of the 8 to 12 byte change is distribute.
See:
https://bitbucket.org/tarek/distribute/issue/283/bdist_egg-issues-with-python-330ax
Some kind of abstraction
Ronan Lamy ronan.l...@gmail.com added the comment:
I see. However, I think that breaking code noisily is better than breaking it
silently, which is what happens when the same protocol is reimplemented many
times. And _loads_pyc() could be made more forward-compatible by returning
(path_stats,
Brett Cannon br...@python.org added the comment:
I point is that it shouldn't break at all if possible, although path_stats does
potentially provide a way to deal with this (as does using keyword-only
arguments for what needs to be verified).
I'll have to think about this. One issue I have
New submission from Ronan Lamy ronan.l...@gmail.com:
I think it would be beneficial to extract the handling of the binary format of
bytecode files from the rest of the loader/finder code in importlib._bootstrap.
That means exposing, internally at least, functions that do nothing more than
Changes by Jesús Cea Avión j...@jcea.es:
--
nosy: +jcea
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15031
___
___
Python-bugs-list mailing list
Brett Cannon br...@python.org added the comment:
So the problem with the function is that had this been implemented in Python
3.2 it already would be outdated thanks to the 3.3 change of storing the file
size in the .pyc file, and simply changing the length of the sequence returned
would
39 matches
Mail list logo
