On Wed, 24 Dec 2008 11:23:56 -0800 Danek Duvall <[email protected]> wrote:
> On Wed, Dec 24, 2008 at 12:12:07PM -0600, Tom Mueller (pkg-discuss) wrote:
> We've eliminated the worst side effect of the issue -- that the .pyc files
> are recompiled by the first user with sufficient permissions to write the
> files -- by putting timestamps in the actions of the .py files, so that
> Python doesn't think the .pyc files are out of date and try to recompile
> them.
The problem is worse than that - if python decides the pyc files are
out of date, it will *always* recompile them. That's how it gets the
byte codes it's actually going to feed to the VM, as it properly
doesn't trust the out-of-date pyc files. If it can then write the
appropriate pyc file to save those codes the next user, it will. If it
can't, it won't - and the next person to run that python code will do
it all over again, and so on and so on, until someone with enough
privs to write the file imports them.
> Not shipping pyc files is definitely not the right thing to do, as Python
> will then create files (if it can) in directories that it's not supposed to
> be mucking around in, leaving turds around to be removed by the packaging
> system on package removal.
In that case, the problem is with your packaging system. If python
isn't supposed to be mucking around in python's library directories,
then who is?
pyc files are *not* some kind of compiled file you can use like elc or
jar files. They're a caching mechanism used by the python vm/compiler
combo to improve performance. While they are pretty well platform
independent, they are *not* python version independent. This is why
the python packaging systems (and other packaging systems that were
paying attention) distinguish between "pure python" modules and
modules that have a binary component. A pure python package can - in
theory - be installed using any version of python the author cared to
make it work for. Once you bundle in the pyc files, it's no longer
python version independent - the pyc files won't work on other
versions of python, and someone with write permission running an other
version will cause it to be rewritten, making the checksums wrong,
etc.
Of course, the build system enters into this. I know, IPS folks like
to pretend build systems aren't relevant, but trying to treat pyc
files as deliverables causes a build-time dependency on the run-time
python to appear. Unless you can guarantee that the only run-time,
then they're going to create problems - either, as you discovered,
they'll show up even if you don't provide them, or they'll get changed
(probably multiple times) over the course of their lifetimes.
The best choice for pyc files is to do what the python packaging folks
do, and compile them after installation on the target system. You are
running with the appropriate permissions, know what files you
installed, and know you have the right python - at least for now, so
you can include them in the cleanup list.
Failing that, not delivering pyc files and having a way for the
packaging system to clean up files that it didn't deliver but might
have been created as part of the normal course of running the
application is probably second best.
As a final option, if ON includes a recent enough version of Python
(the one they include is so old I no longer remember if it could do
this or not) is to put all the py files into a zip file that gets
properly added to the path, as python won't try and create pyc files
for files imported from a zip file.
<mike
--
Mike Meyer <[email protected]> http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
