If folks haven't looked at python_generate_wrapper_scripts in
python.eclass, I'd suggest doing so. For examples of it's usage, grep
for 'python_generate_wrapper_scripts' in /usr/bin/; any place you see
it, look for <that-script-name>-${PYTHON_TARGETS} (for example,
/usr/bin/sphinx-build{,-2.7,-3.2}.
Each usage there is a separate custom script for that specific binary;
if there is a bug in the script, well, we're screwed- requires
re-merging the package.
This setup, at least on my hardware, is .04s added to every
invocation; this is ignoring the inode cost for each, and the issue if
a bug ever appears in the script generation code (in which case we're
screwed- would require re-merging the package).
In parallel, we've got python-wrapper (ls /usr/bin/python -l); this is
provided by eselect-python and basically discern what the active
python version is, and use that in the absense of any directives.
This is implemented in C, and is reasonably sane; the overhead for
that is basically non-existant.
Roughly, I'm proposing we do away with python eclass's
generate_python_wrapper_scripts generation of a script, instead having
that just symlink to a binary provided by eselect-python that handles
this. This centralizes the implementation (fix in one spot), and
would allow a c version to be used- basically eliminating the
overhead.
There's a trick to this; currently, those generated scripts hardcode
the allowed/known python versions for that package. We obviously have
to preserve that; I propose we shove it into the symlink path.
Basically, we add a /usr/libexec/python directory; within it, we have
a wrapper binary (explained below), and a set of symlinks pointing at
the root of that directory. To cover our current python versions, the
following would suffice:
for x in {2.{4,5,6,7},3.{0,1,2,3,4}}-cpy 2.5-jython 2.7-pypy-1.{7,8}
\2.7-pypy-1.9; do
ln -s ./ /usr/libexec/python/$x
done
While that seems insane, there is a reason; via that, we can encode
the allowed versions into the symlink. Using pkgcore's pquery for
example (which should support cpy: 2.5, 2.6, 2.7, 3.1, 3.2, 3.3)
instead of a wrapper script at /usr/bin/pquery, we'd have thus:
targets=( 2.{5,6,7}-cpy 3.{1,2,3}-cpy )
targets=$(IFS=/;echo -n "${targets[*]}")
# This results in
# targets=2.5-cpy/2.6-cpy/2.7-cpy/3.1-cpy/3.2-cpy/3.3-cpy
ln -s "/usr/libexec/python/${targets}/wrapper" \
/usr/bin/pquery
/usr/libexec/python/wrapper upon invocation, takes a look at argv[0];
sees how it was invoked basically. This will be the /usr/bin/whatever
pathway. It reads the symlink, in the process getting the allowed
versions and preferred order of the versions.
Few notes; vast majority of filesystems will store the symlink target
into the core inode if at all possible- in doing so, this avoids
wasting an inode and is only limited by the length of the target.
That length is capped by PATH_MAX- which can range from 256 to 4k (or
higher).
For the pquery example above, that comes out to ~73 bytes for the
symlink pathway; well under PATH_MAX.
For the scenarios where PATH_MAX caps the symlink pathway, or for
whatever reason we don't want to use that trick, a tree of files
contained within /usr/libexec/python/ holding the allowed versions for
the matching pathway would suffice.
Either proposal here would be far faster than what we've got now; also
will use less space (ancillary benefit).
One subtle aspect here is that if we did this, it makes it possible to
avoid losing the invocation information- currently if you did
`/usr/bin/python3.2 $(which sphinx-build) blah`, because of how things
are implemented now (specifically the two layers of wrappers)- you'll
get python2.7 running that sphinx-build invocation.
This is wrong (it's directly discarding what the invocation
requested), although you're only going to see it for scripts that
do python introspection.
Via doing the restructuring I'm mentioning above, that issue can be
fixed, while making things faster/saner.
On a related note; we currently install multiple versions of the same
script- the only difference being the shebang. If one ignores the
shebang, in some cases this is necessary- where the script is 2to3
translated, and the code for py2k vs py3k differs. For most, the only
difference is in the shebang however.
While it's minor in space savings, it's possible to eliminate that
redundancy via a shebang target that looks at the pathway it was
invoked via. Fairly easy actually, and basically zero overhead if
done.
Either way, thoughts?
What I'm proposing isn't perfect, but I'm of the view it's a step up
from what's in place now- and via centralizing this crap, makes it
easier to change/maintain this going forward as necessary.
~harring
