Hi all,
so I started to work on packaging Python 3.4 beta 1 and I wanted to create
ensurepip patch also acceptable for upstream - Toshio and Nick talked about
that at [1]. I tried to analyze the situation and here is what I have. First
some facts:
- with ensurepip, Python 3.4 bundles setuptools and pip wheel archives
- when ensurepip is run, it installs (or upgrades) currently installed pip from
the bundled wheels
A way to solve this "properly" in a distro (as mentioned in [1]) would be:
- throw away bundled setuptools and pip wheels when building Python
- making python3 require python-pip
- patching ensurepip (this is the fun part):
-- if ensurepip is run in venv:
--- we will need to reconstruct the wheels and install them into venv; or just
grab the RECORD files and use them as a list of files we need to copy into venv
--- to reconstruct the wheels from installed RPMs, it'd be a good idea to
package python-{setuptools,pip} as wheels (with the ".dist-info" directory and
all the wheel goodies inside) so that we have the RECORD files
--- to package setuptools and pip as wheels, they'll have to BR: python3-pip (I
saw Toshio's notes from flock and I agree with the part about how Fedora should
use wheels [2])
-- if ensurepip is run systemwide:
--- if "--upgrade" is used, should this run "yum update python3-pip"?
Now, when new X.Y version Python is introduced, we will have to:
- bootstrap the new python3 interpreter to not require python3-pip, since
python3-pip will require python(abi)=X.(Y-1)
- then we will need to build setuptools and pip *not as wheels*, because we
still won't have pip that would work with our new Python X.Y - that means
boostrapping them too and building them using the current "setup.py install"
approach
- when we will have the new pip, we will be able to reenable python3 requires
for python3-pip
- and we will also be able to rebuild setuptools and pip *as wheels*
There are also some more minor issues, that are worth mentioning:
- We typically won't update pip after Fedora is released - if there are
security issues, we will backport patches and bump the release. The problem
with this is that "ensurepip --upgrade" is not able to see Fedora releases and
wouldn't work in venv, leaving it vulnerable. I think we have three options
here:
-- Either we use some Fedora-specific RPM magic, which won't be acceptable for
upstream
-- Or we will use a nice feature of wheel RECORD files - they keep file hashes,
so we can check venv pip against system pip and see if something has changed
when "ensurepip --upgrade" is run
-- Or we can just copy whole pip again into venv, regardless of what is
currently there (doesn't seem very nice..., but it's simple and in the end it
works as the above method)
- We can't copy the *.pyc and *.pyo files from system to venv, because they
have file locations hardcoded in them, that are used in tracebacks. We can
solve this by just copying *.py files and running py_compile in our ensurepip.
I hope I covered all the important points. Basically, we can make this work in
a way acceptable for upstream, if we package setupttols and pip as wheels.
It'll require some extra effort, but I think it's worth it.
Thoughts? Anyone has better/simpler ideas?
Thanks a lot,
Slavek.
[1]
https://lists.fedoraproject.org/pipermail/python-devel/2013-November/000535.html
[2]
https://fedoraproject.org/wiki/User:Toshio/Flock2013_Python_Guidelines#Wheel:_the_new_upstream_distribution_format
_______________________________________________
python-devel mailing list
python-de...@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel
