On Thu, Oct 1, 2009 at 7:15 PM, David Malcolm <[email protected]> wrote:
> Proposal: Python 3 in Fedora 13 > > "Evolutionary, not revolutionary": build a python 3 stack > parallel-installable with the python 2 stack. > > = High-level summary = > - Python 3.0 was released almost 10 months ago, on 2008-12-03, and the > latest release of the 3.* branch is 3.1.1, released on 2009-08-17. > - Other distros have python 3, though not necessarily with anything > "on top" resembling the full python 2 stack. > - We have a working, valuable python 2 stack, which is used by > critical system components (yum and anaconda): we must not destabilize > the python 2 stack. > - Python 3 is sufficiently different from python 2 that we need them > to be independent software stacks. > - I plan to spend a large chunk of my $DAYJOB over the next few months > trying to build a useful Python 3 stack for Fedora 13, for some > definition of "useful" (help will be appreciated!) > - I don't want to add extra work for package maintainers: if you > maintain an SRPM of a python 2 module that's working for you, you > shouldn't feel obligated to own a separate SRPM for python 3. If > someone has a need for the module on python 3, they can take on that > work. > > = Background = > Python 3 is intended by upstream to be the future of Python, but we have > many critical components that use Python 2. Python 2 and Python 3 are > sufficiently different that we need both (try writing "print" in each). > Python 2 will be around for a long time. > > An interesting summary of Python 3 adoption can be seen here: > > http://renesd.blogspot.com/2009/09/py3kpython3-more-than-one-year-on-096.html > > An earlier proposal about python 3 in Fedora is here: > https://www.redhat.com/archives/fedora-devel-list/2008-May/msg02417.html > > Going forward, I will have plenty of time to spend, as part of my > dayjob, on Python in Fedora [1] > > = Proposal = > I want to get python 3 into Fedora, but I don't want to break yum or > anaconda (or anything else, for that matter!) > > How to do this? I propose that Fedora shall have separate, > parallel-installable Python 2 and Python 3 stacks. I believe we can get > things to the point where on a Fedora box you'd be able to install both > stacks, and have some processes running python 2 code, and some running > python 3, simultaneously. > > Where I would draw the line is on having both python 2 and python 3 > running within the same _process_: the two libraries share most of their > symbol names, but with differing implementations, and the result of > trying to dynamically link the two into the same address space would be > highly unstable. > > As an example, you'd be able to install both mod_python and mod_python3 > rpms, but you wouldn't be able to (sanely) configure httpd to have both > running simultaneously (I guess we should add a run-time warning for > this case) > > Scoping: > - this work would target Fedora 13. I'd avoid pushing it into F12 > until it's proven safe to do so > - the proposal is for python 2 vs python 3. It could be extended to > support having multiple minor-versions of Python as well, but that's a > big extension of the work involved and not something I'm planning to > work on myself. > > = Details = > We should split python 2 and python 3 at the source RPM level, where > possible. > > The easy case is when upstream release separate tarballs for the python > 2 and python 3 versions of code. For example, given package > "python-foo" in packaging CVS, there would be a separate "python3-foo" > for the python 3 version. There would be no expectation that the two > would need to upgrade in lock-step. (The two SRPMS could have different > maintainers within Fedora: the packager of a python 2 module might not > yet have any interest in python 3) > > The more difficult case is when the python module is emitted as part of > the build of a larger module. Some examples: > - the build of "rpm" itself emits an "rpm-python" subpackage. > - Another example is the "postgres" srpm, which emits a > "postgresql-python" subpackage. > > In a quick attempt at seeing other examples, on my laptop (F11), here > are the packages installed that provide python modules where the package > name differs from the srpm name: > $ rpm -qf /usr/lib/python2.*/site-packages/* | grep -v "is not owned" | > SIA, this is off of topic , i am sure. BUT, it is very strange that could be exists, perhaps, some file or directory not owned by someone. Isn't it ? > sort | uniq | xargs rpm -q --qf "%{NAME}-%{VERSION}-%{RELEASE} > %{SOURCERPM}\n" | sed -e"s/.src.rpm//" | squeal -f table col0, col1 from > - where col0 != col1 > col0| col1| > > ----------------------------------------+----------------------------------+ > at-spi-python-1.26.0-1.fc11| at-spi-1.26.0-1.fc11| > audit-libs-python-1.7.13-1.fc11| audit-1.7.13-1.fc11| > cracklib-python-2.8.13-4| cracklib-2.8.13-4| > gamin-python-0.1.10-4.fc11| gamin-0.1.10-4.fc11| > hplip-libs-3.9.8-12.fc11| hplip-3.9.8-12.fc11| > libproxy-python-0.2.3-10.fc11| libproxy-0.2.3-10.fc11| > libselinux-python-2.0.80-1.fc11| libselinux-2.0.80-1.fc11| > libsemanage-python-2.0.31-4.fc11| libsemanage-2.0.31-4.fc11| > libuser-python-0.56.9-3| libuser-0.56.9-3| > libxml2-python-2.7.3-3.fc11| libxml2-2.7.3-3.fc11| > newt-python-0.52.10-4.fc11| newt-0.52.10-4.fc11| > plague-common-0.4.5.7-5.20090612cvs.fc11| > plague-0.4.5.7-5.20090612cvs.fc11| > policycoreutils-python-2.0.62-12.12.fc11| > policycoreutils-2.0.62-12.12.fc11| > python-magic-5.03-2.fc11| file-5.03-2.fc11| > python-slip-dbus-0.1.15-3.fc11| python-slip-0.1.15-3.fc11| > python-slip-gtk-0.1.15-3.fc11| python-slip-0.1.15-3.fc11| > rpm-python-4.7.1-1.fc11| rpm-4.7.1-1.fc11| > setroubleshoot-server-2.1.14-3.fc11| setroubleshoot-2.1.14-3.fc11| > > system-config-printer-libs-1.1.8-6.fc11|system-config-printer-1.1.8-6.fc11| > > Such SRPMS have a: > BuildRequires: python-devel > which is a subpackage from the python build (2.6) > > My plan here is that we should create a python3-devel subpackage as a > subpackage of the python3 build, and have it parallel-installable with > the python-devel package. > > We could then %prep the rpm build for each of the above so that the > python 3 support is built as a parallel component of the build, > independently of the python 2 support e.g. by copying the python2 > support into a separate dir, then applying a patch as necessary (and > somehow wiring up the configuration/make so it builds both...) The > caveat here is that I haven't tried actually doing this yet for any of > these packages. Issues with this approach: > - I don't yet know if autoconfiguration will work well with both > -devel packages installed > - It will probably involve actually doing the porting work for each > package (yay, we get to be leaders!) > - Whoever does this for a package needs to work closely with the > upstream for that package > > I'll have a go at doing this for rpm when I get back from vacation. > Arguably the existing python 2 binding of rpm isn't great, but it's > probably best to go for close compatibility between python 2 and python > 3 rather than try to overhaul the bindings as part of the port to python > 3: one thing at a time! > > > "Naming convention" proposal: > How does this sound: > - an rpm with a "python-" prefix means a python 2 rpm, of the > "default" python 2 minor version (for Fedora this will be the most > recent stable upstream minor release, for EPEL it will be the minor > release of 2 that came with the distro, so 2.4 for EPEL5) > > - an rpm with a "python3-" prefix means a python 3 rpm, of the > "default" python 3 minor version (for Fedora this will be the most > recent stable upstream release) > > (we could extend this to have specific minor-releases (e.g. use > "python24-" for a python 2.4 stack, in case some brave soul wants to get > zope/plone running. But may be better to try to fix the zope/2.6 > incompatibility at that point (caveat: haven't looked at the details of > the issue). I don't intend to work on such versions)) > > What about packages without a "python-" prefix? Proposal: If upstream > has a naming convention for python2 vs python3, use it. Otherwise, add > a "python3-" prefix to make things clear. I'm not sure about the > details here. Examples? > > There have been various discussions upstream about what to call the > python 3 binary. My favorite quote on the subject is from Guido, > http://mail.python.org/pipermail/python-3000/2008-March/012421.html : > > During the next 3 years or so, installing Py3k as the default "python" > > will be a deed of utter irresponsibility and is likely to break your > > system in subtle ways (both OSX and Linux these days use Python for > > certain system tasks). If you *really* want to shoot yourself in the > > foot this way, go ahead and explicitly use "make altinstall > > bininstall" or link it yourself. > > I propose that for Fedora we have "/usr/bin/python3" for the > system/default version of python 3 and "/usr/bin/python" for the > system/default version of python 2. Both would be symlinks to a binary > with the minor-release embedded in the name ("/usr/bin/python3.1" and > "/usr/bin/python/2.6"). > > As I understand things, this should make us broadly in agreement with > upstream; see e.g.: > http://mail.python.org/pipermail/python-dev/2009-April/088862.html > http://mail.python.org/pipermail/python-dev/2009-April/088884.html > > A rough plan for Fedora 13 might be: > - get python3 packaged in a manner compatible with the above > - (persuade /usr/lib/rpm/brp-python-bytecompile to use the correct > python when building rpms containing .py files) > - get rpm bindings working with python3 > - get some useful components working e.g. a web stack: Django, > TurboGears etc (though e.g. Django's py3k support is a long way off > IIRC); ideas? > - solidify packaging guidelines for python 2 vs python 3 once we've > got some experience with the above and hopefully proven the techniques > - look at porting major components over to python3 (but probably don't > actually do this for F13; leave python 2 as the critical component, I > suspect): > - yum (rpm) > - anaconda > > However "no plan survives contact with the enemy", we'll see how things > turn out in reality when trying to get a full integrated stack working. > > Future work (F14?) could involve cutting over the major components, so > that the base install would bring in "python3", and "python" would > become optional. Obviously there's a _lot_ to be done before that can > be done sanely. > > = Progress so far = > I've put together a somewhat-working python3 srpm, based on the python > srpm (with lots of FIXMEs added...) It's not yet ready for a formal > package review (I'm working through the various patches, and it's not > yet fully installable in parallel with the python 2 stack), but you can > see the specfile here: > http://people.redhat.com/dmalcolm/python3/python3.spec > and an SRPM here: > http://people.redhat.com/dmalcolm/python3/python3-3.1.1-1.fc13.src.rpm > > After I did this work, I saw that a couple of other people have written > Python 3 srpms for Fedora: > http://ivazquez.fedorapeople.org/packages/python3000/ > and > https://bugzilla.redhat.com/show_bug.cgi?id=526126 > and there was this proposal for doing Python 3 in F10: > > https://www.redhat.com/archives/fedora-devel-list/2008-May/msg02417.html > similar to this one. Obviously I want to work with those people to come > up with a working python 3 rpm in Fedora. > > There's also the merge-review for python: > https://bugzilla.redhat.com/show_bug.cgi?id=226342 ; many of the > comments there would thus also apply to my srpm, given that I used the > former as my starting point. > > On a tangent: we currently have 2.6.2 in F12/rawhide. Upstream is > preparing a 2.6.3 release, with many bugfixes. This seems to me like a > candidate for F13. (the release notes for the 2.6.3 rc are long; I'd > want it to have a _lot_ of testing before pushing it to F12)) > > > Thoughts? > Dave > > [1] I'm being transferred at work, and will be able to spend a lot of > time on Python within Fedora. Previously, my job involved keeping > various internal RH systems working (with any Fedora work done > afterhours or as a side project). Having said that, I'm about to go on > vacation with no access to a computer for the period October 3rd-10th > > > -- > fedora-devel-list mailing list > [email protected] > https://www.redhat.com/mailman/listinfo/fedora-devel-list >
-- fedora-devel-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/fedora-devel-list
