Re: Python libraries and backwards compat [was Re: What would it take to make Software Collections work in Fedora?]
On Mon, 2013-03-04 at 22:51 +, Mark McLoughlin wrote: (following up with more thoughts from the distutils-sig thread) It started here: http://mail.python.org/pipermail/distutils-sig/2013-February/020030.html and now we're talking about Software Collections here: http://mail.python.org/pipermail/distutils-sig/2013-March/020074.html Two things I'm picking up from the thread: - A trend towards semantic versioning and, implicit in that, an acceptance of API breakages so long as the major number of a library version is incremented - Supporting the parallel installation of incompatible versions of libraries isn't seen as an issue because you can just use virtual environments ... which amounts to Python Software Collections. I think this parallel installs issue is the key thing we (Fedora, OpenStack, etc.) need to help figure out. There will be incompatible updates to libraries and, like with e.g. gtk2 and gtk3, I think we'll need to support apps using different versions of the same library at the same time. Right now, Software Collections is the only way I can see that OpenStack can be packaged so we don't get screwed when an incompatible library update comes along. Nick Coghlan laid out some ideas for what could be done for Python which I summarised as: http://mail.python.org/pipermail/distutils-sig/2013-March/020082.html - the system has multiple versions of somedep installed under /usr somewhere - the latest version (2.1) is what you get if you naively just do 'import somedep' - most applications should instead somehow explicitly say they need ~1.3, -1.6 or ~-2.0 or whatever - distros would have multiple copies of the same library, but only one copy for each incompatible stream rather than one copy for each application This is a much happier prospect than either Software Collections or no support for parallel installs. It needs help to make it happen, though! Cheers, Mark. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Python libraries and backwards compat [was Re: What would it take to make Software Collections work in Fedora?]
On Thu, 2012-12-06 at 07:06 -0800, Adam Williamson wrote: On Thu, 2012-12-06 at 15:30 +0100, Nicolas Mailhot wrote: IMHO use of software collections is a symptom of a badly run organisation not devoting enough cycles to maintain the software it uses, and hoping (as in wishful thinking) no problem will go critical before the product they built on top of those collections is end-of-lifed I completely fail to see how entities with that problem will manage to maintain the package number explosion creating software collections will induce. On the one hand, I agree completely - I think the 'share all dependencies dynamically' model that Linux distros have traditionally embraced is the right one, and that we're a strong vector for spreading the gospel when it comes to that model, and it'd be a shame to compromise that. On the other hand, we've been proselytizing the Java heretics for over a decade now, and the Ruby ones for a while, and neither shows any signs of conversion or just plain going away, so we may have to call it an ecumenical matter and deal with their models somehow. Sucky as it may be. I don't know, I'm a bit conflicted. It's interesting that you call out Java and Ruby folks as being heretics. I guess that means all is kosher with Python? OpenStack is getting burned by API instability in some Python packages, so I've started a thread on Python's distutils-sig to try and guage the level of heresy amongst Python folks :) It started here: http://mail.python.org/pipermail/distutils-sig/2013-February/020030.html and now we're talking about Software Collections here: http://mail.python.org/pipermail/distutils-sig/2013-March/020074.html Two things I'm picking up from the thread: - A trend towards semantic versioning and, implicit in that, an acceptance of API breakages so long as the major number of a library version is incremented - Supporting the parallel installation of incompatible versions of libraries isn't seen as an issue because you can just use virtual environments ... which amounts to Python Software Collections. The combination of those two things suggests to me that the Python world will start looking a lot less sane to packagers - i.e. library maintainers breaking API compatibility more often and assuming we can just use SC or similar to have multiple incompatible versions installed. I can see OpenStack upstream reacting to this by capping its required version range for each library it depends so that if the library does release an incompatible version, OpenStack sticks with the latest compatible version. If that happens, I think OpenStack packagers will need to look seriously at using Software Collections. Basically, we'd look to package and maintain our own stack of all the Python libraries we need above the core Python libraries. So, you'd have openstack-nova, openstack-glance, etc. all installed as normal in /usr, /var, etc. but they'd require python libraries from the openstack-grizzly SC like openstack-grizzly-python-eventlet which would be installed in /opt/fedora/openstack-grizzly/root/usr/lib/python. I'd appreciate it if someone else with a Fedora Python packaging background could look into this and, hopefully, explain how the discussion on distutils-sig isn't so terrifying after all. Cheers, Mark. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
[Fwd: Python libraries and backwards compat [was Re: What would it take to make Software Collections work in Fedora?]]
Hey, Just forwarding it here so Python folks don't miss it on the main devel list. Thanks, Mark. Forwarded Message From: Mark McLoughlin mar...@redhat.com Reply-to: Mark McLoughlin mar...@redhat.com To: Development discussions related to Fedora devel@lists.fedoraproject.org Subject: Python libraries and backwards compat [was Re: What would it take to make Software Collections work in Fedora?] Date: Mon, 04 Mar 2013 22:51:31 + On Thu, 2012-12-06 at 07:06 -0800, Adam Williamson wrote: On Thu, 2012-12-06 at 15:30 +0100, Nicolas Mailhot wrote: IMHO use of software collections is a symptom of a badly run organisation not devoting enough cycles to maintain the software it uses, and hoping (as in wishful thinking) no problem will go critical before the product they built on top of those collections is end-of-lifed I completely fail to see how entities with that problem will manage to maintain the package number explosion creating software collections will induce. On the one hand, I agree completely - I think the 'share all dependencies dynamically' model that Linux distros have traditionally embraced is the right one, and that we're a strong vector for spreading the gospel when it comes to that model, and it'd be a shame to compromise that. On the other hand, we've been proselytizing the Java heretics for over a decade now, and the Ruby ones for a while, and neither shows any signs of conversion or just plain going away, so we may have to call it an ecumenical matter and deal with their models somehow. Sucky as it may be. I don't know, I'm a bit conflicted. It's interesting that you call out Java and Ruby folks as being heretics. I guess that means all is kosher with Python? OpenStack is getting burned by API instability in some Python packages, so I've started a thread on Python's distutils-sig to try and guage the level of heresy amongst Python folks :) It started here: http://mail.python.org/pipermail/distutils-sig/2013-February/020030.html and now we're talking about Software Collections here: http://mail.python.org/pipermail/distutils-sig/2013-March/020074.html Two things I'm picking up from the thread: - A trend towards semantic versioning and, implicit in that, an acceptance of API breakages so long as the major number of a library version is incremented - Supporting the parallel installation of incompatible versions of libraries isn't seen as an issue because you can just use virtual environments ... which amounts to Python Software Collections. The combination of those two things suggests to me that the Python world will start looking a lot less sane to packagers - i.e. library maintainers breaking API compatibility more often and assuming we can just use SC or similar to have multiple incompatible versions installed. I can see OpenStack upstream reacting to this by capping its required version range for each library it depends so that if the library does release an incompatible version, OpenStack sticks with the latest compatible version. If that happens, I think OpenStack packagers will need to look seriously at using Software Collections. Basically, we'd look to package and maintain our own stack of all the Python libraries we need above the core Python libraries. So, you'd have openstack-nova, openstack-glance, etc. all installed as normal in /usr, /var, etc. but they'd require python libraries from the openstack-grizzly SC like openstack-grizzly-python-eventlet which would be installed in /opt/fedora/openstack-grizzly/root/usr/lib/python. I'd appreciate it if someone else with a Fedora Python packaging background could look into this and, hopefully, explain how the discussion on distutils-sig isn't so terrifying after all. Cheers, Mark. ___ python-devel mailing list python-de...@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: glusterfs and hekafs release number for f16 and rawhide; systemd switch-over
On Mon, 2011-10-31 at 09:01 -0400, Kaleb S. KEITHLEY wrote: Up to now the glusterfs and hekafs versions and releases have been the same for f16 and rawhide, i.e.: glusterfs-3.2.4-1.x86_64.fc16.rpm, glusterfs-3.2.4-1.x86_64.fc17.rpm, hekafs-0.7-16.x86_64.fc16.rpm, and hekafs-0.7-16.x86_64.fc17.rpm. I did that because the source, thus far, is exactly the same for both f16 and rawhide. In f16 and rawhide both glusterfs and hekafs used sysv init.d scripts. Now for rawhide I'm going to switch to systemd. I know I can't switch to systemd for f16, so the question is, what scheme should I used for the release numbering? Honestly, I wouldn't fuss about keeping release numbers in sync - as soon as someone does the first Fedora 17 mass rebuild, they'll be out of sync anyway. Cheers, Mark. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Wed, 2011-09-21 at 15:54 -0400, Peter Jones wrote: On 09/21/2011 03:39 PM, Mark McLoughlin wrote: On Wed, 2011-09-21 at 18:48 +0100, Matthew Garrett wrote: Remember that the incompatibility isn't between libguestfs and the guest, it's between the host grub and the guest grub. Both of those can change without libguestfs's knowledge. Sounds like we need a 'Conflicts: libguestfs' in both grub and grub2 then? Yes, but this will hardly help the situation, which right now is that the distro pulls in grub 2, because that's what we've collectively chosen to do, and libguestfs pulls in grub on the host, even though it isn't really using it there. So effectively your solution is to keep the problem we've got right now. Sigh. I was joking. Obviously, if maintainers went around inserting Conflicts with other packages because they don't like how the other package works, then there'd be an order of magnitude more unpleasantness on fedora-devel. Not liking the way libguestfs works is no justification for an arbitrary grub2 Conflicts in the grub package. It sounds like there are issues which necessitate the Conflicts - e.g. the grubby issue - but that they could be resolved. Can we focus on those issues, what exactly they are and how folks can do to help resolve them? Cheers, Mark. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, 2011-09-22 at 14:05 +0100, Matthew Garrett wrote: On Thu, Sep 22, 2011 at 11:27:35AM +0100, Mark McLoughlin wrote: Sigh. I was joking. Obviously, if maintainers went around inserting Conflicts with other packages because they don't like how the other package works, then there'd be an order of magnitude more unpleasantness on fedora-devel. The grub maintainer is telling you that the way in which you're trying to use grub is broken. You *need* to use the grub files that are in guest, not the host. This will be even more true with grub 2. It's not a matter of disliking the approach, it's a matter of it being demonstrably technically incorrect. There's nothing technically incorrect about the approach, demonstrably or otherwise, if the version of grub in the guest and host is compatible. Cheers, Mark. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: grub / grub2 conflicts
On Thu, 2011-09-22 at 17:00 +0100, Matthew Garrett wrote: On Thu, Sep 22, 2011 at 04:50:16PM +0100, Mark McLoughlin wrote: On Thu, 2011-09-22 at 14:05 +0100, Matthew Garrett wrote: The grub maintainer is telling you that the way in which you're trying to use grub is broken. You *need* to use the grub files that are in guest, not the host. This will be even more true with grub 2. It's not a matter of disliking the approach, it's a matter of it being demonstrably technically incorrect. There's nothing technically incorrect about the approach, demonstrably or otherwise, if the version of grub in the guest and host is compatible. grub provides no mechanism for you to know that, which means you can't reliably know that. Which means relying on them being compatible is incorrect. You described yourself how libguestfs could check it. And failing libguestfs doing it, the user could be warned to check it. But again, all of this orthogonal to the issue of the Conflicts. Whether the Conflicts is correct is a completely separate discussion from how libguestfs should use grub and grub2. Conflating the two discussions makes it appear like the maintainer of one package is refusing to fix a bug in his package until another maintainer agrees to change the design of his program to the first maintainers taste. Cheers, Mark. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
