Re: Software management: Call for RFEs results!
Vít Ondruch wrote: It is interesting to see such response from somebody who appears to be maintainer of Qt. Don't we ship 3 parallel installable version of Qt? We indeed ship major (first digit!) versions of Qt as parallel-installable versions. They are for all practical purposes different libraries (with different sonames and even different unversioned library names), and they have different package names (as they should). (We also make them both coexist in /usr, without requiring kludges such as SCLs.) I don't think allowing multiple versions of the qt package to be installed at the same time would be beneficial, at all. In fact, some people would then try to co-install qt-4.8.5 with qt-4.7.x or even qt-4.8.4, which is of course NOT supported; stuff should just be made to work with the latest 4.x.x (which we ship as an update) instead. As for renaming qt to qt4 once qt5 is current (and maybe also qt5-* to qt-*, I'm not sure whether we will do that), that's what (versioned) Obsoletes is for. To be honest, the Kernel is the last package, which should be paraller installable, since you can run just one kernel at time. But removing the running kernel is problematic because of the dynamic module loading. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
Dne 17.10.2013 01:15, Kevin Kofler napsal(a): Vít Ondruch wrote: Sorry, this has nothing to do with FPC yet. RPM/YUM/DNF should first provide reasonable support. For example this issue [1] could take us closer as a first approximation. Vít [1] https://bugzilla.redhat.com/show_bug.cgi?id=845247 Parallel-installing multiple versions of the same package is a flawed idea from the onset, the only package we support it for is the kernel (and that's a hack). I don't think it makes sense to try to support it elsewhere. It would be much more productive to just package the latest version of the library in Fedora (if necessary as an update) and make the client packages work with that (if necessary in a grouped update). Kevin Kofler It is interesting to see such response from somebody who appears to be maintainer of Qt. Don't we ship 3 parallel installable version of Qt? To be honest, the Kernel is the last package, which should be paraller installable, since you can run just one kernel at time. Vít -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On 10/17/2013 09:15 AM, Vít Ondruch wrote: Dne 17.10.2013 01:15, Kevin Kofler napsal(a): Vít Ondruch wrote: Sorry, this has nothing to do with FPC yet. RPM/YUM/DNF should first provide reasonable support. For example this issue [1] could take us closer as a first approximation. Vít [1] https://bugzilla.redhat.com/show_bug.cgi?id=845247 Parallel-installing multiple versions of the same package is a flawed idea from the onset, the only package we support it for is the kernel (and that's a hack). I don't think it makes sense to try to support it elsewhere. It would be much more productive to just package the latest version of the library in Fedora (if necessary as an update) and make the client packages work with that (if necessary in a grouped update). Kevin Kofler It is interesting to see such response from somebody who appears to be maintainer of Qt. Don't we ship 3 parallel installable version of Qt? To be honest, the Kernel is the last package, which should be paraller installable, since you can run just one kernel at time. Yeah, admins will love that, when after updating the kernel the machine won't boot with the new one and there is no easy way to switch to the old one... --J. Vít -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
Dne 17.10.2013 10:05, Jiri Moskovcak napsal(a): On 10/17/2013 09:15 AM, Vít Ondruch wrote: To be honest, the Kernel is the last package, which should be paraller installable, since you can run just one kernel at time. Yeah, admins will love that, when after updating the kernel the machine won't boot with the new one and there is no easy way to switch to the old one... --J. This is OT and that was definitely not the point. Vít -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/16/2013 07:15 PM, Kevin Kofler wrote: Vít Ondruch wrote: Sorry, this has nothing to do with FPC yet. RPM/YUM/DNF should first provide reasonable support. For example this issue [1] could take us closer as a first approximation. Vít [1] https://bugzilla.redhat.com/show_bug.cgi?id=845247 Parallel-installing multiple versions of the same package is a flawed idea from the onset, the only package we support it for is the kernel (and that's a hack). I don't think it makes sense to try to support it elsewhere. It would be much more productive to just package the latest version of the library in Fedora (if necessary as an update) and make the client packages work with that (if necessary in a grouped update). This would be nice in a perfect world where all upstreams understand backwards-compatibility and proper API/ABI versioning. Unfortunately, we live in a world where even major players (like Google with v8 or anyone in the Ruby community) break compatibility constantly. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlJf8RgACgkQeiVVYja6o6PuaQCfYRWcTHhPAq8Zny4pO1emDg3a f30AoIdIUBJaJ94atD08M38w0vMpvufy =8RDh -END PGP SIGNATURE- -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On 15. 10. 2013 at 09:40:41, Matthew Miller wrote: On Tue, Oct 15, 2013 at 01:15:26PM +0200, Jan Zelený wrote: Not to be only negative here, take a look at the COPR initiative, I expect it will solve the problem you are talking about by offering external repositories that will be easily reachable from Fedora but won't be a part of the Fedora itself. The content of these repositories will be governed by the same law as Fedora packages are (SW patents, ...) but technical policies should be a lot less strict. Would that address your concerns? I won't speak for Michael, but I think the answer is no. COPRs fills a need, but it's _too_ wild west (no package signatures, for example). We need to support multiple language runtimes and native upstream packaging *in* Fedora. Ok then, talk to FPC about this. Personally I'd be against creating the wild west from Fedora itself and I'd rather like to have have it in COPRs. Fedora should keep its high standard of Software packaging (which usually doesn't apply for upstream packages). Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
Dne 16.10.2013 10:04, Jan Zelený napsal(a): On 15. 10. 2013 at 09:40:41, Matthew Miller wrote: On Tue, Oct 15, 2013 at 01:15:26PM +0200, Jan Zelený wrote: Not to be only negative here, take a look at the COPR initiative, I expect it will solve the problem you are talking about by offering external repositories that will be easily reachable from Fedora but won't be a part of the Fedora itself. The content of these repositories will be governed by the same law as Fedora packages are (SW patents, ...) but technical policies should be a lot less strict. Would that address your concerns? I won't speak for Michael, but I think the answer is no. COPRs fills a need, but it's _too_ wild west (no package signatures, for example). We need to support multiple language runtimes and native upstream packaging *in* Fedora. Ok then, talk to FPC about this. Sorry, this has nothing to do with FPC yet. RPM/YUM/DNF should first provide reasonable support. For example this issue [1] could take us closer as a first approximation. Vít [1] https://bugzilla.redhat.com/show_bug.cgi?id=845247 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On 16. 10. 2013 at 10:46:01, Vít Ondruch wrote: Dne 16.10.2013 10:04, Jan Zelený napsal(a): On 15. 10. 2013 at 09:40:41, Matthew Miller wrote: On Tue, Oct 15, 2013 at 01:15:26PM +0200, Jan Zelený wrote: Not to be only negative here, take a look at the COPR initiative, I expect it will solve the problem you are talking about by offering external repositories that will be easily reachable from Fedora but won't be a part of the Fedora itself. The content of these repositories will be governed by the same law as Fedora packages are (SW patents, ...) but technical policies should be a lot less strict. Would that address your concerns? I won't speak for Michael, but I think the answer is no. COPRs fills a need, but it's _too_ wild west (no package signatures, for example). We need to support multiple language runtimes and native upstream packaging *in* Fedora. Ok then, talk to FPC about this. Sorry, this has nothing to do with FPC yet. RPM/YUM/DNF should first provide reasonable support. For example this issue [1] could take us closer as a first approximation. Again? Really? As I wrote before, I'm not going to go for that bait yet again. Also this sub-thread was about policies, not about the technical side per se. BTW great demonstration about the upstream wild west I was referring to in the previous email, thank you for that example. Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On Wed, Oct 16, 2013 at 10:04:55AM +0200, Jan Zelený wrote: I won't speak for Michael, but I think the answer is no. COPRs fills a need, but it's _too_ wild west (no package signatures, for example). We need to support multiple language runtimes and native upstream packaging *in* Fedora. Ok then, talk to FPC about this. Personally I'd be against creating the wild west from Fedora itself and I'd rather like to have have it in COPRs. Fedora should keep its high standard of Software packaging (which usually doesn't apply for upstream packages). I *am* talking to FPC about this. But I think you're misunderstanding me here. We need something that is more reliable, trustable, and consistent than the wild west, in _addition_ to what COPRs provides. In order to actually have an impact on making the world better -- applying those high standards where they have an effect -- we need to be relevant and interesting to what people are actually doing. If we aren't, why would people use our distribution? And if they don't, what are we doing this for? We have a mantra of upstream! upstream! upstream! for software development and patches. In the olden days, we didn't do that for packaging, because there was no consistent upstream packaging at all (just the occasional upstream shipping terrible distro-targetted packages, or else some weird custom installer). But now, there are important and well-used packaging systems at an upstream ecosystem level, and we're fighting that rather than adapting -- to our own detriment. We need to take the upstream mantra there too, for the exact same reasons. We can bring library packages to a higher standard at our level for a small impact via constant, ongoing work for ourselves. Or, we can make those _same_ improvements upstream, where we can share the ongoing maintenance and make things better for all users of the software. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On 16/10/13 13:57, Matthew Miller wrote: We have a mantra of upstream! upstream! upstream! for software development and patches. In the olden days, we didn't do that for packaging, because there was no consistent upstream packaging at all (just the occasional upstream shipping terrible distro-targetted packages, or else some weird custom installer). But now, there are important and well-used packaging systems at an upstream ecosystem level, and we're fighting that rather than adapting -- to our own detriment. There have been important and well-used packaging systems at an upstream ecosystem level for a long time - how long has CPAN been around exactly? That hasn't stopped us saying that they don't provide a good experience to Fedora users however, and that it is better to repackage things as RPMs so that our users only have to deal with a single interface to installing and updating packages and that they will get a set of packages customised (where necessary) to all work nicely together. Basically your argument seems to be that we can't keep up, so in order to match the ruby and node people at their own game we should reduce our package management to their level. It's basically just admitting defeat in the quest to produce something better. I made the decision some years ago to stop using CPAN and only use perl modules via rpm and yum and it's made my life a whole let better and I have absolutely no desire. I do currently use gem to manage ruby stuff I will admit, mostly because I'm more actively developing in that area so I have more need to keep close to the bleeding edge. Plus it's not as much of a pain to use as CPAN. So far I've mostly managed to stick with rpm and yum for node stuff. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On Wed, Oct 16, 2013 at 10:04 AM, Jan Zelený jzel...@redhat.com wrote: Ok then, talk to FPC about this. Personally I'd be against creating the wild west from Fedora itself and I'd rather like to have have it in COPRs. Fedora should keep its high standard of Software packaging (which usually doesn't apply for upstream packages). Although I support the desire to have library versioning and package versioning that makes sense, I'm not sure that just insisting that we have the high standards and we are doing it right is enough. We are _not_ doing it right. We should have given Linux users an operating system that gives users all of the major functionality in various ruby gems and Python modules and ... _within our design with high standards_. We haven't done so; in fact if we look at the Linux API (as opposed to the language-specific APIs), we can't claim any high standards: we provide various C libraries with completely inconsistent APIs, and various command-line tools with completely inconsistent interfaces, and the like. If you don't look only at the packaging, most of the language-specific ecosystems are an _improvement_ over the C-based perfectly packaged stuff. I can't in good conscience tell users of these languages your libraries are bad, stop using them when I don't have an alternative to offer them. So, yes, we should keep a high standard for packaging. However the only way we can sustain that high standard for packaging is to also have a high standard for what is packaged and provided to applications; if we are not willing to do that, the high standard for packaging will be unsutainable. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
Tom Hughes wrote: That hasn't stopped us saying that they don't provide a good experience to Fedora users however, and that it is better to repackage things as RPMs so that our users only have to deal with a single interface to installing and updating packages and that they will get a set of packages customised (where necessary) to all work nicely together +1 I think that native upstream packaging systems, which I call NON-native packaging systems (RPM is native, everything else is not) are not the way to go. They fail particularly whenever something needs to be compiled. Some (e.g. DKMS, or R packages containing native code, or common-lisp-controller) even go as far as compiling code on the user's system, which defeats the purpose of a precompiled distribution entirely. Even for scripting languages, bytecode compilation is often necessary or at least beneficial, and the bytecode depends on the version of the scripting language (and thus on the distribution and its version). So there is really no way around packages compiled natively for the distribution, i.e., in our case, RPMs. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
Vít Ondruch wrote: Sorry, this has nothing to do with FPC yet. RPM/YUM/DNF should first provide reasonable support. For example this issue [1] could take us closer as a first approximation. Vít [1] https://bugzilla.redhat.com/show_bug.cgi?id=845247 Parallel-installing multiple versions of the same package is a flawed idea from the onset, the only package we support it for is the kernel (and that's a hack). I don't think it makes sense to try to support it elsewhere. It would be much more productive to just package the latest version of the library in Fedora (if necessary as an update) and make the client packages work with that (if necessary in a grouped update). Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On 13. 10. 2013 at 03:05:20, Alek Paunov wrote: On 04.10.2013 15:34, Jan Zelený wrote: If you have any other questions, comments or notes regarding the document, feel free to to use this list for the discussion. Where (list threads, wikis, sources) one should seek more details about the DB aspects of the plan, e.g.: * A1: Delta metadata in yum and dnf (format, replication mechanism) We have an early proof of concept of this one but the final design is still not 100% clear so there is no information on this. I'm CCing Zdenek who is working on delta metadata, feel free to ask him about the details. * B1-4: New format of rpmdb (db engine, schema, relation with the future yum/dnf db) This is in even more early stages. Currently Jan Silhan (CCed) is working on this. If you have any suggestions/requirements, feel free to start the discussion. This topic has also been discussed several months ago on rpm-maint list [1], however the scope was much more narrow. [1] http://lists.rpm.org/pipermail/rpm-maint/2013-April/003520.html Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On 13. 10. 2013 at 22:19:16, Michael Stahnke wrote: On Fri, Oct 4, 2013 at 5:34 AM, Jan Zelený jzel...@redhat.com wrote: Hello everyone, as you might remember I issued a call for RFEs on this list during the spring. The participation was not bad at all, we have collected so many data that it took us several months to discuss and process it. Now I have some results for you. Attached to this email you can find a strategy document that a) outlines the strategy that we will commit to in the next 3-5 years and b) contains all the RFEs that were recognized as valid RFEs and were accepted to be implemented as a part of our strategy. Please note that the rest of the RFEs from the discussion was also evaluated but most likely rejected. If your RFE is not on the list, you can drop me an email and I'll tell you more specific reasons why we decided not to put it on the list. If you have any other questions, comments or notes regarding the document, feel free to to use this list for the discussion. Thanks Jan PS: I'll be AFK for the weekend so I'll comment on your replies on Monday -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct First off, thanks for this. I'm glad some people are really trying to look at the path forward. I'm also sorry I missed your RFE then. High traffic-lists sometimes get skipped over :( I like the goals of the paper. My main concerns is this addresses technical issues that are already in play. It doesn't address several items that are not technical issues which IMHO, is the main reason RPM isn't used for everything. Developers don't do deployments with RPM...at least not inside Fedora. Anything sane is actually against Packaging Guidelines. So that becomes a problem, and developers skip it. If developers (or operations people) are savvy enough to make RPMs, they are used once and not shared because they wouldn't get accepted into Fedora/EPEL. Also, sometimes developers/deployments need multiple versions of things installed. This horse has been beaten enough. See [1] for advice how to package multiple versions of one package. That's the best we can do and that's also the best we consider being *reasonably* maintainable. Is there a an effort that complements this one on the policy/non-technical side? Well, if you have some problems with Fedora packaging policy, I suggest communicating your ideas to the Fedora Packaging Committee. If your request is really reasonable, I'm sure they'll reflect it in the policy. However the problem usually is that developers have very different idea of reasonable than distribution maintainers because both groups have very different goals. Not to be only negative here, take a look at the COPR initiative, I expect it will solve the problem you are talking about by offering external repositories that will be easily reachable from Fedora but won't be a part of the Fedora itself. The content of these repositories will be governed by the same law as Fedora packages are (SW patents, ...) but technical policies should be a lot less strict. Would that address your concerns? Thanks Jan [1] http://rpm.org/wiki/PackagerDocs/MultipleVersions -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On Tue, Oct 15, 2013 at 01:15:26PM +0200, Jan Zelený wrote: Not to be only negative here, take a look at the COPR initiative, I expect it will solve the problem you are talking about by offering external repositories that will be easily reachable from Fedora but won't be a part of the Fedora itself. The content of these repositories will be governed by the same law as Fedora packages are (SW patents, ...) but technical policies should be a lot less strict. Would that address your concerns? I won't speak for Michael, but I think the answer is no. COPRs fills a need, but it's _too_ wild west (no package signatures, for example). We need to support multiple language runtimes and native upstream packaging *in* Fedora. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On Sun, Oct 13, 2013 at 10:19:16PM -0700, Michael Stahnke wrote: Developers don't do deployments with RPM...at least not inside Fedora. Anything sane is actually against Packaging Guidelines. So that becomes a problem, and developers skip it. If developers (or Can you elaborate on anything sane? Also, sometimes developers/deployments need multiple versions of things installed. Is there a an effort that complements this one on the policy/non-technical side? Sort of. This is basically the charter of the Environments and Stacks Working Group -- see https://fedoraproject.org/wiki/Fedora.next/callfornominations and https://fedoraproject.org/wiki/Fedora.next/WG_Nominations -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On 10/14/2013 05:19 AM, Michael Stahnke wrote: On Fri, Oct 4, 2013 at 5:34 AM, Jan Zelený jzel...@redhat.com wrote: Hello everyone, as you might remember I issued a call for RFEs on this list during the spring. The participation was not bad at all, we have collected so many data that it took us several months to discuss and process it. Now I have some results for you. Attached to this email you can find a strategy document that a) outlines the strategy that we will commit to in the next 3-5 years and b) contains all the RFEs that were recognized as valid RFEs and were accepted to be implemented as a part of our strategy. Please note that the rest of the RFEs from the discussion was also evaluated but most likely rejected. If your RFE is not on the list, you can drop me an email and I'll tell you more specific reasons why we decided not to put it on the list. If you have any other questions, comments or notes regarding the document, feel free to to use this list for the discussion. Thanks Jan PS: I'll be AFK for the weekend so I'll comment on your replies on Monday -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct First off, thanks for this. I'm glad some people are really trying to look at the path forward. I'm also sorry I missed your RFE then. High traffic-lists sometimes get skipped over :( I like the goals of the paper. My main concerns is this addresses technical issues that are already in play. It doesn't address several items that are not technical issues which IMHO, is the main reason RPM isn't used for everything. Developers don't do deployments with RPM...at least not inside Fedora. Anything sane is actually against Packaging Guidelines. So that becomes a problem, and developers skip it. If developers (or operations people) are savvy enough to make RPMs, they are used once and not shared because they wouldn't get accepted into Fedora/EPEL. Also, sometimes developers/deployments need multiple versions of things installed. Is there a an effort that complements this one on the policy/non-technical side? FESCO recently approved SCL which was a bit odd since Tom had explicetly put a big fat warning [1] that reads *Not approved for Fedora Packages* Please note that official Fedora packages *must not* be configured as Software Collection packages. Fedora does not permit relocatable packages, packages using hierarchies that conflict or violate the FHS, or packages storing files in /opt. This documentation is *NOT* part of the Fedora Packaging Guidelines, and is only here should you wish to generate unofficial Software Collections against Fedora in a third-party repository. as opposed to be pushing things forward by only approving the application stack being installed in an isolate container. ( which needs to be solved anyone on the not to distant future ) I do believe the SCL is what you are looking for... 1. https://fedoraproject.org/w/index.php?title=SoftwareCollections -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On Mon, Oct 14, 2013 at 6:46 AM, Matthew Miller mat...@fedoraproject.org wrote: On Sun, Oct 13, 2013 at 10:19:16PM -0700, Michael Stahnke wrote: Developers don't do deployments with RPM...at least not inside Fedora. Anything sane is actually against Packaging Guidelines. So that becomes a problem, and developers skip it. If developers (or Can you elaborate on anything sane? Basically having to break out applications into FHS pathing when it's something like a ruby application that is all in on directory normally. Putting it in /opt/application_name or /srv/appname is pretty typical in the field. Doing that in Fedora is a sin. The people who package things like redmine or mediawiki spend more time making symlinks and debugging pathing issues than actually being able to use the software. (At least when I maintained a few web apps, that was how I felt). I don't love the all-in-one container style deployment, but I think it's the least-worst way to deploy many types of applications. Also, sometimes developers/deployments need multiple versions of things installed. Is there a an effort that complements this one on the policy/non-technical side? Sort of. This is basically the charter of the Environments and Stacks Working Group -- see https://fedoraproject.org/wiki/Fedora.next/callfornominations and https://fedoraproject.org/wiki/Fedora.next/WG_Nominations -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On Fri, Oct 4, 2013 at 5:34 AM, Jan Zelený jzel...@redhat.com wrote: Hello everyone, as you might remember I issued a call for RFEs on this list during the spring. The participation was not bad at all, we have collected so many data that it took us several months to discuss and process it. Now I have some results for you. Attached to this email you can find a strategy document that a) outlines the strategy that we will commit to in the next 3-5 years and b) contains all the RFEs that were recognized as valid RFEs and were accepted to be implemented as a part of our strategy. Please note that the rest of the RFEs from the discussion was also evaluated but most likely rejected. If your RFE is not on the list, you can drop me an email and I'll tell you more specific reasons why we decided not to put it on the list. If you have any other questions, comments or notes regarding the document, feel free to to use this list for the discussion. Thanks Jan PS: I'll be AFK for the weekend so I'll comment on your replies on Monday -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct First off, thanks for this. I'm glad some people are really trying to look at the path forward. I'm also sorry I missed your RFE then. High traffic-lists sometimes get skipped over :( I like the goals of the paper. My main concerns is this addresses technical issues that are already in play. It doesn't address several items that are not technical issues which IMHO, is the main reason RPM isn't used for everything. Developers don't do deployments with RPM...at least not inside Fedora. Anything sane is actually against Packaging Guidelines. So that becomes a problem, and developers skip it. If developers (or operations people) are savvy enough to make RPMs, they are used once and not shared because they wouldn't get accepted into Fedora/EPEL. Also, sometimes developers/deployments need multiple versions of things installed. Is there a an effort that complements this one on the policy/non-technical side? -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software management: Call for RFEs results!
On 04.10.2013 15:34, Jan Zelený wrote: If you have any other questions, comments or notes regarding the document, feel free to to use this list for the discussion. Where (list threads, wikis, sources) one should seek more details about the DB aspects of the plan, e.g.: * A1: Delta metadata in yum and dnf (format, replication mechanism) * B1-4: New format of rpmdb (db engine, schema, relation with the future yum/dnf db) Thanks, Alek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Software Management call for RFEs
On Wed, May 22, 2013 at 03:43:32PM +0200, Jan Zelený wrote: Please send your requests as replies to this email so they can be properly discussed. I know I'm a little slow on this, and I hope I didn't miss a duplicate in the big thread, but here's mine: Better handling of documentation and languages which aren't installed initially. We often get requests to remove these things from the cloud image (see for example http://groveronline.com/2013/06/fedora-for-servers/, or look what oVirt node does). This would save a considerable amount of space -- in the cloud images, docs and man pages are 5% of the installed footprint and locale is 10%. That's a pretty big win for one thing. We have been reluctant to use the big RPM flag hammers to prevent these from being installed, though, because we want the core to be a minimal _building block_, and there's no graceful way to add those things back if they're initially not selected. So, handling this better would really help. One approach for docs might be to automatically separate %doc files into a -doc subpackage, and have a yum option which automatically installs those subpackages when the main package is installed. That's a bit ugly but wouldn't take a lot of changes. Or something like deltarpm could be used to add in missing doc and lang files if requested. But I'm not really wedded to any solution; just offering up a problem we need solved. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 2. 6. 2013 at 14:43:15, enclair wrote: I'd like a tool similar to portaudit in FreeBSD or debscan in Debian. This tool should list all packages which have a security issue. Currently there is yum-security-plugin but it lists packages only if an update is available. The new tool would list vulnerable packages even if no update is available yet, so that the user can take precaution. Are you sure this is within the scope of Software management? I'd say that the line is exactly between the existing and requested solution. The existing solution is within our scope (there is a security update) while the requested is more of an independent tool taking information from any place there is for this sort of information (there has been a security risk reported at XXX, take a look). Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 06/02/2013 02:43 PM, enclair wrote: I'd like a tool similar to portaudit in FreeBSD or debscan in Debian. This tool should list all packages which have a security issue. I don't know about portaudit, but debsecan works completely out of the usual software management stack. Part of the reason for that is that you even get reports if you haven't configured the security archive properly (so that the package manager won't notice that there are updates available). The real work is in the backend and the data collection; debsecan is a short Python script which just runs a few version comparisons. In Debian's case, this isn't fully integrated with the repository management, either, which is mostly due to historical accident and not deliberate design. But the key point is that this is not a question of software. It's all about the data that describes vulnerabilities and fixed packages, and this is currently not available for Fedora in consistent, machine-readable form. -- Florian Weimer / Red Hat Product Security Team -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Sun, Jun 02, 2013 at 02:43:15PM +0200, enclair wrote: I'd like a tool similar to portaudit in FreeBSD or debscan in Debian. This tool should list all packages which have a security issue. Currently there is yum-security-plugin but it lists packages only if an update is available. The new tool would list vulnerable packages even if no update is available yet, so that the user can take precaution. You probably want SCAP (specifically OpenSCAP). It's the proper solution for all of this. However the real issue is going to be that the OpenSCAP vulnerability databases for Fedora are not maintained. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
I'd like a tool similar to portaudit in FreeBSD or debscan in Debian. This tool should list all packages which have a security issue. Currently there is yum-security-plugin but it lists packages only if an update is available. The new tool would list vulnerable packages even if no update is available yet, so that the user can take precaution. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Wed, May 29, 2013 at 08:33:16PM +0200, Florian Weimer wrote:
Users sometimes misdiagnose issues, *especially* when complaining. 8-)
I did some tests and cold cache performance tests on an old Debian
installation. Performance with cold caches is more than adequate.
Full-text searches take about two seconds. Package installation reaches
the confirmation prompt (after dependency resolution) in less than three
seconds, even for ridiculously complex tasks such as installing the entire
KDE desktop (365 additional packages on my test system).
In contrast, on Fedora or RHEL systems, 30 seconds for dependency
resolution with a cold cache are common, plus around 6 seconds for loading
all the Python code for yum. /usr/bin/time reports much more I/O than it
does non Debian (about ten times as much, as reflected in the wall clock
time).
If you want to do some measurements for depsolving without the python
overhead, you can try the libsolv-demo package (it's a subpackage
of libsolv, so it should be available for Fedora). It consists of
a single binary, /usr/bin/solv, that is a little package manager
on top of libsolv.
Cheers,
Michael.
--
Michael Schroeder m...@suse.de
SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg
main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);}
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/29/2013 11:06 PM, James Antill wrote: On Wed, 2013-05-29 at 20:33 +0200, Florian Weimer wrote: I did some tests and cold cache performance tests on an old Debian installation. Performance with cold caches is more than adequate. Full-text searches take about two seconds. Package installation reaches the confirmation prompt (after dependency resolution) in less than three seconds, even for ridiculously complex tasks such as installing the entire KDE desktop (365 additional packages on my test system). In contrast, on Fedora or RHEL systems, 30 seconds for dependency resolution with a cold cache are common, plus around 6 seconds for loading all the Python code for yum. /usr/bin/time reports much more I/O than it does non Debian (about ten times as much, as reflected in the wall clock time). Yeh, it's almost like yum is dealing with more data: http://yum.baseurl.org/wiki/apt2yum#Generalpoints Are the yum numbers current? I don't think filelists are processed by current yum if that can be avoided. Anyway, let's hope that DNF will make all that history. 8-) -- Florian Weimer / Red Hat Product Security Team -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Wed, May 22, 2013 at 3:43 PM, Jan Zelený jzel...@redhat.com wrote: Dear Fedora community, several months ago, at the Developer conference in Brno, Software Management team received a whole bunch of proposals for new functionality in RPM and related software stack. We acknowledge the need for some changes in Software Management stack in Fedora but we don't want to make changes just by guessing what our users want. Therefore I call to you, consumers of our products (dnf, yum and rpm): what are the changes that you would like to see in the foreseeable future (say 2-3 years) and why would you like to see them (what would they help you with)? There is already a list of some RFEs on rpm.org wiki, you can use it as an inspiration, to see what RFEs we have already received: http://rpm.org/wiki/FeaturePlanning The only limitation for your requests is our manifest which defines the scope of SW management stack for the future. It is attached to this email (note that it's quite extensive but the first part should give you a good image of what is the planned scope of SW management stack). Please send your requests as replies to this email so they can be properly discussed. How about improving delta rpm performance? Currently we save download time but require a lot of time to rebuild the rpms. Can we just sign the deltas and then don't compress the generated rpms? We waste time and cycles building xz compressed rpms (from the deltas) just to decompress them a few minutes later. Skipping this (just create uncompressed rpms when building from deltas) should improve performance a lot. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
drago01 (drag...@gmail.com) said: How about improving delta rpm performance? Currently we save download time but require a lot of time to rebuild the rpms. Can we just sign the deltas and then don't compress the generated rpms? We waste time and cycles building xz compressed rpms (from the deltas) just to decompress them a few minutes later. Skipping this (just create uncompressed rpms when building from deltas) should improve performance a lot. The entire delta framework is built around generating the original RPM and handing that off wholesale to yum/rpm - it's how it can be done as a yum plugin without changing anything in RPM itself. Changing it to create something *different* than the full RPM (in terms of checksum, size, signature, and so on) would be a non-trivial change to how they work. I think if you want to go down this road, you probably want to start by integrating delta technology into RPM core itself, probably redefining the delta format into something much more efficient. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Thu, May 30, 2013 at 4:13 PM, Bill Nottingham nott...@redhat.com wrote: drago01 (drag...@gmail.com) said: How about improving delta rpm performance? Currently we save download time but require a lot of time to rebuild the rpms. Can we just sign the deltas and then don't compress the generated rpms? We waste time and cycles building xz compressed rpms (from the deltas) just to decompress them a few minutes later. Skipping this (just create uncompressed rpms when building from deltas) should improve performance a lot. The entire delta framework is built around generating the original RPM and handing that off wholesale to yum/rpm - it's how it can be done as a yum plugin without changing anything in RPM itself. Changing it to create something *different* than the full RPM (in terms of checksum, size, signature, and so on) would be a non-trivial change to how they work. Yeah but the goal is to have the same files installed on disk, not really a 1:1 copy of the rpm file (doing this just causes additional work). I think if you want to go down this road, you probably want to start by integrating delta technology into RPM core itself, probably redefining the delta format into something much more efficient. Yeah that makes sense. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Am 30.05.2013 16:13, schrieb Bill Nottingham:
drago01 (drag...@gmail.com) said:
How about improving delta rpm performance? Currently we save download
time but require a lot of time to rebuild the rpms.
Can we just sign the deltas and then don't compress the generated
rpms? We waste time and cycles building xz
compressed rpms (from the deltas) just to decompress them a few
minutes later. Skipping this (just create uncompressed rpms when
building from deltas) should improve performance a lot.
The entire delta framework is built around generating the original
RPM and handing that off wholesale to yum/rpm - it's how it can be
done as a yum plugin without changing anything in RPM itself. Changing
it to create something *different* than the full RPM (in terms of
checksum, size, signature, and so on) would be a non-trivial change
to how they work.
and it would break infrastructure like below where the one admin-machine
get any update, benefits from delta-rpms and deploy the updates later
to all other internal machines - you can calculate how much traffic
is saved on both sides (mine and fedora infrastructure) with this fro
more than 20 machines
[root@buildserver:~]$ cat /buildserver/repo-cache.sh
#!/usr/bin/bash
basearch=`uname -i`
releasever=`rpm -q --qf %{version}\n fedora-release`
# Alle Subfolder unter /var/cache/yum durchlaufen und RPM-Pakete
# in das eigene Repo verschieben
for g in `ls -1b /var/cache/yum`
do
if [ -d /var/cache/yum/$g/packages ]
then
echo /var/cache/yum/$g/packages/ /repo/cache/fc$releasever/
sudo mv --verbose /var/cache/yum/$g/packages/*.rpm /repo/cache/fc$releasever/
2 /dev/null
fi
done
/buildserver/repo-create.sh
signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Tue, May 28, 2013 at 11:51:21AM -0400, seth vidal wrote: I simply got tired of tilting at that particular windmill when confronted with some particularly egregious cases (see libguestfs sometime). $ rpm -qR libguestfs|grep ^/ /sbin/ldconfig /sbin/ldconfig /lib64/rtkaio/librt.so.1 /usr/lib64/sasl2/libanonymous.so.2 /usr/lib64/sasl2/libsasldb.so.2 To resolve 3 strings we have to download 26 MB of data. Getting rid of filelists seems like a bad idea because they are so useful. Implementing them better on the other hand ... At the moment they are stored in a sqlite database which is bzip2 compressed. The filelists DB for Fedora 18 is 26 MB compressed or 143 MB uncompressed. The sqlite database just stores basically the strings as-is. There are some structures which are better for storing strings that have a lot of common prefixes, such as tries and suffix trees. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Also do we know how many mirrors support byte ranges? We could go all the way and have a relatively large uncompressed database stored on the mirrors, but have the client only access small byte ranges from it. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Wed, 29 May 2013 11:52:04 +0100 Richard W.M. Jones rjo...@redhat.com wrote: Also do we know how many mirrors support byte ranges? We could go all the way and have a relatively large uncompressed database stored on the mirrors, but have the client only access small byte ranges from it. We used to use byte-ranges but what we discovered is how many proxies do NOT support byte-ranges and how quickly that falls apart. -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Wed, 29 May 2013 11:48:14 +0100 Richard W.M. Jones rjo...@redhat.com wrote: On Tue, May 28, 2013 at 11:51:21AM -0400, seth vidal wrote: I simply got tired of tilting at that particular windmill when confronted with some particularly egregious cases (see libguestfs sometime). $ rpm -qR libguestfs|grep ^/ /sbin/ldconfig /sbin/ldconfig /lib64/rtkaio/librt.so.1 /usr/lib64/sasl2/libanonymous.so.2 /usr/lib64/sasl2/libsasldb.so.2 this must be in f19 - in f18 I see: /lib/i686/nosegneg/libc.so.6 /lib/i686/nosegneg/libm.so.6 /lib/i686/nosegneg/libpthread.so.0 /lib/i686/nosegneg/librt.so.1 /lib/i686/nosegneg/libthread_db.so.1 /lib/rtkaio/i686/nosegneg/librt.so.1 /lib/rtkaio/librt.so.1 /sbin/ldconfig /usr/lib/sasl2/libanonymous.so.2 /usr/lib/sasl2/libsasldb.so.2 /usr/lib/sse2/libgmp.so.10 /usr/lib/sse2/libgmpxx.so.4 /usr/lib/sse2/libmp.so.3 /lib64/rtkaio/librt.so.1 /sbin/ldconfig /usr/lib64/sasl2/libanonymous.so.2 /usr/lib64/sasl2/libsasldb.so.2 and it was much much worse in the past. To resolve 3 strings we have to download 26 MB of data. Getting rid of filelists seems like a bad idea because they are so useful. Implementing them better on the other hand ... At the moment they are stored in a sqlite database which is bzip2 compressed. The filelists DB for Fedora 18 is 26 MB compressed or 143 MB uncompressed. The sqlite database just stores basically the strings as-is. There are some structures which are better for storing strings that have a lot of common prefixes, such as tries and suffix trees. Actually the sqlite db doesn't just store the strings it stores a table which has a pkg id (a number) then all the files in a specific dir for each row. like: 13960|/usr/share/locale/pt/LC_MESSAGES|gnokii.mo|f 13960|/usr/share/man/man8|mgnokiidev.8.gz/gnokiid.8.gz|ff 13960|/usr/share/doc/gnokii-0.6.31|sample/protocol/ringtones.txt/logos.txt/gnokii.nol/gnokii-ir-howto/gnokii-hackers-howto/gnokii-IrDA-Linux/gettext-howto/TODO/README.libsms/README-siemens/README-ericsson/README-dancall/README-WINDOWS/README-Symbian/README-PCSC/README-MacOSX/README-DKU2/README-7110/README-6510/README-6110/README-3810/README-2110/README/MAINTAINERS/KNOWN_BUGS/FAQ/DataCalls-QuickStart/CodingStyle/ChangeLog/CREDITS/COPYRIGHT/COPYING/Bugs just as an example. -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/27/2013 10:24 AM, Jan Zelený wrote: As far as I can tell, the main difference is that apt-get and apt-cache read very few, relatively large files at the beginning, so they don't block on disk reads early. dpkg, on the other hand, uses a database scatter across many small files on disk, so you get the delay only when you actually install or remove any packages. At the beginning, this is quite fast, but eventually, the files will be scattered quite badly, and there is a considerable delay at this step. This part is about disk read-write but that was not what I was writing about. From my experience users mostly complain about the metadata download which is explained above. Users sometimes misdiagnose issues, *especially* when complaining. 8-) I did some tests and cold cache performance tests on an old Debian installation. Performance with cold caches is more than adequate. Full-text searches take about two seconds. Package installation reaches the confirmation prompt (after dependency resolution) in less than three seconds, even for ridiculously complex tasks such as installing the entire KDE desktop (365 additional packages on my test system). In contrast, on Fedora or RHEL systems, 30 seconds for dependency resolution with a cold cache are common, plus around 6 seconds for loading all the Python code for yum. /usr/bin/time reports much more I/O than it does non Debian (about ten times as much, as reflected in the wall clock time). Regarding network traffic, in addition to the explicit apt-get update step (which avoids time-consuming downloads in the first place), Debian also forces users to pick a single mirror close to them. The Fedora master mirror list instructs yum to pick a mirror from a larger list, which causes much greater variance in performance. -- Florian Weimer / Red Hat Product Security Team -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Wed, 2013-05-29 at 20:33 +0200, Florian Weimer wrote: I did some tests and cold cache performance tests on an old Debian installation. Performance with cold caches is more than adequate. Full-text searches take about two seconds. Package installation reaches the confirmation prompt (after dependency resolution) in less than three seconds, even for ridiculously complex tasks such as installing the entire KDE desktop (365 additional packages on my test system). In contrast, on Fedora or RHEL systems, 30 seconds for dependency resolution with a cold cache are common, plus around 6 seconds for loading all the Python code for yum. /usr/bin/time reports much more I/O than it does non Debian (about ten times as much, as reflected in the wall clock time). Yeh, it's almost like yum is dealing with more data: http://yum.baseurl.org/wiki/apt2yum#Generalpoints -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Tue, 2013-05-28 at 14:18 -0400, seth vidal wrote: * Why the sql schema is so denormalized (IMO, leads to both bandwidth and disk overspending without speed benefits)?. For example: Why provides and requires tables do not use the common domain table? B/c it was designed 8yrs ago and we were going for compressable space and making it as quick as possible to search? It wasn't just that, I did the work to normalize it ~5 years ago: http://james.fedorapeople.org/yum/patches/yum-metadata-parser-ids.patch ...but the download savings were so minimal (IIRC ~1% saving on the .bz2 file) it just didn't seem worth the effort, and any downsides. Ofc. anyone else has been free to run with it if they want to. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 27. 5. 2013 at 10:31:29, Reindl Harald wrote: Am 27.05.2013 09:32, schrieb Jan Zelený: On 25. 5. 2013 at 09:34:32, Nico Kadel-Garcia wrote: On Wed, May 22, 2013 at 11:55 AM, Michael Ekstrand mich...@elehack.net wrote: Performance improvement: improve scaling to 5K+ installed packages. * Amen. This is particularly compounded by poor caching default behavior, so that a few yum commands in a row each wind up reaching out to downloading metadata again, and again, and again. I think this can be addressed by moving the metadata updates to a different function, and calling it *separately* only as needed. The Debian apt tool does this quite effectively. Unfortunately there is not much we can do about this. Debian has completely different repository policy - they keep all versions of packages in the repo so there is no need to update metadata on client machines every time what does keep old versions or not change besides you need to do apt-get update if you want to find apt-get upgrade to find new packages? Something like that, yes. If you don't update your metadata you won't get updates but installation will still work, even if the installed version is not the latest available. the real problem is that the metadata are *way too fat* in Fedora That's not the real problem. It's just one of the problems which in combination cause this pain. after a yum clean metadata yum update on a slow line you have to wait a very long time and even the download of the presto-metadata often is larger and takes longer as the packages which are updated in reality hey on my 100 Mbit all is nice and fine but on a machine behind DSL with around 100 KB/Second it is way too slow and large and i refuse to imagine how this feels on a 56kbit modem I couldn't agree more. But as I have said, we need to find the most simple and unintrusive things that can be done to improve this. For instance: file lists take a considerable portion of the entire metadata size. But if we were to remove them, things like yum install /usr/bin/vim would not work any more. And you get similar scenario with almost all the metadata that we store - we store them for a reason and without them some things that people use will not work. Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/22/2013 05:47 PM, Paulo César Pereira de Andrade wrote: As a packager, some way to transparently handle an upgrade when a directory changes to a symlink or vice-versa. +1 -- Miroslav Suchy Red Hat Systems Management Engineering -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/27/2013 09:32 AM, Jan Zelený wrote: Unfortunately there is not much we can do about this. Debian has completely different repository policy - they keep all versions of packages in the repo so there is no need to update metadata on client machines every time. Actually we can do something. Have diffs on metadata. Let imagine that yesterday I done full upgrade. Today PackageKit is saying that I have one package for upgrade. Avarage package has 800kB. Because of those few kB I have to download all those metadata again. On my machine it is 28 MB (!). But the diff on those metadata is actually just few kB. We have presto plugin for diffing rpm packages, but I would actually save more time and more bandwidth if we would have diff on metadatas. -- Miroslav Suchy Red Hat Systems Management Engineering -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 28.05.2013 09:51, Jan Zelený wrote: I couldn't agree more. But as I have said, we need to find the most simple and unintrusive things that can be done to improve this. For instance: file lists take a considerable portion of the entire metadata size. But if we were to remove them, things like yum install /usr/bin/vim would not work any more. And you get similar scenario with almost all the metadata that we store - we store them for a reason and without them some things that people use will not work. Not so unintrusive, but would it be acceptable if we merge all .sqlite DBs from all repos in a single .sqlite with tree-like schema? Let say we achieve overall size reduction by factor of 4, at the price of more complicated but faster SQL queries. [I hope that such a change will also make the incremental by the XML sources easier] I am going to experiment this, if make sense ... -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 28. 5. 2013 at 11:39:35, Alek Paunov wrote: On 28.05.2013 09:51, Jan Zelený wrote: I couldn't agree more. But as I have said, we need to find the most simple and unintrusive things that can be done to improve this. For instance: file lists take a considerable portion of the entire metadata size. But if we were to remove them, things like yum install /usr/bin/vim would not work any more. And you get similar scenario with almost all the metadata that we store - we store them for a reason and without them some things that people use will not work. Not so unintrusive, but would it be acceptable if we merge all .sqlite DBs from all repos in a single .sqlite with tree-like schema? Let say we achieve overall size reduction by factor of 4, at the price of more complicated but faster SQL queries. [I hope that such a change will also make the incremental by the XML sources easier] I am going to experiment this, if make sense ... Perhaps it's just that I don't fully understand your vision but I'm not sure that's a feasible solution. How exactly would you solve the fact that users have different repos enabled on their machines? Or did you mean merging them on the client side? That would not solve the issue of data download. Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/22/2013 03:43 PM, Jan Zelený wrote: Please send your requests as replies to this email so they can be properly discussed. Have equivalent of apt-get autoremove. I.e. If you run yum install foo and it will install packages bar and bra for dependencies. Then when I remove package foo, those two packages will be left on my system. dnf autoremove should tell me that packages bar and bra were installed as dependencies for package, which is no more present on disk (and no other package requires them) and can be removed. -- Miroslav Suchy Red Hat Systems Management Engineering -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Tue, 2013-05-28 at 13:14 +0200, Miroslav Suchý wrote: On 05/22/2013 03:43 PM, Jan Zelený wrote: Please send your requests as replies to this email so they can be properly discussed. Have equivalent of apt-get autoremove. That's what yum-plugin-remove-with-leaves does. -- Mathieu -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 28.05.2013 13:54, Jan Zelený wrote: On 28. 5. 2013 at 11:39:35, Alek Paunov wrote: On 28.05.2013 09:51, Jan Zelený wrote: I couldn't agree more. But as I have said, we need to find the most simple and unintrusive things that can be done to improve this. For instance: file lists take a considerable portion of the entire metadata size. But if we were to remove them, things like yum install /usr/bin/vim would not work any more. And you get similar scenario with almost all the metadata that we store - we store them for a reason and without them some things that people use will not work. Not so unintrusive, but would it be acceptable if we merge all .sqlite DBs from all repos in a single .sqlite with tree-like schema? Let say we achieve overall size reduction by factor of 4, at the price of more complicated but faster SQL queries. [I hope that such a change will also make the incremental by the XML sources easier] I am going to experiment this, if make sense ... Perhaps it's just that I don't fully understand your vision but I'm not sure that's a feasible solution. How exactly would you solve the fact that users have different repos enabled on their machines? Or did you mean merging them on the client side? That would not solve the issue of data download. Oh sorry. On the client side of course - these which are under the /var/cache/yum tree. My question was more targeted on the complains about 1) metadata local size, 2) speed and 3) capabilities of the local yum queries. [Furthermore you currently reformulating Package Management as Software Lifecycle Management, so it would be normal for me to expect that the data backend will have to be enhanced accordingly. Or will libsolv stores be enough for all?] Regarding the metadata download speedup, I completely agree with Florian and others on the thread, that current bulk downloads should be replaced with XML only downloads and incremental update of the local DB as a first step and introducing *optional* metadata services (just XML git or something smarter) for the mirrors which are willing to upgrade. Thank you, Alek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/28/2013 06:25 AM, Mathieu Bridon wrote: On Tue, 2013-05-28 at 13:14 +0200, Miroslav Suchý wrote: On 05/22/2013 03:43 PM, Jan Zelený wrote: Please send your requests as replies to this email so they can be properly discussed. Have equivalent of apt-get autoremove. That's what yum-plugin-remove-with-leaves does. Almost, but not quite (it must be done when you remove the package, can't be done after). The clean_requirements_on_remove=1 setting is also helpful, like 'autoremove' after each 'remove'. Still can't be done after-the-fact, but is nice (and more accurate, AFAIK, since it uses the yumdb 'reason' tag). -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Am 28.05.2013 13:14, schrieb Miroslav Suchý: On 05/22/2013 03:43 PM, Jan Zelený wrote: Please send your requests as replies to this email so they can be properly discussed. Have equivalent of apt-get autoremove. I.e. If you run yum install foo and it will install packages bar and bra for dependencies. Then when I remove package foo, those two packages will be left on my system not on mine, the only big question is why it is not default cat /etc/yum.conf | grep clean clean_requirements_on_remove=1 signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/28/2013 01:14 PM, Miroslav Suchý wrote: dnf autoremove should tell me that packages bar and bra were installed as dependencies for package, which is no more present on disk (and no other package requires them) and can be removed. There's an RFE for this already: https://bugzilla.redhat.com/show_bug.cgi?id=963345 Ales -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
This is basically the major impediment to the uninstall of a product that consists of several packages. Some of these packages may be, at time of uninstall, also required by other packages, so the and no other package requires them part is essential. --Fernando - Original Message - From: Ales Kozumplik akozu...@redhat.com To: Development discussions related to Fedora devel@lists.fedoraproject.org Sent: Tuesday, May 28, 2013 9:27:23 AM Subject: Re: Software Management call for RFEs On 05/28/2013 01:14 PM, Miroslav Suchý wrote: dnf autoremove should tell me that packages bar and bra were installed as dependencies for package, which is no more present on disk (and no other package requires them) and can be removed. There's an RFE for this already: https://bugzilla.redhat.com/show_bug.cgi?id=963345 Ales -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/24/2013 09:20 PM, Rahul Sundaram wrote: On 05/23/2013 07:08 AM, Jan Zelený wrote: Have you tried using dnf instead of yum? It is much faster. To be perfectly honest we don't plan to invest much effort in developing new things for yum, it will more and more shift towards maintenance mode and the focus will be on dnf. What does the we stand for here? If this is Red Hat on the whole, a broader announcement of the plan would be helpful. I will note that while dnf is faster overall, it doesn't have many of the important features of yum and I still ran into bugs in some trivial tests from time to time. https://fedoranext.wordpress.com/2013/05/18/status-of-dnf-experimental-fork-of-yum/ Rahul The one weird bug (dnf remove wants to install new packages) mentioned by the article just got fixed: https://bugzilla.redhat.com/show_bug.cgi?id=916662 Ales -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Tue, 28 May 2013 08:51:03 +0200 Jan Zelený jzel...@redhat.com wrote: after a yum clean metadata yum update on a slow line you have to wait a very long time and even the download of the presto-metadata often is larger and takes longer as the packages which are updated in reality hey on my 100 Mbit all is nice and fine but on a machine behind DSL with around 100 KB/Second it is way too slow and large and i refuse to imagine how this feels on a 56kbit modem I couldn't agree more. But as I have said, we need to find the most simple and unintrusive things that can be done to improve this. For instance: file lists take a considerable portion of the entire metadata size. But if we were to remove them, things like yum install /usr/bin/vim would not work any more. And you get similar scenario with almost all the metadata that we store - we store them for a reason and without them some things that people use will not work. Jan, the above is not correct. Files in *bin/* are in the primary metadata - not in the filelists. That was specifically designed to handle the 90% of file-deps which are *bin/* or /etc/*. It's not accidental. so if you nuked filelists entirely you'd only lose people who have filedeps on something outside of those wildcards above. I've spent HERCULEAN amounts of effort to whittle down the set of filedeps outside of that area. I filed hundreds of bugs on the subject in years passed. I simply got tired of tilting at that particular windmill when confronted with some particularly egregious cases (see libguestfs sometime). But it is absolutely NOT TRUE that removing filelists will cause 'yum install /usr/bin/vim' to not work. If you have further questions about how the metadata works or was designed please feel free to ask me, directly. I believe I and Adrian Likins are the only current Red Hat Employees or Fedora Contributors who were present/involved in its original 'design'. Such as it was. It might prove helpful to know that background to avoid walking down blindalleys in DNF. Thanks! -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Tue, 2013-05-28 at 13:14 +0200, Miroslav Suchý wrote: On 05/22/2013 03:43 PM, Jan Zelený wrote: Please send your requests as replies to this email so they can be properly discussed. Have equivalent of apt-get autoremove. Try yum autoremove in F19. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 28.05.2013 18:51, seth vidal wrote: On Tue, 28 May 2013 08:51:03 +0200 Jan Zelený jzel...@redhat.com wrote: after a yum clean metadata yum update on a slow line you have to wait a very long time and even the download of the presto-metadata often is larger and takes longer as the packages which are updated in reality hey on my 100 Mbit all is nice and fine but on a machine behind DSL with around 100 KB/Second it is way too slow and large and i refuse to imagine how this feels on a 56kbit modem I couldn't agree more. But as I have said, we need to find the most simple and unintrusive things that can be done to improve this. For instance: file lists take a considerable portion of the entire metadata size. But if we were to remove them, things like yum install /usr/bin/vim would not work any more. And you get similar scenario with almost all the metadata that we store - we store them for a reason and without them some things that people use will not work. Jan, the above is not correct. Files in *bin/* are in the primary metadata - not in the filelists. That was specifically designed to handle the 90% of file-deps which are *bin/* or /etc/*. It's not accidental. so if you nuked filelists entirely you'd only lose people who have filedeps on something outside of those wildcards above. I've spent HERCULEAN amounts of effort to whittle down the set of filedeps outside of that area. I filed hundreds of bugs on the subject in years passed. I simply got tired of tilting at that particular windmill when confronted with some particularly egregious cases (see libguestfs sometime). I just tested on a F18 box the following: yum erase -y datalog yum clean all yum install sqlite-datalog #3 non-existing package yum install -y /usr/bin/datalog In the above sequence, yum do not downloaded the filelists at all. yum erase -y datalog yum clean all yum install sqlite-datalog #3 non-existing package yum install -y /usr/share/lua/5.1/datalog.lua #4 In the second sequence, yum started the filelists downloading at #4. So, it seems that yum already have the filelists on demand optimization implemented. Why you are asking for removing a feature, which do not make the things worse ... ? If you have further questions about how the metadata works or was designed please feel free to ask me, directly. I believe I and Adrian Likins are the only current Red Hat Employees or Fedora Contributors who were present/involved in its original 'design'. Such as it was. I have a few questions: * What is the reasoning behind the splitting of the database across many .sqlite files? * Why the sql schema is so denormalized (IMO, leads to both bandwidth and disk overspending without speed benefits)?. For example: Why provides and requires tables do not use the common domain table? * Why the incremental update mechanism (eg. applying xml diffs to the sqlite database) was not been considered from the very beginning? Thanks! Alek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Tue, 28 May 2013 20:42:13 +0300 Alek Paunov a...@declera.com wrote: So, it seems that yum already have the filelists on demand optimization implemented. Why you are asking for removing a feature, which do not make the things worse ... ? I'm not. But when you download the filelists - it is A LOT of data. I'd rather not have filedeps so it doesn't get pulled in for other things in depsolving. I have a few questions: * What is the reasoning behind the splitting of the database across many .sqlite files? many? it's 3 afaik. primary, filelists, other. how do you mean 'many? * Why the sql schema is so denormalized (IMO, leads to both bandwidth and disk overspending without speed benefits)?. For example: Why provides and requires tables do not use the common domain table? B/c it was designed 8yrs ago and we were going for compressable space and making it as quick as possible to search? * Why the incremental update mechanism (eg. applying xml diffs to the sqlite database) was not been considered from the very beginning? It wasn't necessary? There was a massively smaller number of pkgs to consider. -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Dne Út 28. května 2013 10:11:55, Miroslav Suchý napsal(a): On 05/27/2013 09:32 AM, Jan Zelený wrote: Unfortunately there is not much we can do about this. Debian has completely different repository policy - they keep all versions of packages in the repo so there is no need to update metadata on client machines every time. Actually we can do something. Have diffs on metadata. Let imagine that yesterday I done full upgrade. Today PackageKit is saying that I have one package for upgrade. Avarage package has 800kB. Because of those few kB I have to download all those metadata again. On my machine it is 28 MB (!). But the diff on those metadata is actually just few kB. We have presto plugin for diffing rpm packages, but I would actually save more time and more bandwidth if we would have diff on metadatas. We have been working on this for some time now. However there are some other consequent problems that we need to figure out first. There are already some proposals that came up from our last week meeting with Richard Hughes. Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Dne Út 28. května 2013 09:33:11, Fernando Nasser napsal(a): This is basically the major impediment to the uninstall of a product that consists of several packages. Some of these packages may be, at time of uninstall, also required by other packages, so the and no other package requires them part is essential. yum repo-pkgs should handle the other situation (other packages require them) as well. Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Dne Út 28. května 2013 14:59:20, Alek Paunov napsal(a): On 28.05.2013 13:54, Jan Zelený wrote: On 28. 5. 2013 at 11:39:35, Alek Paunov wrote: On 28.05.2013 09:51, Jan Zelený wrote: I couldn't agree more. But as I have said, we need to find the most simple and unintrusive things that can be done to improve this. For instance: file lists take a considerable portion of the entire metadata size. But if we were to remove them, things like yum install /usr/bin/vim would not work any more. And you get similar scenario with almost all the metadata that we store - we store them for a reason and without them some things that people use will not work. Not so unintrusive, but would it be acceptable if we merge all .sqlite DBs from all repos in a single .sqlite with tree-like schema? Let say we achieve overall size reduction by factor of 4, at the price of more complicated but faster SQL queries. [I hope that such a change will also make the incremental by the XML sources easier] I am going to experiment this, if make sense ... Perhaps it's just that I don't fully understand your vision but I'm not sure that's a feasible solution. How exactly would you solve the fact that users have different repos enabled on their machines? Or did you mean merging them on the client side? That would not solve the issue of data download. Oh sorry. On the client side of course - these which are under the /var/cache/yum tree. My question was more targeted on the complains about 1) metadata local size, 2) speed and 3) capabilities of the local yum queries. I'm not sure merging database would help in any of these areas. But then again I can be wrong. If you plan to do the testing, I am very curious about the results. [Furthermore you currently reformulating Package Management as Software Lifecycle Management, so it would be normal for me to expect that the data backend will have to be enhanced accordingly. Or will libsolv stores be enough for all?] This has to be discussed along the way, at this point it is too early to say what will happen to metadata. Regarding the metadata download speedup, I completely agree with Florian and others on the thread, that current bulk downloads should be replaced with XML only downloads and incremental update of the local DB as a first step and introducing *optional* metadata services (just XML git or something smarter) for the mirrors which are willing to upgrade. We are performing some tests and so far it seems that for yum this is not the way to go, as the speed gain would be compensated by the local yum db rebuild. We will continue to work on this but so far it seems that dnf is the way to go so the optimization will most likely happen there. Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Dne Út 28. května 2013 11:51:21, seth vidal napsal(a): On Tue, 28 May 2013 08:51:03 +0200 Jan Zelený jzel...@redhat.com wrote: after a yum clean metadata yum update on a slow line you have to wait a very long time and even the download of the presto-metadata often is larger and takes longer as the packages which are updated in reality hey on my 100 Mbit all is nice and fine but on a machine behind DSL with around 100 KB/Second it is way too slow and large and i refuse to imagine how this feels on a 56kbit modem I couldn't agree more. But as I have said, we need to find the most simple and unintrusive things that can be done to improve this. For instance: file lists take a considerable portion of the entire metadata size. But if we were to remove them, things like yum install /usr/bin/vim would not work any more. And you get similar scenario with almost all the metadata that we store - we store them for a reason and without them some things that people use will not work. Jan, the above is not correct. Files in *bin/* are in the primary metadata - not in the filelists. That was specifically designed to handle the 90% of file-deps which are *bin/* or /etc/*. It's not accidental. so if you nuked filelists entirely you'd only lose people who have filedeps on something outside of those wildcards above. I've spent HERCULEAN amounts of effort to whittle down the set of filedeps outside of that area. I filed hundreds of bugs on the subject in years passed. I simply got tired of tilting at that particular windmill when confronted with some particularly egregious cases (see libguestfs sometime). But it is absolutely NOT TRUE that removing filelists will cause 'yum install /usr/bin/vim' to not work. If you have further questions about how the metadata works or was designed please feel free to ask me, directly. I believe I and Adrian Likins are the only current Red Hat Employees or Fedora Contributors who were present/involved in its original 'design'. Such as it was. It might prove helpful to know that background to avoid walking down blindalleys in DNF. I knew there were some optimizations in the filelist case, I just wasn't completely sure what exactly would cause downloads of the filelist. Anyway, thanks for the explanation Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 28.05.2013 21:18, seth vidal wrote: On Tue, 28 May 2013 20:42:13 +0300 Alek Paunov a...@declera.com wrote: So, it seems that yum already have the filelists on demand optimization implemented. Why you are asking for removing a feature, which do not make the things worse ... ? I'm not. But when you download the filelists - it is A LOT of data. It is of course :-). It is big and slow now, but it implements one more distinguishing and convenient Fedora feature ... and under careful schema and encoding, can be scaled down several times in both space and query time. Actually, every positive (install, update) yum operation implies access to the repos. Repos contain everything. If our software was perfectly optimized, not only filelists but all other parts of the database (including primary.files, which you have cited initially) should be lazily synced, right? I'd rather not have filedeps so it doesn't get pulled in for other things in depsolving. Sorry, I do not know how this amount of data will impact libsolv in the future. IMO, for yum (I mean in the sqlite based solution) it is a matter of optimizations. I have a few questions: * What is the reasoning behind the splitting of the database across many .sqlite files? many? it's 3 afaik. primary, filelists, other. how do you mean 'many? Multiplied by the number of the repos. That is what I am trying to understand - Why not just single .sqlite file for the whole yum database? * Why the sql schema is so denormalized (IMO, leads to both bandwidth and disk overspending without speed benefits)?. For example: Why provides and requires tables do not use the common domain table? B/c it was designed 8yrs ago and we were going for compressable space and making it as quick as possible to search? In the provides and requires example, we do not have any space/speed benefits achieved by the missing common domain (dependency + dependency_evr tables). In the current situation we have fat and slow text duplication and indexes instead of integer references to the domain subnodes (dependencies is the biggest domain in the primary). Yes, in bunch of cases a little denormalization is inevitable when we fight for speed, but IMO, this and few other space flaws are with negative impact on the speed too. * Why the incremental update mechanism (eg. applying xml diffs to the sqlite database) was not been considered from the very beginning? It wasn't necessary? There was a massively smaller number of pkgs to consider. Indeed. Also, 8 years ago the possibilities and the number of ideas to reuse were definitely different :-) Thank you, Alek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 25. 5. 2013 at 09:34:32, Nico Kadel-Garcia wrote: On Wed, May 22, 2013 at 11:55 AM, Michael Ekstrand mich...@elehack.net wrote: Performance improvement: improve scaling to 5K+ installed packages. * Amen. This is particularly compounded by poor caching default behavior, so that a few yum commands in a row each wind up reaching out to downloading metadata again, and again, and again. I think this can be addressed by moving the metadata updates to a different function, and calling it *separately* only as needed. The Debian apt tool does this quite effectively. Unfortunately there is not much we can do about this. Debian has completely different repository policy - they keep all versions of packages in the repo so there is no need to update metadata on client machines every time. * Script parseable output without extraneous labeling. The output of yum list is cluttered with unnecessary headers, and whitespace handling that winds up trimming package names or inserting extraneous new lines. I'd fiind it invaluable if yum list --tsv gave a tab separated variables list of: nameversion release arch reponame And leave *OFF* the Installed Packages and Available Packages entries for such tab separated variable output. I loathe having to sort those out for scriptable operations. Interesting idea, might be worth looking into. Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/27/2013 09:32 AM, Jan Zelený wrote: On 25. 5. 2013 at 09:34:32, Nico Kadel-Garcia wrote: On Wed, May 22, 2013 at 11:55 AM, Michael Ekstrand mich...@elehack.net wrote: Performance improvement: improve scaling to 5K+ installed packages. * Amen. This is particularly compounded by poor caching default behavior, so that a few yum commands in a row each wind up reaching out to downloading metadata again, and again, and again. I think this can be addressed by moving the metadata updates to a different function, and calling it *separately* only as needed. The Debian apt tool does this quite effectively. Unfortunately there is not much we can do about this. Debian has completely different repository policy - they keep all versions of packages in the repo so there is no need to update metadata on client machines every time. I don't quite understand this comment. Debian repository policy varies quite a bit. Some repositories keep old versions, some don't. Mostly the latter, actually, because not all repository managers (there a couple of implementations) can deal with multiple versions for a single package/architecture combination. As far as I can tell, the main difference is that apt-get and apt-cache read very few, relatively large files at the beginning, so they don't block on disk reads early. dpkg, on the other hand, uses a database scatter across many small files on disk, so you get the delay only when you actually install or remove any packages. At the beginning, this is quite fast, but eventually, the files will be scattered quite badly, and there is a considerable delay at this step. -- Florian Weimer / Red Hat Product Security Team -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 24. 5. 2013 at 15:20:24, Rahul Sundaram wrote: On 05/23/2013 07:08 AM, Jan Zelený wrote: Have you tried using dnf instead of yum? It is much faster. To be perfectly honest we don't plan to invest much effort in developing new things for yum, it will more and more shift towards maintenance mode and the focus will be on dnf. What does the we stand for here? If this is Red Hat on the whole, a broader announcement of the plan would be helpful. I will note that while dnf is faster overall, it doesn't have many of the important features of yum and I still ran into bugs in some trivial tests from time to time. https://fedoranext.wordpress.com/2013/05/18/status-of-dnf-experimental-fork- of-yum/ The we means the Software management team. Obviously the dnf is still nowhere near the state of yum but it's continuously getting there. If we don't stop development of features in yum at some point, it will be impossible to replace yum with dnf in the future. If you feel that there are some important features missing, please let us know which ones. Although we won't automatically prioritize all of them, we will take your input into consideration. Note that everyone can have his own set of features that he considers to be important. Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 27. 5. 2013 at 09:43:15, Florian Weimer wrote: On 05/27/2013 09:32 AM, Jan Zelený wrote: On 25. 5. 2013 at 09:34:32, Nico Kadel-Garcia wrote: On Wed, May 22, 2013 at 11:55 AM, Michael Ekstrand mich...@elehack.net wrote: Performance improvement: improve scaling to 5K+ installed packages. * Amen. This is particularly compounded by poor caching default behavior, so that a few yum commands in a row each wind up reaching out to downloading metadata again, and again, and again. I think this can be addressed by moving the metadata updates to a different function, and calling it *separately* only as needed. The Debian apt tool does this quite effectively. Unfortunately there is not much we can do about this. Debian has completely different repository policy - they keep all versions of packages in the repo so there is no need to update metadata on client machines every time. I don't quite understand this comment. Debian repository policy varies quite a bit. Some repositories keep old versions, some don't. Mostly the latter, actually, because not all repository managers (there a couple of implementations) can deal with multiple versions for a single package/architecture combination. I'm sorry but the Debian repositories say otherwise, see Iceweasel for instance: http://ftp.cz.debian.org/debian/pool/main/i/iceweasel/ Multiple old versions are kept in there. That's why they don't need to update their metadata every single time - the old ones are simply still valid. As far as I can tell, the main difference is that apt-get and apt-cache read very few, relatively large files at the beginning, so they don't block on disk reads early. dpkg, on the other hand, uses a database scatter across many small files on disk, so you get the delay only when you actually install or remove any packages. At the beginning, this is quite fast, but eventually, the files will be scattered quite badly, and there is a considerable delay at this step. This part is about disk read-write but that was not what I was writing about. From my experience users mostly complain about the metadata download which is explained above. Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Le Dim 26 mai 2013 11:10, Lars Seipel a écrit : On Sat, May 25, 2013 at 09:34:32AM -0400, Nico Kadel-Garcia wrote: I think this can be addressed by moving the metadata updates to a different function, and calling it *separately* only as needed. The Debian apt tool does this quite effectively. You can kind of simulate that by forcing yum to run from cache only. Use the -C flag for that. But the basic reason stands: yum only handles well perfectly synced repos, and workarounds this by trying to get the freshest metadata all the time. If yum's error handling was better, it would not fail as soon as there is the slightest sync error, a semi-stale repo, a caching proxy, etc, etc -- Nicolas Mailhot -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
And there package diffs, which are ed-style diffs of the Packages file I mentioned above. This approach would work quite well for primary.xml because it doesn't contain cross-references between packages using non-natural keys. It doesn't work for the SQLite database, either in binary or SQL dump format, because of the reliance on artificial primary keys (such as package IDs). I've once tried this. With about 10k packages in fedora-updates, the delta over 2-3 days was +491 -479. Assuming deletions are cheap, the delta should ideally be 5%. As expected, binary bsddiff yields much bigger (~29%) delta. Very roughly, it's 5% that really describe new packages, plus an almost constant 24% overhead to fix up the inevitable changes in surrogate keys. Not as bad as I was afraid, but still not worth it (IMO). So, we need *.xml deltas. Yum can rebuild xml = .sqlite locally, but this needs quite a lot of memory and takes TENS of seconds. Add the time needed to patch the quite large uncompressed xml file, and suddenly the fact that you're downloading just 1/10th of data hardly pays off (ignoring very specific use cases, like mobile data for a moment) For DNF, it's different. It has to rebuild xml = .solv anyway, so this comes for free. However, for many users that follow unstable or testing, package diffs are currently slower than downloading the full Packages file because the diffs are incremental (i.e., they contain the changes from file version N to N+1, and you have to apply all of them to get to the current version) and apt-get can easily write 100 MB or more because the Packages file is rewritten locally multiple times. Yes, patch chaining should be avoided. I'd like to use N = 1 deltas, that could be applied to many recent snapshots. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/27/2013 11:48 AM, Zdenek Pavlas wrote: And there package diffs, which are ed-style diffs of the Packages file I mentioned above. This approach would work quite well for primary.xml because it doesn't contain cross-references between packages using non-natural keys. It doesn't work for the SQLite database, either in binary or SQL dump format, because of the reliance on artificial primary keys (such as package IDs). I've once tried this. With about 10k packages in fedora-updates, the delta over 2-3 days was +491 -479. Assuming deletions are cheap, the delta should ideally be 5%. As expected, binary bsddiff yields much bigger (~29%) delta. A line-wise diff is much smaller because dependencies and package descriptions mostly stay the same. (This assumes consistent sorting of the primary.xml file.) Can you point me to the primary.xml - SQLite translation in yum? I've got a fairly efficient primary.xml parser. It might be interesting to see if it's possible to reduce the latency introduced by the SQLite conversion to close to zero. (Decompression and INSERTs can be interleaved with downloading, and maybe the index creation improvements in SQLite are sufficient these days.) However, for many users that follow unstable or testing, package diffs are currently slower than downloading the full Packages file because the diffs are incremental (i.e., they contain the changes from file version N to N+1, and you have to apply all of them to get to the current version) and apt-get can easily write 100 MB or more because the Packages file is rewritten locally multiple times. Yes, patch chaining should be avoided. I'd like to use N = 1 deltas, that could be applied to many recent snapshots. The Debian package diffs could be combined efficiently in the client because it's possible to combine diffs for two adjacent versions without actually knowing what the old or new versions look like. But this hasn't been implemented in APT because ABI impact (which is a bit puzzling, but anyway). Instead, the diffs should soon be combined on the archive side. -- Florian Weimer / Red Hat Product Security Team -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Mon, May 27, 2013 at 6:17 AM, Florian Weimer fwei...@redhat.com wrote: On 05/27/2013 11:48 AM, Zdenek Pavlas wrote: And there package diffs, which are ed-style diffs of the Packages file I mentioned above. This approach would work quite well for primary.xml because it doesn't contain cross-references between packages using non-natural keys. It doesn't work for the SQLite database, either in binary or SQL dump format, because of the reliance on artificial primary keys (such as package IDs). I've once tried this. With about 10k packages in fedora-updates, the delta over 2-3 days was +491 -479. Assuming deletions are cheap, the delta should ideally be 5%. As expected, binary bsddiff yields much bigger (~29%) delta. A line-wise diff is much smaller because dependencies and package descriptions mostly stay the same. (This assumes consistent sorting of the primary.xml file.) The diffs are not the problem. The problem is hte excessively frequent downloads of the repodata, which are compressed binaries with checksums, not published as deltas or diffs. The result is a grossly inefficient and far-too-frequent download of upstream repository information. It's not the local SQLite database operations in the yum cache that are killing me, at least, it's the short metadata_expire in /etc/yum.conf. Very few of us really need our metadata expired between our first cup of coffee in the morning, and luncthtime. And very few of us need yum-updatesd and other auto-magic update tools grinding our host and our proxies bandwidth for several 20 Megabyte files every few hours, nor grinding our local disk with the uncompressed *80 Megabyte* file of primary.xml sitting in /var/cache/yum/. This is an inherent problem with the let's store more, and more, and more data in the database. The databases for yum have gotten bulky and cumbersome, and the automatic churn involved in updating them with fresh repodata has become quite large. Can you point me to the primary.xml - SQLite translation in yum? I've got a fairly efficient primary.xml parser. It might be interesting to see if it's possible to reduce the latency introduced by the SQLite conversion to close to zero. (Decompression and INSERTs can be interleaved with downloading, and maybe the index creation improvements in SQLite are sufficient these days.) Good luck with that! It's not what I, personally, was just looking for, but improving that would be nice. But the SQLite files are getting larger, and larger, and larger. At 80 Meg and counting for the Fedora primary.sqlie, it's getting out of hand. However, for many users that follow unstable or testing, package diffs are currently slower than downloading the full Packages file because the diffs are incremental (i.e., they contain the changes from file version N to N+1, and you have to apply all of them to get to the current version) and apt-get can easily write 100 MB or more because the Packages file is rewritten locally multiple times. Yes, patch chaining should be avoided. I'd like to use N = 1 deltas, that could be applied to many recent snapshots. And it has little to do with the yum issue I've raised, which is entirely about the metadata. My personal experience is that the use of deltarpms is a bandwidth and resource issue completely overshadowed by the constant grinding of disk and bandwidth for hte metadata. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Can you point me to the primary.xml - SQLite translation in yum? I've got a fairly efficient primary.xml parser. Just set mddownloadpolicy=xml in yum.conf. It should work, but since downloading sqlite.bz2 is much better, very few use this. Yum uses fairly efficient parser, written in C, using libxml2 (the yum-metadata-parser package). It's always bundled, because Yum has to support xml-only repositories anyway. Oh, there's a typo in yum.conf.5 .. fixed. It might be interesting to see if it's possible to reduce the latency introduced by the SQLite conversion to close to zero. (Decompression and INSERTs can be interleaved with downloading, and maybe the index creation improvements in SQLite are sufficient these days.) We have to checksum the downloaded data before processing, and this kills pipelining. Also, when updating primary_db with a bunch of INSERTs and DELETEs, your database differs from the one on server: - different *.sqlite checksum - different pkgId = PkgKey mapping - different order of packages from SELECTs For speed, Yum joins primary_db and filelists_db via pkgKey, so #2 breaks Yum, unless you always download/delta-update both- so this kills the win in we don't need filelists case. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Hey, On Wed, May 22, 2013 at 05:08:33PM -0400, Rahul Sundaram wrote: On Wed, May 22, 2013 at 9:43 AM, Jan Zelený wrote: We acknowledge the need for some changes in Software Management stack in Fedora but we don't want to make changes just by guessing what our users want. Therefore I call to you, consumers of our products (dnf, yum and rpm): what are the changes that you would like to see in the foreseeable future (say 2-3 years) and why would you like to see them (what would they help you with)? Thanks for taking on this effort. What I would like to see is solid git integration. Git has become the standard distributed vcs and github and google code etc have stopped hosting tarballs and/or discouraging it and GNOME is planning to do that as well. If Source URL could point directly to a git url with a hash or git tag, we would benefit. I'd add to that an optional GPG key to check the git tag against. Christophe pgpsvySRo_dBL.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Well, another thought is that if some software has a lot of optional dependencies, how to handle them from the view of end-users? Should we list these optionals or add a option for install all with deps? Thanks. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Sat, May 25, 2013 at 09:34:32AM -0400, Nico Kadel-Garcia wrote: I think this can be addressed by moving the metadata updates to a different function, and calling it *separately* only as needed. The Debian apt tool does this quite effectively. You can kind of simulate that by forcing yum to run from cache only. Use the -C flag for that. Lars -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Hi, On Fri, May 24, 2013 at 10:08:06AM -0700, Adam Williamson wrote: A 'metapackage' is an actual package shipped in the repositories which contains no files, and whose raison d'etre is to express some dependencies. There are a few of these in Fedora, xorg-x11-drivers being the classic example, but they are generally strongly discouraged. The idea is that Fedora uses comps groups to express the concept 'this group of packages forms some kind of cohesive set and can be installed together', not metapackages. Some things in favor of metapackages: 1. The packager has to know that such thing as comps exists. (I know about it. I have even seen patches for it, so I know it is some sort of XML. But I have no idea where to look for it. Is there a repo for it? fedpkg clone -B comps was unsuccessful.) On the other side, the packager already knows how to make a (sub-)package that depends on other package(s). 2. A metapackage uses rpm syntax for specifying the dependencies. No need to learn anything new. 3. A metapackage is under the packager's control. A comps group is not. (I doubt packagers have commit access to to comps repo (if there is a repo). That means creating a patch, opening a bug, waiting for reaction (probably for days),... tadda yadda... Or I can create a subpackage and be done in a minute.) D. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Sat, 2013-05-25 at 08:36 +0200, David Tardon wrote: Hi, On Fri, May 24, 2013 at 10:08:06AM -0700, Adam Williamson wrote: A 'metapackage' is an actual package shipped in the repositories which contains no files, and whose raison d'etre is to express some dependencies. There are a few of these in Fedora, xorg-x11-drivers being the classic example, but they are generally strongly discouraged. The idea is that Fedora uses comps groups to express the concept 'this group of packages forms some kind of cohesive set and can be installed together', not metapackages. Some things in favor of metapackages: 1. The packager has to know that such thing as comps exists. (I know about it. I have even seen patches for it, so I know it is some sort of XML. But I have no idea where to look for it. Is there a repo for it? fedpkg clone -B comps was unsuccessful.) On the other side, the First google result for 'fedora comps' is: https://fedoraproject.org/wiki/How_to_use_and_edit_comps.xml_for_package_groups It has a 'how to edit comps' section which gives the repo URL. 3. A metapackage is under the packager's control. A comps group is not. (I doubt packagers have commit access to to comps repo (if there is a repo). Guess again. All packagers can commit to comps, I believe. But please, please, try to avoid committing to it during release freezes without checking in with notting/qa/releng, especially if your change adds packages to release media. Or is completely broken, as has happened. ;) That means creating a patch, opening a bug, waiting for reaction (probably for days),... tadda yadda... Or I can create a subpackage and be done in a minute.) Or you could not make assumptions :) -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Le vendredi 24 mai 2013 à 07:08 +0200, Ralf Corsepius a écrit : On 05/23/2013 06:05 PM, Simone Caronni wrote: On 23 May 2013 17:38, James Antill ja...@fedoraproject.org mailto:ja...@fedoraproject.org wrote: On Thu, 2013-05-23 at 13:52 +0200, Simone Caronni wrote: mv /etc/nagios/nagios.cfg /etc/nagios/nagios.cfg.mine yum reinstall nagios --onlyfile /etc/nagios/nagios.cfg How is this functionally different from yum reinstall nagios ? Yes, yum/rpm will currently replace files with identical copies but functionally the extra copies are just wasting reinstall time. By doing yum reinstall nagios I don't have the the default config files (i.e. nagios.cfg.rpmnew is not created). This happens only on upgrades. yum remove pkg yum install pkg should do what you want (backups of modified config files). But that's unfortunately not always convenient if you need to remove dependent packages for yum remove ? I guess yum-shell may help in that case ? -- Michael Scherer -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Thu, May 23, 2013 at 9:52 AM, john.flor...@dart.biz wrote: From: Rahul Sundaram methe...@gmail.com What I would like to see is solid git integration. Git has become the standard distributed vcs and github and google code etc have stopped hosting tarballs and/or discouraging it and GNOME is planning to do that as well. If Source URL could point directly to a git url with a hash or git tag, we would benefit. Amen to that! I roll my own rpms daily from locally developed sources where we have no policy of pushing tarballs. From everything I've ever been able to figure out, it's necessary for me to make temporary tarballs just to feed rpmbuild. It always seems such a huge waste of time, especially for very large packages. Nahh, you can work around this. Manipulate %setup to build a local compilation directory, but not unload tarballs, and then in the %build step do a git pull froom your git repo. And add BuildRequires: /usr/bin/git. But this means that your SRPM's will not have the available build materials, and will require an outside web access to download source code. This is *tremendous* source control problem. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Thu, May 23, 2013 at 4:53 AM, Stijn Hoop st...@sandcat.nl wrote: Hi, I would like better integration with domain-specific package managers. By which I mean npm (for node.js), gem (for ruby), pip (for python), cpan (for perl), pecl/pear (for PHP), CRAN (for R), CTAN (for TeX), and many more I'm sure. You left out maven. And first off, those are not yum behaviors, they're rpm or rpmbuild behaviors. . Second, the second you start letting yum and RPM grab modules automatically from any of those third party repositories, you are in dependency and compatibility hell. To take an example, look at the perl-HTML-Mason package. It's been evolving reasonably quickly and bopping back and forth between distinct authors with distinct styles, so the dependency lists keep changing and updating to very recent versions of other modules. As you bring in more Perl dependencies, automatically, via local cpan builds, you lose track of which RPM you built and installed in what fashion and can wind up with dependencies based on the latest builds from CPAN that *break* your existing codebase, or even wind up upgrading your version of perl. (Been there with cpan build, done that, had one hell of a time cleaning up the mess!) I've done a lot of work with CPAN - RPM building, and it's painful (That's why I keep updated versions of cpan2rpm and cpanspec up at https://github.com/nkadel/) By integrating RPM with these package managers, I feel it would be possible to provide a consistent view of the system, as well as a consistent management interface for sysadmins as opposed to application developers. The latter I might expect to continue to use the domain specific package managers, simply because they add value to domain experts -- but for the common usecase install this app on the server it would be nice to use RPM only. It's possible to have some tools to help with such building, but that's a package creation issue, not a package management issue. The same issues exist Another advantage that I see is that it saves Fedora packager manpower -- if the translation is good enough, it should be possible to work with upstream packages and simply automate the fedora rpm process as much as possible. Current examples are R2spec and the TeXLive package scripts. TexLive is well defined. CPAN, rubygems, Maven, php/pear, etc.? Not so much. Rubygems is particularly scary because the gems *do not publish compilable source* for any well defined build environment, they rely extensively on pre-built modules. Maven has the same I publish binaries, not buildable source code problem. CPAN is a nightmare: the stylistic differences among the developers make consistent RPm building a very, very manual process: TexLive is so well organized as a monolithic, buildable source environment that it works well, I'll admit, but but not the others. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 24 May 2013 22:02, Przemek Klosowski przemek.klosow...@nist.gov wrote: On 05/23/2013 07:52 AM, Simone Caronni wrote: I fiddle around with a new Nagios installation, then something stops working. I'm pretty sure it is some modifications in /etc/nagios/nagios.cfg but I cannot track it down. As an example I could do: mv /etc/nagios/nagios.cfg /etc/nagios/nagios.cfg.mine yum reinstall nagios --onlyfile /etc/nagios/nagios.cfg What's the advantage over just yum reinstall nagios, given that modified config files are preserved? A known baseline is a huge advantage of RPM, and allowing retail installation seems to go against that. Well, that's true; I was thinking of a more obscure case that happened to me with some config files and manipulated files under /usr/share but that's really a corner case. If I just save the file I get the same behaviour. Forget it. Regards, --Simone -- You cannot discover new oceans unless you have the courage to lose sight of the shore (R. W. Emerson). http://xkcd.com/229/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/24/2013 09:26 AM, Michael Scherer wrote: Le vendredi 24 mai 2013 à 07:08 +0200, Ralf Corsepius a écrit : On 05/23/2013 06:05 PM, Simone Caronni wrote: On 23 May 2013 17:38, James Antill ja...@fedoraproject.org mailto:ja...@fedoraproject.org wrote: On Thu, 2013-05-23 at 13:52 +0200, Simone Caronni wrote: mv /etc/nagios/nagios.cfg /etc/nagios/nagios.cfg.mine yum reinstall nagios --onlyfile /etc/nagios/nagios.cfg How is this functionally different from yum reinstall nagios ? Yes, yum/rpm will currently replace files with identical copies but functionally the extra copies are just wasting reinstall time. By doing yum reinstall nagios I don't have the the default config files (i.e. nagios.cfg.rpmnew is not created). This happens only on upgrades. yum remove pkg yum install pkg should do what you want (backups of modified config files). But that's unfortunately not always convenient if you need to remove dependent packages for yum remove ? I guess yum-shell may help in that case ? c.f. what I wrote in http://lists.fedoraproject.org/pipermail/devel/2013-May/183321.html This works in most cases (I haven't seen it fails in any real world case, it's imaginable it fails in some stateful packages). Ralf -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Sat, 25 May 2013 01:42:44 +0300 Oron Peled o...@actcom.co.il wrote: I think you missed the whole point of Debian's multi-arch -- instead of special handling for sister architectures (e.g: x86/x86_64), or proving there aren't (e.g: aarch64/armv7) -- it creates a symmetric world. The *huge* benefit of multi-arch is to people that *cross-compile*. ...snip... * This means cross-toolchains becomes first class citizens. I understand that some people cross compile, and that's nice, but I'm not convinced it's important enough a use case to rearrange lots of things with all the pain involved. Since I've not actually used debian in years, I'll bow out of this discussion now. ;) kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 2013-05-23, Orion Poplawski or...@cora.nwra.com wrote: On 05/23/2013 09:34 AM, James Antill wrote: On Wed, 2013-05-22 at 16:55 -0600, Orion Poplawski wrote: I'll get more specific then: python-pyface can use two different graphics backends - either wxPython or pyQt4. In no way do these two packages provide the same thing in any meaningful way other than to pyface. So, while one could go the provides route, it doesn't seem to make sense to me. I suppose I could ask for: Provides: pyface-backend in both PyQt4 and wxPython. Think that would make sense? Add a layer, and it works perfectly: python-pyface: Requires: python-pyface-backend python-pyface-backend-wxPython: Provides: python-pyface-backend Requires: wxPython python-pyface-backend-pyQt4: Provides: python-pyface-backend Requires: pyQt4 Took me a little bit to realize that no actual code needed to be split out to do this (Sandro's suggestion too much in my mind). But yes, this does the trick nicely, at the expense of some extra dummy packages. Do you know virtual packages are forbidden in Fedora? -- Petr -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 2013-05-22, Orion Poplawski or...@cora.nwra.com wrote: On 05/22/2013 04:44 PM, Adam Williamson wrote: On Wed, 2013-05-22 at 23:30 +0100, Richard W.M. Jones wrote: Still, I don't see how that's really worth doing instead of just using a virtual provide. The OP's reasoning for not using a virtual provide sounded rather specious to me. The two things are functionally equivalent so far as I can see. Just different syntax. I'll get more specific then: python-pyface can use two different graphics backends - either wxPython or pyQt4. In no way do these two packages provide the same thing in any meaningful way other than to pyface. So, while one could go the provides route, it doesn't seem to make sense to me. I suppose I could ask for: Provides: pyface-backend in both PyQt4 and wxPython. Think that would make sense? In my option this is wrong becuse: (1) You have to touch foreign package for that. You have to persuade maintainer of that packages to put to provide there and bump release just for that purpose. (2) The feaure to use PyQt4 as a back-end is ability of pyface, not ability of PyQt4. So you should declare the feature in pyface, not in the PyQt4. (3) What will happen when pyface looses ability to use PyQt4 as a back-end? Will there keep stray Provides in the PyQt4 forever? In short, interchangable requirement list (the OR in Requires) is more natural and efficent way how to express it than putting yet another layer of virtual provides. -- Petr -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Dne Čt 23. května 2013 11:54:04, john.flor...@dart.biz napsal(a): From: pmati...@laiskiainen.org On 05/23/2013 06:09 PM, john.flor...@dart.biz wrote: And even though I have to give rpmbuild a tarball, I don't believe it ever reuses it as is. My understanding is that the content gets extracted, processed and tarballed again. I dont know what gave you such an idea Me neither. Perhaps the apparent slowness with large packages and/or a lack of coffee this morning. , rpm certainly does nothing of the sort. The tarball is obviously extracted for building, but what ends up in the src.rpm is the original tarball and the patches defined in the spec - this is the pristine sources principle: http://rpm.org/max-rpm-snapshot/ch-rpm-philosophy.html#S1-RPM- PHILOSOPHY-PRISTINE-SOURCES You are, of course, right. I actually knew that for srpms. I just tend not to think about the srpms much since for nearly all the builds I do, *I am the upstream* source so I'm really only interested in the binary rpms. I'd like to see it behave more the way I expected it to when I naively first started rolling my own packages. Specifically, it would be nice if the %Source URI was processed intelligently to automatically retrieve the content via HTTP, FTP, GIT, FILE or whatever (within reason) happens to be specified there. Rpm = 4.10 can automatically download remote sources and patches over http and ftp, but since there's (currently) no way to verify downloaded content the feature is disabled by default as its quite a security risk to download arbitrary content from the internet without checking checksums at least. This sounds more like something that should be a part of rpmdev-tools (if it isn't already. I recommend getting in touch with the maintainer of those and see if there is something that can be done there. Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Fri, 2013-05-24 at 08:03 +, Petr Pisar wrote: Do you know virtual packages are forbidden in Fedora? Sorry, I just scanned through the guidelines and didn't see this anywhere. Do you mind citing a reference, please? Thanks, Jonathan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Dne Čt 23. května 2013 18:05:40, Till Maas napsal(a): On Thu, May 23, 2013 at 03:13:30PM +0200, Jan Zelený wrote: On 23. 5. 2013 at 13:52:39, Simone Caronni wrote: I fiddle around with a new Nagios installation, then something stops working. I'm pretty sure it is some modifications in /etc/nagios/nagios.cfg but I cannot track it down. As an example I could do: mv /etc/nagios/nagios.cfg /etc/nagios/nagios.cfg.mine yum reinstall nagios --onlyfile /etc/nagios/nagios.cfg Thanks for the explanation. I will mark the email and make sure to put the use case to the updated list of RFEs. I would like it even more if RPM keeps a backup copy of every %config file somewhere protected to allow to restore config files even without having the original RPM available, because the RPM might not be available anymore. Already on the list (see the very first RFE). Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
Dne Čt 23. května 2013 16:41:04, Vít Ondruch napsal(a): Dne 23.5.2013 16:29, Miloslav Trmač napsal(a): On Thu, May 23, 2013 at 4:23 PM, Vít Ondruch vondr...@redhat.com mailto:vondr...@redhat.com wrote: *It is not possible to convert the packages technically nor philosophically* You might think million times that the sentence is not truth, but that is as it is. I'll give you several examples: * Gems cannot express dependencies on system libraries such sqlite3, libxml2, etc. Doesn't matter for the system administrator who has already installed the gem. * Gems does not undergoing legal review, i.e. you have to trust to the author of the gem, that the license is correct, which sadly may not true. Doesn't matter for the system administrator who has already installed the gem. * Bundling, quite common phenomenon. Doesn't matter for the system administrator who has already installed the gem. Just this short list of issues should be enough. It might work for you, when you decide to neglect all the issue mentioned above and probably several else, but it does not work for distribution. Sorry. I don't think this is necessarily targeted at changing how _Fedora_ distributes things; just giving system administrators a single command that works on an installed system might be an useful improvement. Mirek Ok, so speaking for gem2rpm, it might be made more dumber to include automatically * and if no license is found, then put there foo for example. Actually, everybody is free to do such a template for himself, or even submit such patch upstream, but then the RPM kind of loosing its purpose. Agreed, at least to a certain degree. If we are talking just about a single point of access for users, perhaps we should rather consider putting this in one of the layers above, sort of gnome software center approach. Thanks Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/24/2013 10:21 AM, Reindl Harald wrote: Am 24.05.2013 07:08, schrieb Ralf Corsepius: By doing yum reinstall nagios I don't have the the default config files (i.e. nagios.cfg.rpmnew is not created). This happens only on upgrades. yum remove pkg yum install pkg should do what you want (backups of modified config files) and how do you imagine work around a dependency hell unintsalling half of your system depending on what package you uninstall? If you for some reasons want to avoid this (in most cases there is no reason to avoid this), you can replace the yum remove with rpm --nodeps -e pkg i.e. use rpm --nodeps -e pkg yum install pkg It will generate config backup files and install fresh ones. there is a reason reinstall exists reinstall only reinstalls but doesn't reinstall from scratch and your idea is not smart nor a solution Well, ... if you think so, ... I disagree, it works quite well. Ralf -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 2013-05-24, Jonathan Dieter jdie...@lesbg.com wrote: On Fri, 2013-05-24 at 08:03 +, Petr Pisar wrote: Do you know virtual packages are forbidden in Fedora? Sorry, I just scanned through the guidelines and didn't see this anywhere. Do you mind citing a reference, please? Well it's probably not forbidden by guidelines, but I've heard this rule many times. There exists a thread on fedora-packaging mailing list http://lists.fedoraproject.org/pipermail/packaging/2009-January/005481.html. I also think rpmlint considers empty package as an error and I remember I saw an rpmbuild run that did not produce an sub-package just because there was no %files section. Despite all of that I consider this solution with meta sub-packages as the best option until RPM will support disjunction. -- Petr -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/24/2013 01:37 PM, Petr Pisar wrote: On 2013-05-24, Jonathan Dieter jdie...@lesbg.com wrote: On Fri, 2013-05-24 at 08:03 +, Petr Pisar wrote: Do you know virtual packages are forbidden in Fedora? Sorry, I just scanned through the guidelines and didn't see this anywhere. Do you mind citing a reference, please? Well it's probably not forbidden by guidelines, but I've heard this rule many times. There exists a thread on fedora-packaging mailing list http://lists.fedoraproject.org/pipermail/packaging/2009-January/005481.html. What's generally frowned upon is the task-foo style gigantic metapackages, but metapackages are used in Fedora for more specific purposes. Take xorg-x11-drivers or wine for example. I also think rpmlint considers empty package as an error and I remember I saw an rpmbuild run that did not produce an sub-package just because there was no %files section. Missing %files section is different from an empty %files section: a non-existing %files section indeed causes no package to be generated (which various people are taking advantage of for simplifying conditional builds), but empty %files section is perfectly legal, always been. - Panu - -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Fri, 2013-05-24 at 10:37 +, Petr Pisar wrote:
On 2013-05-24, Jonathan Dieter jdie...@lesbg.com wrote:
On Fri, 2013-05-24 at 08:03 +, Petr Pisar wrote:
Do you know virtual packages are forbidden in Fedora?
Sorry, I just scanned through the guidelines and didn't see this
anywhere. Do you mind citing a reference, please?
Well it's probably not forbidden by guidelines, but I've heard this rule
many times. There exists a thread on fedora-packaging mailing list
http://lists.fedoraproject.org/pipermail/packaging/2009-January/005481.html.
Well, I think there's a bit of conflation going on here. 'Virtual
packages', 'metapackages', and 'virtual provides' are somewhat different
things, and what has historically been discouraged in Fedora is one
specific form. Let's call this form specifically a 'metapackage', to be
clear.
A 'metapackage' is an actual package shipped in the repositories which
contains no files, and whose raison d'etre is to express some
dependencies. There are a few of these in Fedora, xorg-x11-drivers being
the classic example, but they are generally strongly discouraged. The
idea is that Fedora uses comps groups to express the concept 'this group
of packages forms some kind of cohesive set and can be installed
together', not metapackages.
What was being discussed in this sub-thread was not a metapackage at
all, only the use of virtual Provides:. A virtual Provide is kind of a
slithery concept when you sit down and start thinking about it, but I
guess I'd say it's when a package Provides: something that is a kind of
abstract concept (in order that multiple packages can provide the same
thing, and other packages can express 'I need some capability that all
of these packages provide'). Really, almost any Provides: statement that
actually has to be written into the spec file by the packager is a
'virtual provide'.
There's certainly nothing in the guidelines that restricts the use of
virtual provides, and it even specifically requires them in one
scenario:
Static libraries only. When a package only provides static libraries
you can place all the static library files in the *-devel subpackage.
When doing this you also must have a virtual Provide for the *-static
package:
%package devel
Provides: foo-static = %{version}-%{release}
I don't think there's any problem with the use of 'virtual provides' in
Fedora policy or tradition, it's 'metapackages' that are usually
discouraged.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Thu, 2013-05-23 at 15:36 -0300, Paulo César Pereira de Andrade wrote: 2013/5/22 Björn Persson bj...@xn--rombobjrn-67a.se: Jan Zelený wrote: what are the changes that you would like to see in the foreseeable future (say 2-3 years) and why would you like to see them (what would they help you with)? Dare I say ... (puts on a helmet) ... Recommends and Suggests? We really should have a way for Yum and Packagekit to say Hey, if you're installing that, maybe you also want this?. A -devel package and its corresponding -doc package should recommend each other for example. They shouldn't require each other, because then they could just as well be the same package, but usually you want to install them together and it would be helpful to at least notify users who install the -devel package that the -doc package also exists. Another example is a database server and its command line client, which you often but not always want to install together. The server should recommend the client, while the client might only suggest the server. A third example is graphical administration tools for some daemon that are in a separate package so that the daemon can be installed without pulling in half a desktop environment. In this case the daemon should suggest the tools, but perhaps not recommend them. The only few times I used Suggests (in Mandriva) was to suggest optional, but better experience/functionality if installed, packages in non-free, but I think I never fully understood what it is supposed to mean, just that it is a Requires that does not break dependencies. Usually softer dependencies don't have a single specific 'meaning' exactly. Even Requires doesn't _quite_, because you can override Requires with rpm --nodeps . It's more useful (I've found, anyway) to think of this as metadata that it's the package manager's responsibility to interpret, and that you take all the possible ways the package manager can interpret that metadata into account when you create it. Usually, package managers are configurable to do various things with soft dependencies. IIRC, Mandriva was configured to install the soft dependencies of packages by default. However, you could then remove them without the 'requiring' package being removed. So if foo Suggests: bar, 'urpmi foo' would install bar, but 'urpme bar' would not require the removal of foo. However, you could configure urpmi not to install Suggested packages, if you wanted to keep a very minimal system. So the semantics of Mandriva's system are fairly simple, and the executive takeaway for a packager is pretty simple too: use Suggests: for things that a package doesn't strictly require to work, but which might make the experience of using it better for many people. With a more complex three-level system like Debian's there are likely to be more configuration options for the package manager's behaviour with regards to each form of dependency, and hence more possible semantics, but again once you understand the various 'supported' options for how to handle the different types of dependency at the package manager level, you should be able to understand which form of dependency to express in your package. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Thu, 2013-05-23 at 18:29 +0300, Panu Matilainen wrote: Rpm = 4.10 can automatically download remote sources and patches over http and ftp, but since there's (currently) no way to verify downloaded content the feature is disabled by default as its quite a security risk to download arbitrary content from the internet without checking checksums at least. And note that it's as much Fedora policy as packaging stack capabilities that prevents this happening at present: as discussed in another thread it's a fundamental part of the Fedora packaging system's design that the builders have no outside access, and it's the package maintainer's explicit responsibility to provide the source files to the build system. (The implication of this is that it is the package maintainer's responsibility to provide, and verify that they are providing, the _correct_ sources.) We could of course build the smarts into the fedpkg layer - have some fedpkg commands for checking out and building tarballs of SCM-hosted content - but then you've just moved the security risk Panu mentioned to that layer; if we do that it kind of sends a bad implication that it's fine to just trust whatever you get from the SCM URL. Thinking about this, it's one reason the style of doing 'git snapshot' builds where you have Source0 be the last stable tarball and then include the full patch series to master as generated by 'git format-patch' as Patches could be considered superior to simply including a git master snapshot tarball: at least if someone's concerned about some kind of breach, they have an easily-verifiable base to work off - as there should be an official checksum for the last release tarball - and only have to check the patches for problems, rather than checking the entire tree. I think, to be honest, a lot of us as packagers slip some way short of the 'ideals' here in day-to-day life, but that's probably no excuse for making it _easier_ to avoid our responsibilities :) -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/23/2013 09:10 AM, Jan Zelený wrote: TBH I'm not perfectly sure what causes the problem but since the new multilib in Ubuntu started I am dealing with a whole bunch of issues like the one I described. So to sum up, I personally think we are doing much better with the way how things work in Fedora. Not really. We are focusing only on the x86_64/x86 case and ignoring the broader problem which Debian has tackled. Jumping to the conclusion that because you had some multi-lib issues in Ubuntu, we are doing better is premature. Considering the fact that ARM is going to become primary architecture for Fedora, we really need to solve the broader problem one way or the other. Rahul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/23/2013 07:08 AM, Jan Zelený wrote: Have you tried using dnf instead of yum? It is much faster. To be perfectly honest we don't plan to invest much effort in developing new things for yum, it will more and more shift towards maintenance mode and the focus will be on dnf. What does the we stand for here? If this is Red Hat on the whole, a broader announcement of the plan would be helpful. I will note that while dnf is faster overall, it doesn't have many of the important features of yum and I still ran into bugs in some trivial tests from time to time. https://fedoranext.wordpress.com/2013/05/18/status-of-dnf-experimental-fork-of-yum/ Rahul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Fri, 24 May 2013 15:15:30 -0400 Rahul Sundaram methe...@gmail.com wrote: On 05/23/2013 09:10 AM, Jan Zelený wrote: TBH I'm not perfectly sure what causes the problem but since the new multilib in Ubuntu started I am dealing with a whole bunch of issues like the one I described. So to sum up, I personally think we are doing much better with the way how things work in Fedora. Not really. We are focusing only on the x86_64/x86 case and ignoring the broader problem which Debian has tackled. Jumping to the conclusion that because you had some multi-lib issues in Ubuntu, we are doing better is premature. Considering the fact that ARM is going to become primary architecture for Fedora, we really need to solve the broader problem one way or the other. It was my understanding that arm is not going to do any multilib. aarch64 cannot run other stuff, so you cannot run armv7 or whatever on a aarch64 box, it's just completely different. (I sure hope this is the case) kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On 05/23/2013 07:52 AM, Simone Caronni wrote: I fiddle around with a new Nagios installation, then something stops working. I'm pretty sure it is some modifications in /etc/nagios/nagios.cfg but I cannot track it down. As an example I could do: mv /etc/nagios/nagios.cfg /etc/nagios/nagios.cfg.mine yum reinstall nagios --onlyfile /etc/nagios/nagios.cfg What's the advantage over just yum reinstall nagios, given that modified config files are preserved? A known baseline is a huge advantage of RPM, and allowing retail installation seems to go against that. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Fri, May 24, 2013 at 01:41:41PM -0600, Kevin Fenzi wrote: It was my understanding that arm is not going to do any multilib. aarch64 cannot run other stuff, so you cannot run armv7 or whatever on a aarch64 box, it's just completely different. (I sure hope this is the case) Looks like you're right: https://fedoraproject.org/wiki/Architectures/ARM/AArch64 (I should note that the new 64 bit ARM processors *can* run 32 bit code, it's just that Fedora won't be supporting that mode) Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Fri, 2013-05-24 at 10:11 -0700, Adam Williamson wrote: We could of course build the smarts into the fedpkg layer - have some fedpkg commands for checking out and building tarballs of SCM-hosted content - but then you've just moved the security risk Panu mentioned to that layer; if we do that it kind of sends a bad implication that it's fine to just trust whatever you get from the SCM URL. I'm not going to debate this extensively, because unless someone who can actually change things is planning to do so, it's just pointless noise. But basically there are two threats: 1) MITM attacks by third parties. Answer: SSL. Yes, it's not perfect, but it's good enough for online banking. Yes, governments and affiliated groups have wildcard certificates, but there are defenses. Manual human signoff on new root CAs would be pretty good. 2) Corrupted repository server side: The answer to this is to have a system that actually *encourages* people to look at the source code. If you truly wanted to be serious about that, we could have a UI that actually you know, unpacks the source by default, diffs it from the previous, and requires human signoff before building it. But at the moment, all the crappy package metadata sadly is what's front and center, not the actual source code. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Software Management call for RFEs
On Friday 24 May 2013 13:41:41 Kevin Fenzi wrote: On Fri, 24 May 2013 15:15:30 -0400 Rahul Sundaram methe...@gmail.com wrote: Not really. We are focusing only on the x86_64/x86 case and ignoring the broader problem which Debian has tackled. Jumping to the conclusion that because you had some multi-lib issues in Ubuntu, we are doing better is premature. Considering the fact that ARM is going to become primary architecture for Fedora, we really need to solve the broader problem one way or the other. It was my understanding that arm is not going to do any multilib. aarch64 cannot run other stuff, so you cannot run armv7 or whatever on a aarch64 box, it's just completely different. I think you missed the whole point of Debian's multi-arch -- instead of special handling for sister architectures (e.g: x86/x86_64), or proving there aren't (e.g: aarch64/armv7) -- it creates a symmetric world. The *huge* benefit of multi-arch is to people that *cross-compile*. Example (without multi-arch -- like in Fedora today): * You cross compile libfoo (e.g: on x86 you compile for arm) * You can install the resulting libfoo rpm on the arm device * Now you want to cross-compile an application that use this library, so you need to install libfoo-devel on your workstation -- BUT! * The libfoo-devel you have is for arm (not installable on your x86) * Furthermore, even if you manage to install it, it will put the files in the wrong location (/usr/lib) and not where the cross-compiler will search them (e.g: /usr/arm-unknown-linux/.../lib) * So you have to extract the arm libraries from libfoo-devel and manually put them into the correct location. (btw: Debian embedded developers has automatic tool to convert these packages -- dpkg-cross). * If you maintain an embedded arm based product, multiply this scenario by the number of libraries your developers maintain and their daily commits does it scale? How does it affect your whole build environment, continuous integration tools, etc. Same example (but with multi-arch): * Every compiler/linker/dynamic-linker for every architecture search first in /usr/lib/arch and falls back to /usr/lib (to cover legacy libraries which haven't been migrated yet). * Under this scheme, *both* the native and cross toolchains search for libraries in the same location -- /usr/lib/arch * With careful package design, there should be no problem installing the arm based libfoo-devel on our x86 system (no conflicting files). * Please note, that it doesn't matter if libfoo-devel was built on an arm machine (native compilation) or on x86 with a cross-compiler -- in both cases the resulting library files would be on /usr/lib/arm-unknown-linux * This means cross-toolchains becomes first class citizens. Bye, -- Oron Peled Voice: +972-4-8228492 o...@actcom.co.il http://users.actcom.co.il/~oron Gratis is nice, Libre is an inalienable right. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
