Re: [modularity] Modules and AppStream
Matthew Miller wrote: > Sure, that's fair. Here's the goal: users will have more options > without exponentially increasing work for packagers and > distro-creators. The exponential number of combinations will still exist, it will just be out of our, or really anybody's, control. It is impossible to test the exponential number of module version combinations. And as evidenced in the next point, due to module version conflicts, users can actually end up with FEWER options than before. And the tighter you set the module version requirements in inter-module dependencies to work around the first issue, the worse the second issue becomes. >> That destroys the possibility to install all Fedora packages on all >> Fedora spins, which the shared Everything repository guaranteed. With >> Modularity fully implemented, you can end up being unable to install KDE >> Plasma on the GNOME Workstation or the other way round (at least without >> resorting to containers) because the desktops depend on conflicting >> versions of some library module. How is that an improvement? > > That scenario clearly isn't an improvement, but it's also an edge case. I do not see installing multiple desktops as an edge case. Sure, it might not be the common case, but it is still very frequent. It is often desired as soon as the computer has more than one user, and even a single user may want to try out another desktop without uninstalling his/her primary desktop environment. > Anecdotally, I see "I want KDE without all this GNOME stuff" more than > I see "I want all these desktops installed at once". Yet, Fedora does not actually provide that, only Kannolo does. :-) > But, in any case, yes, that's a tradeoff. The upside is that we'll be > able to scale up in ways we can't otherwise. Such as? > And it means that desktops _do_ have the flexibility to have conflicting > libraries where that is beneficial. But that is a major disservice to our users. There is a reason that such a solution was ruled out for the Fedora 15 NetworkManager conflict. (Unfortunately, the solution that was implemented also sucked. The late NM 0.9 change should have been rejected instead.) Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
De: "Kevin Kofler" Matthew Miller wrote: >> And, it will also allow those of us working on assembling spins and more >> options — for example, we can have different streams for Atomic without >> needing to actually duplicate every package. (And we could do the same for >> KDE or whatever other artifacts would benefit from that, if we want.) >That destroys the possibility to install all Fedora packages on all Fedora >spins, which the shared Everything repository guaranteed. Indeed :( The initiative posits it is technically possible to define clear-cut autonomous modules, it is human-effective to do so and that the result will satisfy users. It underestimates the huge end-user value of an integrated unified repository, that drastically reduces the human cost of using Fedora. The technical possibility seems dubious to me given that an awful lot of the software ecosystem Fedora ships is developed in a bazaar interconnected style. Case in point the various attempts to make perl optional — it's always rising back somewhere. However Fedora and Red Hat are good at technical stuff, throw enough devs at it it and some sort of technical solution to identify and manage possible module boundaries will probably emerge. The human effectiveness seems harder. Fedora was never this ambitious before. Spins are polite abstractions that are mere starting points (by design). I doubt any big deployments of "pure" Fedora spins without added packages exist. How many market studies do one need to define sensible functional module boundaries? What will be the shelf life of the result? How many people need to work on creating and maintaining those boundaries? Is the community interested on working on this or will it have to be shouldered by the generous sponsor, making Fedora less a community organisation? A general-purpose OS is much more complex than the simplistic single-purpose leaf GUI apps shipped on smartphones. I happily use rygel on a headless box to stream music to various appliances, making it a dlna server installation even though it is created for desktops. Plus the whole additional module definition step introduces friction and delay in a fast-moving software distribution. Finally, do the users ask for this modularity? Will they like the result? The huge selling point of live images some years ago was that one could install packages without being constrained by the predefined selection. Customer feedback on RHEL split repositories and multiplication of additional channels was dire enough Red Hat had to nix them. I can see the benefit from a monetization point of view, modules are a great way to construct textbook market segmentation. However at this stage the benefits of modularity for end users seem less clear to me. Certainly not clear, concrete and immediate enough to make huge and inflexible blobs more palatable than modular packages. I guess Modularity needs to progress to have some answers, but as an experiment it seems awfully risky to me. >With Modularity >fully implemented, you can end up being unable to install KDE Plasma on the >GNOME Workstation or the other way round (at least without resorting to >containers) because the desktops depend on conflicting versions of some >library module. How is that an improvement? The problem is not containerization versus parallel installation. Because, let's be honest, network, storage and hardware will absorb it if not now then in the next years. And inter-container communication can probably be solved with enough technical investment. Sure, this investment may seem huge and unnecessary, but that's for the people doing the work to judge. And I hope they won't forget the dire reputation for slowness Solaris acquired after years of priorizing convenience over performance (SUN played with zones before us!). The problem is that different versions of the same components will have slightly different behaviour and configuration rules. The amount of divergence users are willing to tolerate on a single system is way way lower that what can be technically installed using technical means such as containers or parallel install. That will drastically limit the imaginary savings of deferring porting apps to the same deps. Unified theming history show it is not so easy to hide behaviour differences even for simple stupid stuff such as colour values. Best regards, -- Nicolas mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On 2017-08-28, Matthew Miller wrote: >> > Modularity will allowing *us* at the packaging end to separate source >> > and spec lifecycle from binary and artifact lifecycle. >> That, by itself, is not a goal. It is a way to achieve an unspecified goal. > > Sure, that's fair. Here's the goal: users will have more options > without exponentially increasing work for packagers and > distro-creators. > This is not currently true. Actually the reality is opposite. If a module needs a build-time dependency, the packager must add it including all recursive dependencies into the module's modulemd file. If another packager wants to create another module with the same build-time dependency, he needs to do it again. So now you have to maintain the same thing twice. Pure theoratically that could have been resolved by creating a new module with the common dependency, but that means we would have to create a module for every package. And that's very labor intensive. And it cannot be automated because Fedora packages constitute a graph. Not a tree. A packager must come and cut build cycles or optional features to make the dependency chain sane and buildable. And such a common dependency module cannot satisfy everybody because it will be missing the required optional features that were removed. Traditional Fedora manages the enormous work of maintanance by distributing it among packagers so that everybody cares only about her packages. If we want to create modular Fedora comparable in feature sets with the traditional Fedora, we must levarage this technique. Each source package must declare its dependencies and set of optional features so that it will be possible to compute minimal dependency tree for a given package programatically. RPM cannot do it now. We only have build conditions (%bcond_without) that cannot be handled programatcially. We must change it, oterwise modular Fedora is doomed. -- Petr ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Sat, Aug 26, 2017 at 04:04:27AM +0200, Kevin Kofler wrote: > You could actually just have skipped the October-November release as you did > during the F21 cycle, leading to a single 9-month cycle, then back to 6 > months. I think a 9-month cycle would be much more realistic than a 3-month > cycle to fix a 3-month offset from the desired release dates. Yes, possibly; let's see how this goes in practice. I think in the future we may want to officially say that if a release is delayed into July or January, we skip the next one. > > On the one hand, the new https://src.fedoraproject.org/ site is > > awesome. > Is it really? The Pagure git browser is still missing as basic features as > browsing the history of individual files or directories, something I already That'd be a nice feature and is important _in general_, but since for Fedora RPMs and containers the repo changes are _mostly_ to one control file (spec or dockerfile), with other files generally replaced in their entirety if not just added or removed, I don't think it's a huge shortcoming for this use case. [some stuff snipped here where I think we just disagree and that's okay] > > and reuse of @, or simply a different syntax). In any case, this > > feedback _is_ important and _is_ listened to. > > I, too, hope that this will be addressed. > > What I also hope is that there will be a way (by uninstalling a plugin RPM > or by setting some dnf.conf option) to avoid retrieving the module metadata > entirely. (In fact, I'd hope that that'll be the default on the traditional > Spins!) At least in the short term, yeah, modularity-using release artifacts (spins, editions, containers, whatever) will use that and traditional ones will not. > > Modularity will allowing *us* at the packaging end to separate source > > and spec lifecycle from binary and artifact lifecycle. > That, by itself, is not a goal. It is a way to achieve an unspecified goal. Sure, that's fair. Here's the goal: users will have more options without exponentially increasing work for packagers and distro-creators. > That destroys the possibility to install all Fedora packages on all Fedora > spins, which the shared Everything repository guaranteed. With Modularity > fully implemented, you can end up being unable to install KDE Plasma on the > GNOME Workstation or the other way round (at least without resorting to > containers) because the desktops depend on conflicting versions of some > library module. How is that an improvement? That scenario clearly isn't an improvement, but it's also an edge case. Anecdotally, I see "I want KDE without all this GNOME stuff" more than I see "I want all these desktops installed at once". It's _nice_ to be able to do that, but I don't think _vital_, as long a there's an easy way to switch if you want to. Or, it may be that "resorting to containers" solves the problem nicely anyway. But, in any case, yes, that's a tradeoff. The upside is that we'll be able to scale up in ways we can't otherwise. And it means that desktops _do_ have the flexibility to have conflicting libraries where that is beneficial. > The arbitrary branching is what leads to users having to track a different > end of life date for, in the worst case, every single package. If Modularity > is a success, users will have installed dozens of modules. If each of them > comes with a different branch EOL, how does the user know when it's time to > upgrade to a supported branch? (Or else, if you do the upgrade > automatically, how do you ensure it does not break things? We have releases > rather than a rolling release for a reason!) This is a case of "just because we can do anything doesn't mean we should do everything". See my earlier proposal on guidelines for EOL dates (in short: let's make sure they always line up with traditional release EOLs to simplify the problem). -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Fri, Aug 25, 2017 at 07:26:20PM -0400, Ben Rosser wrote: > I have similar concerns and frustrations as Neal does, I think. > > I first want to comment that I appreciate your willingness to engage > people (like Neal, and like myself) who seem frustrated with the > future direction of Fedora. And also, I think modularity-- as you > described it above-- is a very admirable goal. I have plenty of > packages that do not really need separate versions per each Fedora > release, and it would be nice to only need to maintain one branch for > them. > > My fear is that Fedora, in the process of rolling out modularity, will > get halfway to the idealized goal and then discover that it's not > possible to go the rest of the way (for whatever reason), but also > that it's not possible to easily roll back to a non-modular world, and > we'll be stuck. Even if we don't encounter some critical design flaw, > I could certainly see us learning that it's far more complex to > maintain modules in practice than we think, or perhaps that it > ultimately makes things more complicated for users rather than less. Thanks Ben. You're right that we need to make sure we have good contingency plans every step of the way. There's some irony here because (see thread on this from a little bit ago) _successful_ modularity will give us a mechanism for backing things out that we've never had before. But we have to get there. :) -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Mon, Aug 28, 2017 at 4:34 AM, Kevin Kofler wrote: > Neal Gompa wrote: >> RPM itself is now sufficiently advanced that it can take on all the >> roles of composition group metadata. They would essentially be transformed >> into metapackages. This has already happened on the SUSE side, where >> their patterns are now just metapackages with Provides that the >> package manager knows to work from. > > Using metapackages or even the good old RPM Group tag could work, if > 1. we actually ship this information in our packages, and > 2. tools like Dnfdragora actually know how to use them. (Dnfdragora has >support for RPM Group tags, but I don't think it supports the SUSE >approach at this time, does it?) > It's in development, as Mageia also uses the same approach (the task-* packages function the same way that pattern-* packages on SUSE do). -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
Neal Gompa wrote: > RPM itself is now sufficiently advanced that it can take on all the > roles of composition group metadata. They would essentially be transformed > into metapackages. This has already happened on the SUSE side, where > their patterns are now just metapackages with Provides that the > package manager knows to work from. Using metapackages or even the good old RPM Group tag could work, if 1. we actually ship this information in our packages, and 2. tools like Dnfdragora actually know how to use them. (Dnfdragora has support for RPM Group tags, but I don't think it supports the SUSE approach at this time, does it?) Otherwise, let's please stick to what works (i.e., comps). Kevin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Sun, Aug 27, 2017 at 7:57 PM, Kevin Kofler wrote: > Igor Gnatenko wrote: >> Can we finally kill this comps stuff please? > > Please no! AppStream is no suitable replacement, due to limitations both of > the spec and of its implementation in Fedora. E.g., where are the -devel > packages in Fedora AppStream metadata? They are in comps. There are more > such limitations. In addition, AppStream does or will also include non-RPM > artefacts, which is not necessarily what the user wants. > > I really like dnfdragora, which allows browsing comps groups and RPMs, > without any of this newfangled AppStream, Flatpak, etc. stuff. > RPM itself is now sufficiently advanced that it can take on all the roles of composition group metadata. They would essentially be transformed into metapackages. This has already happened on the SUSE side, where their patterns are now just metapackages with Provides that the package manager knows to work from. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
Igor Gnatenko wrote: > Can we finally kill this comps stuff please? Please no! AppStream is no suitable replacement, due to limitations both of the spec and of its implementation in Fedora. E.g., where are the -devel packages in Fedora AppStream metadata? They are in comps. There are more such limitations. In addition, AppStream does or will also include non-RPM artefacts, which is not necessarily what the user wants. I really like dnfdragora, which allows browsing comps groups and RPMs, without any of this newfangled AppStream, Flatpak, etc. stuff. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Fri, 2017-08-25 at 16:23 -0400, Matthew Miller wrote: > On Fri, Aug 25, 2017 at 10:06:42AM -0400, Neal Gompa wrote: > [snip] > > Thanks Neal. This is much more useful and I appreciate you taking the > time despite the frusturation. > > > I'm extremely frustrated by how much half-baked-ness has been going > > on > > in the last couple of months. Schedule shrinkage, features getting > > cut, this modularity thing being implemented in a way that's > > looking > > increasingly impossible to bypass while everyone I've talked to > > seems > > to indicate that all of this is prototypical and likely to be > > completely reworked again (which has already happened at least once > > This is a lot to unpack, but I'll try as much as I can. Some of my > comments are pushback, but don't take that to mean I'm not taking the > concerns seriously. > > Fedora *is* a project for baking things, and half-bakedness is an > inherent step in baking. And I don't just mean in a "testing stuff > for > RHEL" sense — Fedora _should_ be leading in the distro space overall. Agree here, but we need to support people when they want to make Fedora leader in some area instead of silently ignoring them for 2 releases (that's just my real example). > > On the other hand, we _don't_ want to be _unbaked_ (or "bleeding > edge", > if you like). We want what gets to users and to our mainstream > contributors to be better than that. > > I hear three different main frustrations here: first, the short F27 > schedule; second, frustration with with the packagedb retirement; and > third, modularity. > > > Schedule > > > I talked about the schedule and features with you a little bit on > Twitter, but I'll say it in more than 140 characters here. Keeping to > the schedule as we've defined for years isn't really "schedule > shrinkage". It just feels new because we've never _really_ stuck to > it > before. In retrospect, I really wish I'd pushed for a shorter _F26_ > schedule when that was being defined last year, but here we are now. > I > previously brought up the idea of flat-out changing the schedule to 9 > months instead of 6 months, but literally no one supported that — > there > was a strong feeling that the six-month cadence is important. So, > here > we are figuring out how to live with that. > > Pushing back features is really only the way to do this. Cramming > everything in would be even more chaotic. But, the important thing is > that when we do it this way, we know with reasonable certainty that > the > F27 release will come out at a predictable time not so far in the > future. Features which miss this release _will_ get to users in a > reasonable timeframe. If, instead, we lengthened the schedule, adding > 6 > months from the F26 release, we'd be in mid/late December, which we > know from experience means January. With delays, maybe February > pushing > March. And then, we'd be in the exact same position for F28, debating > whether to shorten that or instead schedule for September instead of > May. That would actually be _more_ delay for _more_ things. That's why we need to work on automation of things, testing and so on. For example, even small targeted-rebuilds after bumping soname requiring manual work. Retiring of rdma stuff broke composes. There are a lot of bad things which could have been prevented if we had automation. I don't think that we need to extend schedule to 9 months, but for F27 we could have shrinked big changes. > > There's _really_ not a great answer here, but I think what we're > doing is the best of the available choices. So why we didn't push back modularity / Factory 2.0 and other stuff from F27? Since it's most of breaking parts. We could had F27 without that stuff to disturb people less due to short schedule. > > > Pkgdb retirement > === > > On the one hand, the new https://src.fedoraproject.org/ site is > awesome. It's going to open up a whole lot of potential for new > workflows and new contributors. On the other hand, yeah, the pkgdb > retirement _was_ half-baked in retrospect. But, the people working on > that are still baking. When you hit specific issues which impact your > work, *please* file tickets at > https://pagure.io/fedora-infrastructure/issues so they can get > tracked > and fixed. > > I'm not sure I really understand the part about not being able to see > what's available in Fedora. From a user perspective, > https://apps.fedoraproject.org/packages/ works very well for me (and > it's _way_ faster than it was several years ago). From a contributor > perspective, https://src.fedoraproject.org/ seems adequate if not yet > perfect. The ability to display readme files _alone_ seems like a big > step forward. It'd be nice to have better search and browsing, but > the > original pkgdb web UI wasn't great there _either_. Honestly, I was always impatient on waiting pagure-dist-git deployment! I always wanted to send
Re: [modularity] Modules and AppStream
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Sat, 2017-08-26 at 15:13 +0200, Emmanuel Seyman wrote: > * Matthew Miller [25/08/2017 16:23] : > > > > From a user perspective, > > https://apps.fedoraproject.org/packages/ works very well for me > > (and > > it's _way_ faster than it was several years ago). > > I would agree if it wasn't for packages' issue #286 > > I suspect that packages uses pkgdb to get the information it displays > so arguing that the former can replace the latter after its > retirement > seems misguided. > > > From a > > contributor > > perspective, https://src.fedoraproject.org/ seems adequate if not > > yet > > perfect. The ability to display readme files _alone_ seems like a > > big > > step forward. It'd be nice to have better search and browsing, but > > the > > original pkgdb web UI wasn't great there _either_. > > It's still has a number of critical issues (#6208 is the biggest one > that comes to me). > > I think Neal's point is that we knew this was coming. We knew that > pkgdb retirement would be half-baked before it happenned. We even > discussed it on this very list. Couldn't we have delayed the pkgdb > retirement until it had less issues? Can we learn from this change > and ensure future changes are handled better? I share most of Neal's feelings, I don't particularly think that replacing pkgdb with pagure was bad idea even it was not completely ready. We are not able to predict everything what would break if we change / replace piece X. puiterwijk, pingou and others are very responsive and fixing every issues are coming up, this is awesome! While I don't like how modularity does things and stuff like that, I try to help as much as I can.. What I don't like and get frustrated is that modularity is forced a lot and we get huge changes in infrastructure and a lot of other pieces, but when we ask for way smaller changes (supporting rich dependencies), everyone hides (definitely there are exceptions)... For more than half year we are trying to get answers, Rust Packaging got delayed by 2 releases already .. And only now, something is starting to happen (after Flock). That's my main frustration with modularity (it can be replaced with anything else what would be forced so much in Fedora while ignoring other stuff). What I am trying to say, doing changes (even they are not fully prepared) is fine, but completely ignoring people who want to make some changes which affect infrastructure while doing even bigger and less prepared changes is bad. > > Emmanuel > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org - -- - -Igor Gnatenko -BEGIN PGP SIGNATURE- iQIzBAEBCAAdFiEEhLFO09aHZVqO+CM6aVcUvRu8X0wFAlmhgPgACgkQaVcUvRu8 X0w3BBAAhSwbCN67O87K6bhmt96Do2hZ7YY6uaue4i+C4g5b2m+2Xf7A8m3XL9C1 xfR/RZMJnar+a1VHhCbbGROZmLTElgKDsWZOAaCFYIdAkOLbuRoUkM/EfMqE0OUr FjjHH7Uyuxye7KrbiUihjETw0yzVeX3tDWWusgla35PA+RqQe1Z5E+VM6DvL4pFY 953LIwfoA+ReMDKKpwFJatVgTyzNHjx+Co1xpMOIgrgnkkCFWwIK5JYkvRQo31uh zkb7c1Li5Up6wh0lW04morvqp8FVq2G6O/cN/RihT1n8jvztCFP7kYFmRaVNtOKW YwqS6hB+xY5ZogjkmdVkZLDApzEAD/NU4pE/IOhwChTmUFqXSktlveKQHSmS4aLa JwJrHjFpVZ3lxHxDu8ctahfmjMV2XpmaeE8H0d/0cq3y/UhI7WwFnAByOFPYWTwe aGJ1zB9spS+JywwGXs1l5KufQ8l0SSbqELHwglUg2LnqQsRoJOJvh72GfcS2rFCm GJn93ZUNkmVvAghcp9GxfUh3x8nzv4e9mFxTHD4D4duOotC/ggaqHulAJ4vQdXOx Vg+rubv0GFt1tYDeR4JyaC05ZjbB5RSgCRnQIdA9M6J3vvEvG7vPaF9cZyyPLugK xBgghEKHhn8ok72nnzHMlY6gCk+czwaruLA9AdOVUmHkmUEsfcg= =240I -END PGP SIGNATURE- ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
* Matthew Miller [25/08/2017 16:23] : > > From a user perspective, > https://apps.fedoraproject.org/packages/ works very well for me (and > it's _way_ faster than it was several years ago). I would agree if it wasn't for packages' issue #286 I suspect that packages uses pkgdb to get the information it displays so arguing that the former can replace the latter after its retirement seems misguided. > From a contributor > perspective, https://src.fedoraproject.org/ seems adequate if not yet > perfect. The ability to display readme files _alone_ seems like a big > step forward. It'd be nice to have better search and browsing, but the > original pkgdb web UI wasn't great there _either_. It's still has a number of critical issues (#6208 is the biggest one that comes to me). I think Neal's point is that we knew this was coming. We knew that pkgdb retirement would be half-baked before it happenned. We even discussed it on this very list. Couldn't we have delayed the pkgdb retirement until it had less issues? Can we learn from this change and ensure future changes are handled better? Emmanuel ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
Matthew Miller wrote: > I hear three different main frustrations here: first, the short F27 > schedule; second, frustration with with the packagedb retirement; and > third, modularity. These are frustrations I also share: > Schedule > [snip] > If, instead, we lengthened the schedule, adding 6 months from the F26 > release, we'd be in mid/late December, which we know from experience means > January. With delays, maybe February pushing March. And then, we'd be in > the exact same position for F28, debating whether to shorten that or > instead schedule for September instead of May. That would actually be > _more_ delay for _more_ things. You could actually just have skipped the October-November release as you did during the F21 cycle, leading to a single 9-month cycle, then back to 6 months. I think a 9-month cycle would be much more realistic than a 3-month cycle to fix a 3-month offset from the desired release dates. Three 7-month cycles would also have worked (and if the first one really slipped as much as you think, it'd be a 9-month cycle and you'd switch back to 6 months right away, as in the previous paragraph). > Pkgdb retirement > === > > On the one hand, the new https://src.fedoraproject.org/ site is > awesome. Is it really? The Pagure git browser is still missing as basic features as browsing the history of individual files or directories, something I already complained about when it was forced on us as a replacement for the Fedora Hosted Trac. Both Trac and Cgit are nice, mature Free Software projects. Their repository browsing abilities are great. I do not see why they needed to be replaced with something incomplete just because they were Not Invented Here and the incomplete replacement was. > Modularity > == > > Yes, there's a lot of prototyping and reworking. Some of this may fail. > But that's how it should be — it's really not possible to have > innovation without trying some unsure paths. Going down a road that will almost certainly fail is not "how it should be". > If you want to try other paths that help us solve some of the same > problems, I'll support experimenting with them too. I am both not sure that the problems you want to solve are actually problems, and that if they are, they can be solved in any reasonable way. (And yes, this implies that I do not consider the Modularity plans reasonable.) > On your specific complaint about DNF not distinguishing between rpms > and modules, you're definitely not alone; the Boltron feedback > overwhelming tilted that way. See earlier thread on topic — and as I > said in that thread, I think it makes most sense to treat modules like > "super comps groups" (with either a superset of existing groups syntax > and reuse of @, or simply a different syntax). In any case, this > feedback _is_ important and _is_ listened to. I, too, hope that this will be addressed. What I also hope is that there will be a way (by uninstalling a plugin RPM or by setting some dnf.conf option) to avoid retrieving the module metadata entirely. (In fact, I'd hope that that'll be the default on the traditional Spins!) > I do want to _strongly_ stress, though, that the point of modularity > isn't to avoid parallel installability. That's a separate problem. It is not clear to me what problems are solved by Modularity that cannot be solved by either parallel-installable compat packages or just making everything work with the latest versions as we have always done. Modularity just adds conflicts (that need containers to work around), inconsistent EOL dates (that make life harder for our users, including us ourselves), and more complexity for us packagers. > Modularity will allowing *us* at the packaging end to separate source > and spec lifecycle from binary and artifact lifecycle. That, by itself, is not a goal. It is a way to achieve an unspecified goal. > And, it will also allow those of us working on assembling spins and more > options — for example, we can have different streams for Atomic without > needing to actually duplicate every package. (And we could do the same for > KDE or whatever other artifacts would benefit from that, if we want.) That destroys the possibility to install all Fedora packages on all Fedora spins, which the shared Everything repository guaranteed. With Modularity fully implemented, you can end up being unable to install KDE Plasma on the GNOME Workstation or the other way round (at least without resorting to containers) because the desktops depend on conflicting versions of some library module. How is that an improvement? > And that's not even with doing arbitrary branching for individual > packages. The arbitrary branching is what leads to users having to track a different end of life date for, in the worst case, every single package. If Modularity is a success, users will have installed dozens of modules. If each of them comes with a different branch
Re: [modularity] Modules and AppStream
On Fri, Aug 25, 2017 at 4:23 PM, Matthew Miller wrote: > Modularity will allowing *us* at the packaging end to separate source > and spec lifecycle from binary and artifact lifecycle. And, it will > also allow those of us working on assembling spins and more options — > for example, we can have different streams for Atomic without needing > to actually duplicate every package. (And we could do the same for KDE > or whatever other artifacts would benefit from that, if we want.) > > And that's not even with doing arbitrary branching for individual > packages. If everything works out successfully with _that_, it will > eventually make _less_ work for packagers. Right now, I have a package > which I maintain for F25, F26, Rawhide, EPEL6, and EPEL7. There's no > difference in any of the versions and no real reason to keep them > separated; in the future, I will be able to have just one branch that > goes to all of them. Or I could have a "stable" and "experimental" > branch that people could choose from regardless of the base. This can > benefit *way* more packages than simply leaf desktop applications. > > Will we get there? I don't know! It's new and different and definitely > scary. But... it's also worth trying. > > In the meantime, if you're a packager who doesn't care about any of > this, until modularity can demonstrate real success and advantages, you > can _continue_ to not care. Modules are going to be drawn from f27 (and > I expect f28) streams which will act as always, and even if Fedora > Modular Server is a success, I expect we'll also provide a minimal > install from which a traditional Fedora-based server can be built for a > good long time. I have similar concerns and frustrations as Neal does, I think. I first want to comment that I appreciate your willingness to engage people (like Neal, and like myself) who seem frustrated with the future direction of Fedora. And also, I think modularity-- as you described it above-- is a very admirable goal. I have plenty of packages that do not really need separate versions per each Fedora release, and it would be nice to only need to maintain one branch for them. My fear is that Fedora, in the process of rolling out modularity, will get halfway to the idealized goal and then discover that it's not possible to go the rest of the way (for whatever reason), but also that it's not possible to easily roll back to a non-modular world, and we'll be stuck. Even if we don't encounter some critical design flaw, I could certainly see us learning that it's far more complex to maintain modules in practice than we think, or perhaps that it ultimately makes things more complicated for users rather than less. Now, perhaps this won't happen. Indeed, hopefully it does not. But I think the retirement of pkgdb-- which is a prerequisite for modularity-- on a short timescale in a half-baked manner (as you said) is an example of how it's all too easy *for* it to happen. Respectfully, this does not inspire confidence in how well the rollout of modularity across the entire distribution will go. Ben Rosser ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Fri, Aug 25, 2017 at 10:06:42AM -0400, Neal Gompa wrote: [snip] Thanks Neal. This is much more useful and I appreciate you taking the time despite the frusturation. > I'm extremely frustrated by how much half-baked-ness has been going on > in the last couple of months. Schedule shrinkage, features getting > cut, this modularity thing being implemented in a way that's looking > increasingly impossible to bypass while everyone I've talked to seems > to indicate that all of this is prototypical and likely to be > completely reworked again (which has already happened at least once This is a lot to unpack, but I'll try as much as I can. Some of my comments are pushback, but don't take that to mean I'm not taking the concerns seriously. Fedora *is* a project for baking things, and half-bakedness is an inherent step in baking. And I don't just mean in a "testing stuff for RHEL" sense — Fedora _should_ be leading in the distro space overall. On the other hand, we _don't_ want to be _unbaked_ (or "bleeding edge", if you like). We want what gets to users and to our mainstream contributors to be better than that. I hear three different main frustrations here: first, the short F27 schedule; second, frustration with with the packagedb retirement; and third, modularity. Schedule I talked about the schedule and features with you a little bit on Twitter, but I'll say it in more than 140 characters here. Keeping to the schedule as we've defined for years isn't really "schedule shrinkage". It just feels new because we've never _really_ stuck to it before. In retrospect, I really wish I'd pushed for a shorter _F26_ schedule when that was being defined last year, but here we are now. I previously brought up the idea of flat-out changing the schedule to 9 months instead of 6 months, but literally no one supported that — there was a strong feeling that the six-month cadence is important. So, here we are figuring out how to live with that. Pushing back features is really only the way to do this. Cramming everything in would be even more chaotic. But, the important thing is that when we do it this way, we know with reasonable certainty that the F27 release will come out at a predictable time not so far in the future. Features which miss this release _will_ get to users in a reasonable timeframe. If, instead, we lengthened the schedule, adding 6 months from the F26 release, we'd be in mid/late December, which we know from experience means January. With delays, maybe February pushing March. And then, we'd be in the exact same position for F28, debating whether to shorten that or instead schedule for September instead of May. That would actually be _more_ delay for _more_ things. There's _really_ not a great answer here, but I think what we're doing is the best of the available choices. Pkgdb retirement === On the one hand, the new https://src.fedoraproject.org/ site is awesome. It's going to open up a whole lot of potential for new workflows and new contributors. On the other hand, yeah, the pkgdb retirement _was_ half-baked in retrospect. But, the people working on that are still baking. When you hit specific issues which impact your work, *please* file tickets at https://pagure.io/fedora-infrastructure/issues so they can get tracked and fixed. I'm not sure I really understand the part about not being able to see what's available in Fedora. From a user perspective, https://apps.fedoraproject.org/packages/ works very well for me (and it's _way_ faster than it was several years ago). From a contributor perspective, https://src.fedoraproject.org/ seems adequate if not yet perfect. The ability to display readme files _alone_ seems like a big step forward. It'd be nice to have better search and browsing, but the original pkgdb web UI wasn't great there _either_. Modularity == Yes, there's a lot of prototyping and reworking. Some of this may fail. But that's how it should be — it's really not possible to have innovation without trying some unsure paths. If you want to try other paths that help us solve some of the same problems, I'll support experimenting with them too. On your specific complaint about DNF not distinguishing between rpms and modules, you're definitely not alone; the Boltron feedback overwhelming tilted that way. See earlier thread on topic — and as I said in that thread, I think it makes most sense to treat modules like "super comps groups" (with either a superset of existing groups syntax and reuse of @, or simply a different syntax). In any case, this feedback _is_ important and _is_ listened to. I do want to _strongly_ stress, though, that the point of modularity isn't to avoid parallel installability. That's a separate problem. Modularity will allowing *us* at the packaging end to separate source and spec lifecycle from binary and artifact lifecycle. And, it will also allow those of us working on assembling spins and more options — for example, we can have di
Re: [modularity] Modules and AppStream
On Fri, Aug 25, 2017 at 12:17 PM, Stephen John Smoogen wrote: > On 25 August 2017 at 10:06, Neal Gompa wrote: >> On Fri, Aug 25, 2017 at 9:42 AM, Rex Dieter wrote: >>> Neal Gompa wrote: >>> On Fri, Aug 25, 2017 at 8:54 AM, Matthew Miller wrote: > On Fri, Aug 25, 2017 at 08:25:22AM -0400, Neal Gompa wrote: >> This is a stupid idea because it introduces "magic" into picking the >> type of thing installed. It also goes against how we typically do > > Neal, this idea is the idea of other Fedora contributors who have put a > lot of thought into it. Please refrain from childish language like > "stupid". I'm not saying that this is "offensive" or anything, but I > hope for a higher level of discourse in Fedora. There's no need for > insults. You have a technical argument; please stick to that. > I've already turned down my feelings about it several notches. But at least you noticed what I said by the words I chose. That's was the whole point. I'm going to be blunt about how I feel about it and present my technical argument. Being purely technical tends to lead to being ignored. You noticed because I didn't do that. >>> >>> I agree with Matthew on this one. You got noticed, yes, but for *very* much >>> the wrong reasons... which may backfire in the long run: your non- >>> inflamatory/technical feedback will indeed have higher risk of not being >>> considered seriously. >>> >> >> I'm extremely frustrated by how much half-baked-ness has been going on >> in the last couple of months. Schedule shrinkage, features getting >> cut, this modularity thing being implemented in a way that's looking > > I am going to say the following as a guide for other people in the future. > > If you had taken the time to explain this in the first place versus > calling it stupid, I would have paid attention. This clearly explains > why you are frustrated and what the problems you are seeing in details > which can be dealt with. The "stupid" may seem to be a great > distillation of all those problems but it also comes across too many > times as "I am scoring points on the elementary playground" so yes it > gets initial attention but people just want to punch you in the face > versus listen to what is making you so angry/upset. > > To be clearer. Your original email just made me upset at *you* and > pretty much shut me down from wanting to see it from your point of > view. Reading this has made me realize that many of my current > frustrations with how everything is and yours are the same and I can > agree with many of the points. Even to the following though I don't > know what DID is (Developers in Distress?) > DID == Dissociative Identity Disorder. Also known as multiple personality disorder (MPD). Though Developers in Distress works too. :) But here's the thing... Why are you listening to me now about this, as opposed to the many other times I brought it up when they were smaller and more manageable issues to be resolved early? Even Matt knew of several of the problems that wound up happening because *I told him* in person several months ago. I *detailed* exactly what services would break when we did this, and how many workflows would be busted. I've told other people that we don't have anything meaningful to replace these things, and that no one even *knows* what the heck PDC does (which somehow replaces this, maybe?), except maybe Adam Williamson. It's not like logging into the bloody thing gets you anything useful either. Why do I have to have to rip my hair out in frustration (and those who met me in person know that's a lot to rip out!) before someone sits up and pays attention? Why does everything have to go to hell first? It's very clear that there's no point in evaluating things and telling people what you're breaking before you break it, because no one cares. On the flip side, actually trying to be proactive about things goes nowhere too (c.f. Rust and releng). It feels like there's no point to even trying to work with these groups of people anymore, because they *don't care*. I don't know what you want from me anymore... And apparently there are others that feel the same, judging by some of the conversations I've had on various channels with folks. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On 25 August 2017 at 10:06, Neal Gompa wrote: > On Fri, Aug 25, 2017 at 9:42 AM, Rex Dieter wrote: >> Neal Gompa wrote: >> >>> On Fri, Aug 25, 2017 at 8:54 AM, Matthew Miller >>> wrote: On Fri, Aug 25, 2017 at 08:25:22AM -0400, Neal Gompa wrote: > This is a stupid idea because it introduces "magic" into picking the > type of thing installed. It also goes against how we typically do Neal, this idea is the idea of other Fedora contributors who have put a lot of thought into it. Please refrain from childish language like "stupid". I'm not saying that this is "offensive" or anything, but I hope for a higher level of discourse in Fedora. There's no need for insults. You have a technical argument; please stick to that. >>> >>> I've already turned down my feelings about it several notches. But at >>> least you noticed what I said by the words I chose. That's was the >>> whole point. >>> >>> I'm going to be blunt about how I feel about it and present my >>> technical argument. Being purely technical tends to lead to being >>> ignored. You noticed because I didn't do that. >> >> I agree with Matthew on this one. You got noticed, yes, but for *very* much >> the wrong reasons... which may backfire in the long run: your non- >> inflamatory/technical feedback will indeed have higher risk of not being >> considered seriously. >> > > I'm extremely frustrated by how much half-baked-ness has been going on > in the last couple of months. Schedule shrinkage, features getting > cut, this modularity thing being implemented in a way that's looking I am going to say the following as a guide for other people in the future. If you had taken the time to explain this in the first place versus calling it stupid, I would have paid attention. This clearly explains why you are frustrated and what the problems you are seeing in details which can be dealt with. The "stupid" may seem to be a great distillation of all those problems but it also comes across too many times as "I am scoring points on the elementary playground" so yes it gets initial attention but people just want to punch you in the face versus listen to what is making you so angry/upset. To be clearer. Your original email just made me upset at *you* and pretty much shut me down from wanting to see it from your point of view. Reading this has made me realize that many of my current frustrations with how everything is and yours are the same and I can agree with many of the points. Even to the following though I don't know what DID is (Developers in Distress?) > If I wasn't a long-time Fedora user and contributor who actually knew > his stuff and knows how to navigate around all the messes, I'd > probably have left by now because it feels like everything is on fire, > and not in a good way. > > In your own little worlds, everything might seem like it's fine, but > out here in the outer rings, closer to nexus of the intersection of > the regular world with the Fedora worlds, it looks like Fedora is > getting a severe case of DID again. > > The messaging is wrong, the implementation doesn't make sense, and > there isn't actually a problem to solve that you aren't creating for > yourselves already. > > *Throws hands up in the air* > > -- > 真実はいつも一つ！/ Always, there's only one truth! > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org -- Stephen J Smoogen. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Fri, Aug 25, 2017 at 9:42 AM, Rex Dieter wrote: > Neal Gompa wrote: > >> On Fri, Aug 25, 2017 at 8:54 AM, Matthew Miller >> wrote: >>> On Fri, Aug 25, 2017 at 08:25:22AM -0400, Neal Gompa wrote: This is a stupid idea because it introduces "magic" into picking the type of thing installed. It also goes against how we typically do >>> >>> Neal, this idea is the idea of other Fedora contributors who have put a >>> lot of thought into it. Please refrain from childish language like >>> "stupid". I'm not saying that this is "offensive" or anything, but I >>> hope for a higher level of discourse in Fedora. There's no need for >>> insults. You have a technical argument; please stick to that. >>> >> >> I've already turned down my feelings about it several notches. But at >> least you noticed what I said by the words I chose. That's was the >> whole point. >> >> I'm going to be blunt about how I feel about it and present my >> technical argument. Being purely technical tends to lead to being >> ignored. You noticed because I didn't do that. > > I agree with Matthew on this one. You got noticed, yes, but for *very* much > the wrong reasons... which may backfire in the long run: your non- > inflamatory/technical feedback will indeed have higher risk of not being > considered seriously. > I'm extremely frustrated by how much half-baked-ness has been going on in the last couple of months. Schedule shrinkage, features getting cut, this modularity thing being implemented in a way that's looking increasingly impossible to bypass while everyone I've talked to seems to indicate that all of this is prototypical and likely to be completely reworked again (which has already happened at least once now, apparently), and to top it all off, the Pagure over Dist-Git thing has been driving me bonkers for the last few weeks. I also now get complaints from people that they can't figure out what Fedora actually has in the distribution anymore because we don't have PackageDB. Who thought it was a good idea to not have a way for people to figure out who maintains what and what is in the distribution? The package search application is broken and still not updating, too. Not to mention, after trying the Boltron server, I'm extremely disappointed in how it behaves. There's no obvious way to determine whether I'm requesting a module or a package, and the *additional* metadata that uses *yet another parser* means that we're back to Yum 2 levels of slowness with even more complexity than there was before. Look, I don't care if you want to do your modules in TOML, YAML, JSON, or whatever. But it's a complexity and security risk if have to deal with multiple types of repository metadata parsers. It makes independent tool handling of the metadata harder, it makes implementing repository management harder, and it makes things really difficult for build systems and other tools that have to resolve things. You make my job harder as a packager, my job harder as a developer, my job harder as a systems administrator, and my job harder as a user. All of this to simply avoid just setting up packages to be parallel installable. In most stacks, this is already the default. For example, Python, Ruby, Java, etc already do this, we just don't really leverage it. Other stacks (like PHP, Perl, etc.) can be set up that way relatively easily. You talk of independent life cycles, but aside from leaf desktop applications, it's not practical or realistic. Cutting things around all over the place clearly speaks towards some kind of RHEL thing where they don't like the organic growth of the dependency chain. That's fine, it's their distribution and they can shut off features there that are available in Fedora. But the wounds inflicted by creating these "modules" that add more special metadata, require custom depsolving, and so on just make no sense. Insofar as the custom depsolving, it reads to me like something where we want to avoid having our build system be able to automate a lot of the hellish things we do now (like the ImageMagick bump going on right now). Instead, these weird modules rebuild everything again and again. I'm not really going to get into this more, because clearly no one cares as I've attempted to bring this up before in other threads with crickets responding. So, what's the use? And the Flatpak stuff (since Owen did bring that up earlier in the thread) also is strange, because it effectively doubles or triples the maintenance work of software. And Flatpak still has no reproducibility mechanisms, which makes it hard to do secure software delivery. But at least Flatpak operates on an independent space, and doesn't actually change too much of how the regular distribution is built. So it has somewhat of a pass from me. If I wasn't a long-time Fedora user and contributor who actually knew his stuff and knows how to navigate around all the messes, I'd probably have left by now because it feels like everything is on fire, and not
Re: [modularity] Modules and AppStream
Neal Gompa wrote: > On Fri, Aug 25, 2017 at 8:54 AM, Matthew Miller > wrote: >> On Fri, Aug 25, 2017 at 08:25:22AM -0400, Neal Gompa wrote: >>> This is a stupid idea because it introduces "magic" into picking the >>> type of thing installed. It also goes against how we typically do >> >> Neal, this idea is the idea of other Fedora contributors who have put a >> lot of thought into it. Please refrain from childish language like >> "stupid". I'm not saying that this is "offensive" or anything, but I >> hope for a higher level of discourse in Fedora. There's no need for >> insults. You have a technical argument; please stick to that. >> > > I've already turned down my feelings about it several notches. But at > least you noticed what I said by the words I chose. That's was the > whole point. > > I'm going to be blunt about how I feel about it and present my > technical argument. Being purely technical tends to lead to being > ignored. You noticed because I didn't do that. I agree with Matthew on this one. You got noticed, yes, but for *very* much the wrong reasons... which may backfire in the long run: your non- inflamatory/technical feedback will indeed have higher risk of not being considered seriously. -- Rex ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Fri, Aug 25, 2017 at 09:15:03AM -0400, Neal Gompa wrote: > I'm going to be blunt about how I feel about it and present my > technical argument. Being purely technical tends to lead to being > ignored. You noticed because I didn't do that. No. I noticed because I read and think about every message on this list. Throwing in insults to get attention is not appropriate. Please don't do it further. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Fri, Aug 25, 2017 at 8:54 AM, Matthew Miller wrote: > On Fri, Aug 25, 2017 at 08:25:22AM -0400, Neal Gompa wrote: >> This is a stupid idea because it introduces "magic" into picking the >> type of thing installed. It also goes against how we typically do > > Neal, this idea is the idea of other Fedora contributors who have put a > lot of thought into it. Please refrain from childish language like > "stupid". I'm not saying that this is "offensive" or anything, but I > hope for a higher level of discourse in Fedora. There's no need for > insults. You have a technical argument; please stick to that. > I've already turned down my feelings about it several notches. But at least you noticed what I said by the words I chose. That's was the whole point. I'm going to be blunt about how I feel about it and present my technical argument. Being purely technical tends to lead to being ignored. You noticed because I didn't do that. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Fri, Aug 25, 2017 at 08:25:22AM -0400, Neal Gompa wrote: > This is a stupid idea because it introduces "magic" into picking the > type of thing installed. It also goes against how we typically do Neal, this idea is the idea of other Fedora contributors who have put a lot of thought into it. Please refrain from childish language like "stupid". I'm not saying that this is "offensive" or anything, but I hope for a higher level of discourse in Fedora. There's no need for insults. You have a technical argument; please stick to that. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Thu, Aug 24, 2017 at 7:44 AM, Stephen Gallagher wrote: > > > On Thu, Aug 24, 2017 at 3:02 AM Marius Vollmer > wrote: >> >> My understanding from playing with Boltron is that "dnf install foo" >> treats "foo" as a module name. "dnf install" can not install packages >> anymore. So naively I assume that PackageKit will transparently start >> installing modules. (I think PK uses libdnf, and I think I read >> somewhere that libdnf doesn't do anything with modules yet, so...) > > > This is not true. The modularity-aware DNF will first check to see if a > module matching the requested name exists and will install that. If it > doesn't, it will fall back to packages. > > That being said, the implementation is still under debate. I'm personally of > the opinion that this is the correct behavior (since it requires no > adaptation for users), but there are some people who believe that it should > be explicit with `dnf module install` instead of `dnf install`. > This is a stupid idea because it introduces "magic" into picking the type of thing installed. It also goes against how we typically do this. You either have a modifier that indicates you want to install a module, or you have a module subcommand that has install/remove/upgrade subcommands. I really hope the DNF team doesn't go through with that idea, as it's super-Fedora specific and not really helpful for people who may want to choose to ignore Modularity stuff (since it doesn't apply to them), and other distributions now use DNF (Mageia, Yocto, etc.) that really don't want this. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
Stephen Gallagher writes: > That being said, the implementation is still under debate. Thanks for the clarification! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
Hi, I suspect the confusion between group and module names will lead to strange brittle special cases down the road and (worse) people being over-clever building solutions that rely on those special cases (exactly like the under-specified rpm update quirks which are being blamed nowadays). Why not make the module shorthand specific? package ⊂ @group ⊂ @@module Or am I missing something? Regards, -- Nicolas Mailhot ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
Dne 24.8.2017 v 09:00 Marius Vollmer napsal(a): My understanding from playing with Boltron is that "dnf install foo" treats "foo" as a module name. "dnf install" can not install packages The final solution will be: dnf install foo - install rpm package dnf module install foo - install module dnf install @foo - install module (if comps of the same name exists it install comps, otherwise module, or vice versa, somebody from DNF team should clarify). Mirek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Thu, Aug 24, 2017 at 3:02 AM Marius Vollmer wrote: > My understanding from playing with Boltron is that "dnf install foo" > treats "foo" as a module name. "dnf install" can not install packages > anymore. So naively I assume that PackageKit will transparently start > installing modules. (I think PK uses libdnf, and I think I read > somewhere that libdnf doesn't do anything with modules yet, so...) > This is not true. The modularity-aware DNF will first check to see if a module matching the requested name exists and will install that. If it doesn't, it will fall back to packages. That being said, the implementation is still under debate. I'm personally of the opinion that this is the correct behavior (since it requires no adaptation for users), but there are some people who believe that it should be explicit with `dnf module install` instead of `dnf install`. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
Richard Hughes writes: > On 24 August 2017 at 08:45, Marius Vollmer wrote: > >> One approach is just to put them all into the collection data: > > You can't have two components with the same ID inside a > group with the same origin. Okay, that was my understanding of the spec as well. >> My proposed approach was to encode the exact same data via a spec >> extension: >> >> >>Recommended >> >> >>Nightly builds >>foo-nightly >> > > I'm sorry, but that makes almost no sense. What is the software > center supposed to do with variantsummary? Is the application name the > same between the two variants? The variantsummary is meant to be used in the UI element for selecting a variant. > Should reviews for the different variants be treated as the same thing > or as different things? I would say reviews for all variants should be associated with the single component id. I guess reviews carry information about the version of the component that they were made for already, and the same mechanism could be used to say which variant they apply to. > I don't think it makes any sense at all making projects like GNOME > Software, Apper, Discover, Muon and Cockpit (and others) understand > much about the Fedora modularity stuff when everything seems to have > standardized on AppStream -- including next-generation distributions > methods like Flatpak. This is all optional. I am exploring what happens if a package with AppStream metainfo data appears in a module, and I think I have sketched out a path where we can do something correct/sensible with it that doesn't require any changes to existing AppStream consumers. A better path IMO is to decide that Software Centers don't want to see naked modules when dealing with Applications, just flatpaks or other containers. I wasn't sure that this is an acceptable option, but it looks like it is. >> If a client understands "variants", it can trivially expand them back >> into multiple entries for the same id. Clients that don't understand >> this will continue to just see one. > > They're two different things. Would gnome-software and cockpit show > two search entries for the two variants or one? One. > Would you be able to switch between the variants? Yes. > Can both be installed at the same time, by two different users on the > same system? No. >> Maybe I should emphasize that I am only trying to figure out what >> happens if we subject a modularized repository to the usual AppStream >> treatment. > > Can you provide some specifications on what a modularized repository > should actually look like please? Is a repo going to contain .rpm > packages like firefox-1.23.rpm and also firefox-2.34.rpm or something > completely different? This is a modularized repo, I think: https://kojipkgs.fedoraproject.org/compose/26/Fedora-Modular-26-20170720.0/compose/Server/x86_64/os/ It does contain both nodejs-6.10.3-2.module_9c237b2a.x86_64.rpm and nodejs-8.0.0-1.module_42d8f2a0.x86_64.rpm for example. I hope the modularity people on this list can provide more details. > Has anyone talked to the Fedora or GNOME designers about this? I will talk to our Cockpit designers, of course, once everyone is back from vacation. > I'm trying hard not to get frustrated about questions about XML schema > when I don't think anybody has considered the user experience of > modularity. Well, I have learned a lot already from this discussion, so thanks for that! > From a desktop point of view I'm currently of the opinion that it only > makes sense to show modules that are flatpaks, and continue to use the > appstream branch of the flatpak repository to get information about > the modules. This makes a lot of sense to me. >> I am happy to just say that modularized repositories don't have >> AppStream data, period. > > Why are you happy they don't have AppStream? I'm not sure trying to > antagonize the people involved is a very good idea at all. Sorry, again I should have been clearer: Instead of figuring out how to make AppStream work with modularized repositories, we can also say that we wont be installing applications as RPMs from modularized repositories (only as flatpaks etc), and thus a modularized repository doesn't need any AppStream data (period) and we don't need to figure out how the make that work. However, the general concept of variants of a single applications and letting the user choose between them is a good one, no? Firefox Regular, Firefox Nightly, and Firefox Fatfree _are_ related to each other and we can do better than just showing them next to each other in a list, no? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On 24 August 2017 at 08:45, Marius Vollmer wrote: > If appstream-builder finds two packages that both contain metainfo for > the same component id, what does it do? If I understand correctly, in appstream-builder it's an error, and the "first encountered" component wins. > What should it do? What should > the Software Center do. This isn't described in the AppStream spec > beyond the "merge" and "pripority" attributes for the tag. You have to remember, the AppStream spec was initially designed as a way to map application IDs to package names. AppStream metadata is written by basically every distro and consumed by every software center, and it's not Fedora specific, so you probably shouldn't be super surprised it doesn't map into the Fedora modularity system very well. My personal feeling is that this whole modularity initiative is being pushed hard into all layers of Fedora without actually working out how this is going to work with the various upstream specifications and projects. I'm sure it'll be great for Fedora in the short term, but longer term it's going to hurt being a little island of Fedora-specific schemas and tools. > One approach is just to put them all into the collection data: You can't have two components with the same ID inside a group with the same origin. > This is what they do for Flatpaks if I understand Owen correctly. No, flatpaks components have different names for each different branch. e.g. the org.mozilla.Firefox.Nightly.desktop thing I explained in my last email. > I had assumed that the spec doesn't allow this and I thought that > changing it to allow it would be too drastic. If this is established > practice and we just need to update the spec to catch up with reality, > great! No, please can you re-read mine and Owens previous emails please. > My proposed approach was to encode the exact same data via a spec > extension: > > >Recommended > > >Nightly builds >foo-nightly > I'm sorry, but that makes almost no sense. What is the software center supposed to do with variantsummary? Is the application name the same between the two variants? Should reviews for the different variants be treated as the same thing or as different things? To me a module is a superset of a set of packages, much like a -- so it would make sense to have something like org.fedoraproject.modularity.http2-6 which contains the translated name, the translated description, the SPDX license, and any URL links too. Of course, this is mostly the same as the modulemd metadata as specified https://docs.pagure.org/modularity/development/building-modules/developing.html so you can certainly create AppStream from that pretty easily. I don't think it makes any sense at all making projects like GNOME Software, Apper, Discover, Muon and Cockpit (and others) understand much about the Fedora modularity stuff when everything seems to have standardized on AppStream -- including next-generation distributions methods like Flatpak. > If a client understands "variants", it can trivially expand them back > into multiple entries for the same id. Clients that don't understand > this will continue to just see one. They're two different things. Would gnome-software and cockpit show two search entries for the two variants or one? Would you be able to switch between the variants? Can both be installed at the same time, by two different users on the same system? If the modules are flatpaks then they're two different components with two different AppIDs, that can both be installed at the same time. I'm having a very hard time working out how we'd communicate difficult to understand concepts to the end user using packages, even wrapped up with modulemd metadata. > Maybe I should emphasize that I am only trying to figure out what > happens if we subject a modularized repository to the usual AppStream > treatment. Can you provide some specifications on what a modularized repository should actually look like please? Is a repo going to contain .rpm packages like firefox-1.23.rpm and also firefox-2.34.rpm or something completely different? If both packages contain a metainfo/appdata file with the same component it's just not going to work very well using appstream-builder. > I think we would create broken AppStream collection data right now, or > at least AppStream data that doesn't let us take advantage of the new > features that modularity enables (streams and profiles). Has anyone talked to the Fedora or GNOME designers about this? I'm trying hard not to get frustrated about questions about XML schema when I don't think anybody has considered the user experience of modularity. From a desktop point of view I'm currently of the opinion that it only makes sense to show modules that are flatpaks, and continue to use the appstream branch of the flatpak repository to get information about the modules. > I am happy to just say that modulari
Re: [modularity] Modules and AppStream
Richard Hughes writes: > On 23 August 2017 at 13:57, Marius Vollmer wrote: > >> I propose to keep AppStream metainfo data in packages, and map from >> package names to module names during construction of the collection >> data. > > Can you elaborate a bit? At the moment in Fedora we generate the > AppStream metadata from a mirror of all the packages using > appstream-builder. When creating the AppStream collection data for a modularized repository, we might want to insert information about the modules that a package belongs to into the collection data, maybe as a tag. So we need to have a function that takes a package name and returns a list of (module name, stream, profile) tuples. (Alternatively, we could require that the modulemd data for a module contains everything we need for the AppStream collection data. I was proposing not to do this and just keep looking at the packages.) >> - Because of streams and profiles, there can be multiple versions of >>metainfo for a given AppStream component id. These need to be >>merged > > No, I don't think they do. If you have two very different versions of > Firefox (one LTS, one bleeding edge) you want a different description, > different screenshots and different reviews. Sorry, I wasn't clear. I shouldn't have said "merged". If appstream-builder finds two packages that both contain metainfo for the same component id, what does it do? What should it do? What should the Software Center do. This isn't described in the AppStream spec beyond the "merge" and "pripority" attributes for the tag. One approach is just to put them all into the collection data: com.example.foo Foo foo com.example.foo Foo foo-nightly This is what they do for Flatpaks if I understand Owen correctly. I had assumed that the spec doesn't allow this and I thought that changing it to allow it would be too drastic. If this is established practice and we just need to update the spec to catch up with reality, great! My proposed approach was to encode the exact same data via a spec extension: com.example.foo Foo foo Recommended Nightly builds foo-nightly If a client understands "variants", it can trivially expand them back into multiple entries for the same id. Clients that don't understand this will continue to just see one. Maybe I should emphasize that I am only trying to figure out what happens if we subject a modularized repository to the usual AppStream treatment. I think we would create broken AppStream collection data right now, or at least AppStream data that doesn't let us take advantage of the new features that modularity enables (streams and profiles). I am happy to just say that modularized repositories don't have AppStream data, period. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
Owen Taylor writes: > The current expectation is that the only way that modules are going to > show up in GNOME Software is when they are safely wrapped up as a > Flatpak. Ah, that's nice. If we follow the same line for Cockpit, we would only show container images. This would certainly simplify things. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
Richard Hughes writes: > On 23 August 2017 at 13:57, Marius Vollmer wrote: > >> - Metainfo is in packages, but we need to be installing modules. > > How are we going to be installing modules in a modular-system? > PackageKit is only going to be able to install packages so > gnome-software will obviously need to be able to install things. My understanding from playing with Boltron is that "dnf install foo" treats "foo" as a module name. "dnf install" can not install packages anymore. So naively I assume that PackageKit will transparently start installing modules. (I think PK uses libdnf, and I think I read somewhere that libdnf doesn't do anything with modules yet, so...) Bu then again, this all feels quite half baked, so maybe I shouldn't make assumptions based on what Boltron does today. Witness: # docker run --rm -it modularitycontainers/boltron-preview /bin/bash boltron# dnf info guile Name : guile Epoch: 5 Version : 2.0.14 Release : 1.module_39876f37 Arch : x86_64 Size : 3.5 M Source : guile-2.0.14-1.module_39876f37.src.rpm Repo : fedora-modular [...] boltron# dnf install guile No such module: guile Dependencies resolved. Nothing to do. Complete! > [...] > > I don't think you can pretend that applications from different > branches (with different features, bugs and possibly UI) are the > "same" component, especially when you can install zero, one or many at > the same time. With modules, you can not install more than one stream at any time. I think the idea is that module streams provide different versions of packages without having to change the packages in any way. Thus, packages from different streams will mostly likely conflict with each other. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On Wed, 2017-08-23 at 17:25 +0100, Richard Hughes wrote: > On 23 August 2017 at 13:57, Marius Vollmer wrote: > > In a modular repository, I think the same thing should happen so that > > GNOME Software can present interesting modules to the user in a nice > > way. > > What kind of modules would be shown in gnome-software? Are > applications like Firefox going to be supported in the new modularity > system? We're not going to be showing httpd there for instance. The current expectation is that the only way that modules are going to show up in GNOME Software is when they are safely wrapped up as a Flatpak. Because un-encapsulated modules can't be arbitrarily mixed and combined, displaying them to users would add a huge amount of complexity. > > - Metainfo is in packages, but we need to be installing modules. > > How are we going to be installing modules in a modular-system? > PackageKit is only going to be able to install packages so > gnome-software will obviously need to be able to install things. Via flatpak, not via PackageKit. :-) > > Thus, > >the collection data needs to have module names into the AppStream > > tag. > > I think the pkgname tag needs to contain the package name. We have a > tag that might be more suitable for adding things like > version or stream information. yeah, the appstream provided via registry.fedoraproject.org will have what to install in the tag. > > I propose to keep AppStream metainfo data in > >packages, and map from package names to module names during > >construction of the collection data. > > Can you elaborate a bit? At the moment in Fedora we generate the > AppStream metadata from a mirror of all the packages using > appstream-builder. appstream for modules is something that Marius is going to have to eloborate on :-) For the desktop, the appstream data will be collected from the Flatpaks uploaded to registry.fedoraproject.org and made available for download. > > - Because of streams and profiles, there can be multiple versions of > >metainfo for a given AppStream component id. These need to be > >merged > > No, I don't think they do. If you have two very different versions of > Firefox (one LTS, one bleeding edge) you want a different description, > different screenshots and different reviews. > > The way we've solved this in Flatpak is to use a different application > ID, for instance Firefox nightly would be > org.mozilla.Firefox.Nightly.desktop rather than > org.mozilla.Firefox.desktop > > I don't think you can pretend that applications from different > branches (with different features, bugs and possibly UI) are the > "same" component, especially when you can install zero, one or many at > the same time. The Flatpak build tooling currently enforces that the branch is the same as the module stream, but allows the packager to use different IDs for different streams if they want - so they can have a nightly and a stable stream that can be parallel installed as above. (With caveats of needing to modify application code appropriately.) Owen ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: [modularity] Modules and AppStream
On 23 August 2017 at 13:57, Marius Vollmer wrote: > In a modular repository, I think the same thing should happen so that > GNOME Software can present interesting modules to the user in a nice > way. What kind of modules would be shown in gnome-software? Are applications like Firefox going to be supported in the new modularity system? We're not going to be showing httpd there for instance. > - Metainfo is in packages, but we need to be installing modules. How are we going to be installing modules in a modular-system? PackageKit is only going to be able to install packages so gnome-software will obviously need to be able to install things. > Thus, >the collection data needs to have module names into the AppStream > tag. I think the pkgname tag needs to contain the package name. We have a tag that might be more suitable for adding things like version or stream information. > I propose to keep AppStream metainfo data in >packages, and map from package names to module names during >construction of the collection data. Can you elaborate a bit? At the moment in Fedora we generate the AppStream metadata from a mirror of all the packages using appstream-builder. > - Because of streams and profiles, there can be multiple versions of >metainfo for a given AppStream component id. These need to be >merged No, I don't think they do. If you have two very different versions of Firefox (one LTS, one bleeding edge) you want a different description, different screenshots and different reviews. The way we've solved this in Flatpak is to use a different application ID, for instance Firefox nightly would be org.mozilla.Firefox.Nightly.desktop rather than org.mozilla.Firefox.desktop I don't think you can pretend that applications from different branches (with different features, bugs and possibly UI) are the "same" component, especially when you can install zero, one or many at the same time. Richard. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
[modularity] Modules and AppStream
Hi, what happens when the packages in a module contain AppStream metainfo files? In a non-modular repository, all these metainfo files get collected into a big AppStream collection file which is then used by GNOME Software (for example) to present interesting packages to the user in a nice way. In a modular repository, I think the same thing should happen so that GNOME Software can present interesting modules to the user in a nice way. I think we need to change the collection process in the following ways, it unfortunately wont just work: - Metainfo is in packages, but we need to be installing modules. Thus, the collection data needs to have module names into the AppStream tag. I propose to keep AppStream metainfo data in packages, and map from package names to module names during construction of the collection data. - Because of streams and profiles, there can be multiple versions of metainfo for a given AppStream component id. These need to be merged, using something like https://github.com/ximion/appstream/pull/132[1] What do you think? How would you map from a package name to module/stream/profile tuples? Thanks! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
