Re: Managing stream (arbitrary) branch and module lifecycles
On Mon, Sep 24, 2018 at 7:04 AM Adam Samalik wrote: > I thought about this for a while, and I can see some conceptual > similarities between upgrading a major Fedora release and changing a module > stream. I tried to think about major Fedora releases (I mean f28, f29, etc) > as "streams" of Fedora, the same way as streams of modules, with stable API. > > Until now, there was a single type of upgrade in Fedora — the major > release upgrade. But with Modularity, there is more than that. It's no > longer single monolithic upgrade of the whole OS. People can now upgrade > (== change streams) different parts of the OS potentially independently. We > need to think how to present this to users. Will it require a change of > mindset? > I think we have to be very careful about making the default experience have too much complexity to it. For graphical applications we are using Flatpaks to get this separation of OS upgrade and application upgrade, *but* we're planning to have applications be single stream - so applications just roll forward with upstream stable releases. So the total complexity stays low. Also, with the effort of separating apps and the OS we've discussed at > Flock, will the goal be to get the "OS part" upgradable at any time, and, > independently on that, users will choose to upgrade (again, change streams) > individual parts of the system (modules) for new features? That probably > will require a change of mindset. It sounds similar how a phone works. Do > we need to develop a whole new UX around this? > We have no current plans to create a *graphical* user experience around installation of modules as loose packages. Even creating a decent command line experience around it seems very difficult, since if you allow independent module maintenance, two modules can start conflicting at any time (even without a branch switch!). "Updated versions of reviewboard and sagemath have different versions of python2-, do you want to" A) Remove reviewboard B) Remove sagemath C) Use the version of python2- from reviewboard and hope for the best D) Use the version of python2- from sagemath and hope for the best E) Find a nearby brick wall to bang your head against On the other hand, if you don't allow independent module maintenance and enforce compatible versions across the entire set of modules included in the modular compose, you've lost some of the point of modularity... still, that's better, in my opinion than presenting the user with lose-lose options. Owen > On Thu, Sep 20, 2018 at 11:01 PM Randy Barlow < > bowlofe...@fedoraproject.org> wrote: > >> On 9/20/18 1:56 PM, Matthew Miller wrote: >> > If it's "they'll find out when dnf system-upgrade errors out!", then see >> > above. I'm ... not enthused. Something in dnf system-upgrade needs to >> do it; >> > possibly a "dnf system-upgrade prep" step before "download". >> >> I agree. Would it be feasible for the system-upgrade plugin to prompt >> the user with "hey, you are using 1.7 but you need to switch to 1.8 to >> upgrade. Proceed? (y/N)". >> >> ___ >> devel mailing list -- devel@lists.fedoraproject.org >> To unsubscribe send an email to devel-le...@lists.fedoraproject.org >> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> List Archives: >> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org >> > > > -- > > Adam Šamalík > --- > Software Engineer > Red Hat > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
I thought about this for a while, and I can see some conceptual similarities between upgrading a major Fedora release and changing a module stream. I tried to think about major Fedora releases (I mean f28, f29, etc) as "streams" of Fedora, the same way as streams of modules, with stable API. Until now, there was a single type of upgrade in Fedora — the major release upgrade. But with Modularity, there is more than that. It's no longer single monolithic upgrade of the whole OS. People can now upgrade (== change streams) different parts of the OS potentially independently. We need to think how to present this to users. Will it require a change of mindset? Also, with the effort of separating apps and the OS we've discussed at Flock, will the goal be to get the "OS part" upgradable at any time, and, independently on that, users will choose to upgrade (again, change streams) individual parts of the system (modules) for new features? That probably will require a change of mindset. It sounds similar how a phone works. Do we need to develop a whole new UX around this? On Thu, Sep 20, 2018 at 11:01 PM Randy Barlow wrote: > On 9/20/18 1:56 PM, Matthew Miller wrote: > > If it's "they'll find out when dnf system-upgrade errors out!", then see > > above. I'm ... not enthused. Something in dnf system-upgrade needs to do > it; > > possibly a "dnf system-upgrade prep" step before "download". > > I agree. Would it be feasible for the system-upgrade plugin to prompt > the user with "hey, you are using 1.7 but you need to switch to 1.8 to > upgrade. Proceed? (y/N)". > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > -- Adam Šamalík --- Software Engineer Red Hat ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On 9/20/18 1:56 PM, Matthew Miller wrote: > If it's "they'll find out when dnf system-upgrade errors out!", then see > above. I'm ... not enthused. Something in dnf system-upgrade needs to do it; > possibly a "dnf system-upgrade prep" step before "download". I agree. Would it be feasible for the system-upgrade plugin to prompt the user with "hey, you are using 1.7 but you need to switch to 1.8 to upgrade. Proceed? (y/N)". signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Thu, Sep 20, 2018 at 04:20:49PM +0200, Adam Samalik wrote: > > > And when that module is EOL, what is the user experience? > > What I described in my reply earlier: the upgrade should not work and > > the user should be required to switch to a new stream on their current > > environment first. Which, of course, implies that we need a policy > > requiring overlap. > > Or a way of specifying a target stream during the update. One possibility would be to add "--upgrade-obsolete-modules". But I'm not liking that because we get into a circus of: 1. Trying dnf-upgrade download 2. Get errors 3. Add --allowerasing --best 4. Try again 5. Get errors 6. Seriously though, remove some i386 packages 7. Try again 8. Oh, now my modules are failing. 9. Add another option. 10. Try again? I mean, at this rate, we're never going to get to ???, let alone "profit!" -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Thu, Sep 20, 2018 at 09:35:41AM -0400, Stephen Gallagher wrote: > > > Independently of this, you could also retire 1.7 with the end of F27 if > > > there was no need to have it in the future releases. > > Is there a way for users to say "keep me on whatever module is the default" > > when upgrading? > That would be a violation of the design principle that users should > never have their module stream change underneath them without their > consent. That's fine and all, but I don't want to violate a different principle: release-to-release updates on Fedora should be painless. Modularity is supposed to make that better. The no-switch-without-consent design principle is fine, but in order for these two things to both be true, we need a better UX than erroring out and hoping the user knows what to do. > Now, it gets a little trickier when we talk about upgrades from a > release that supports 1.7 to one that no longer does... in that case I > believe our expected behavior is that the user is expected to switch > to the newer stream *before* initiating the upgrade process. So they'd > do: > > ``` > dnf module enable foomodule:1.8 > dnf distro-sync > dnf system-upgrade --releasever=nextversion download > ``` How do they know they need to do the former? If it's "they'll find out when dnf system-upgrade errors out!", then see above. I'm ... not enthused. Something in dnf system-upgrade needs to do it; possibly a "dnf system-upgrade prep" step before "download". Also, since we're enabling modules on Workstation, what happens in GNOME Software? -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Thu, Sep 20, 2018 at 9:12 AM Matthew Miller wrote: > > On Tue, Sep 18, 2018 at 10:48:46AM +0200, Adam Samalik wrote: > > There is another concept in Modularity we can use here: defaults. You could > > have both streams 1.7 and 1.8 built for all active (non-EOL) Fedora > > releases, but have different default for each. So in your case, F27 could > > have 1.7 as a default and F28+ could have 1.8 as a default. > > > > Independently of this, you could also retire 1.7 with the end of F27 if > > there was no need to have it in the future releases. > > Is there a way for users to say "keep me on whatever module is the default" > when upgrading? That would be a violation of the design principle that users should never have their module stream change underneath them without their consent. Now, it gets a little trickier when we talk about upgrades from a release that supports 1.7 to one that no longer does... in that case I believe our expected behavior is that the user is expected to switch to the newer stream *before* initiating the upgrade process. So they'd do: ``` dnf module enable foomodule:1.8 dnf distro-sync dnf system-upgrade --releasever=nextversion download ``` ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Thu, Sep 20, 2018 at 4:17 PM Stephen Gallagher wrote: > On Thu, Sep 20, 2018 at 10:16 AM Matthew Miller > wrote: > > > > On Thu, Sep 20, 2018 at 03:58:20PM +0200, Petr Šabata wrote: > > > > Is there a way for users to say "keep me on whatever module is the > default" > > > > when upgrading? > > > If they enable the module explicitly, they will keep that stream, > > > regardless of what the current defaults are. > > > > And when that module is EOL, what is the user experience? > > What I described in my reply earlier: the upgrade should not work and > the user should be required to switch to a new stream on their current > environment first. Which, of course, implies that we need a policy > requiring overlap. > Or a way of specifying a target stream during the update. > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > -- Adam Šamalík --- Software Engineer Red Hat ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Thu, Sep 20, 2018 at 10:16 AM Matthew Miller wrote: > > On Thu, Sep 20, 2018 at 03:58:20PM +0200, Petr Šabata wrote: > > > Is there a way for users to say "keep me on whatever module is the > > > default" > > > when upgrading? > > If they enable the module explicitly, they will keep that stream, > > regardless of what the current defaults are. > > And when that module is EOL, what is the user experience? What I described in my reply earlier: the upgrade should not work and the user should be required to switch to a new stream on their current environment first. Which, of course, implies that we need a policy requiring overlap. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Thu, Sep 20, 2018 at 03:58:20PM +0200, Petr Šabata wrote: > > Is there a way for users to say "keep me on whatever module is the default" > > when upgrading? > If they enable the module explicitly, they will keep that stream, > regardless of what the current defaults are. And when that module is EOL, what is the user experience? -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Thu, Sep 20, 2018 at 09:11:57AM -0400, Matthew Miller wrote: > On Tue, Sep 18, 2018 at 10:48:46AM +0200, Adam Samalik wrote: > > There is another concept in Modularity we can use here: defaults. You could > > have both streams 1.7 and 1.8 built for all active (non-EOL) Fedora > > releases, but have different default for each. So in your case, F27 could > > have 1.7 as a default and F28+ could have 1.8 as a default. > > > > Independently of this, you could also retire 1.7 with the end of F27 if > > there was no need to have it in the future releases. > > Is there a way for users to say "keep me on whatever module is the default" > when upgrading? If they enable the module explicitly, they will keep that stream, regardless of what the current defaults are. Explicit enablement can happen in two ways: 1. By running `dnf module enable name:stream`, or 2. by installing any package from that module. P signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Tue, Sep 18, 2018 at 10:48:46AM +0200, Adam Samalik wrote: > There is another concept in Modularity we can use here: defaults. You could > have both streams 1.7 and 1.8 built for all active (non-EOL) Fedora > releases, but have different default for each. So in your case, F27 could > have 1.7 as a default and F28+ could have 1.8 as a default. > > Independently of this, you could also retire 1.7 with the end of F27 if > there was no need to have it in the future releases. Is there a way for users to say "keep me on whatever module is the default" when upgrading? -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Mon, Sep 17, 2018 at 12:20 PM Petr Šabata wrote: > On Mon, Sep 03, 2018 at 03:35:50PM +0200, Adam Samalik wrote: > > This is a summary of a recent thread [1]. > > > > Traditional branches (such as "f29") have their EOL (end of life) encoded > > in the name. But what about stream branches [2] (such as "2.4" or > "latest")? > > > > Stream branches of RPM packages would always have an EOL associated with > > them. The format would be on of the following: > > a) A date — mostly tied to an upstream version and its EOL. > > b) A specific Fedora release — for release-specific packages. > > c) Forever — rolling forward with upstream, latest development versions, > > etc. > > > > Module streams would inherit their EOL from the packages they include — > the > > earliest EOL would win. This could be optionally overridden on the module > > stream level by specifying one of the following: > > a) A date. > > b) A specific Fedora release. > > > > There would be a policy that a module can reach its EOL in the middle of > a > > Fedora release to prevent madness. > > > > We need a way to specify the EOL value and to manage it over time, > because > > it might change. For RPM stream branches, there is currently a way to > > specify an EOL value when requesting the branch [3] — the actual format > > might change based on this discussion. However, I'm not aware of a way to > > change the value if necessary nor a process associated with that. Also, > > there is currently no way to specify an EOL for modules. > > > > After we figure this out, we also need to make sure the build system > takes > > that into account (some recent progress [4]) and that the client tooling > > (mostly DNF) presents that to the user. > > > > So... any comments to the concept? Any ideas about workflows or processes > > of managing the EOL values? > > I went through both threads and thought I'd offer my point > of view. > > From experience I can tell that defining the EOL, be it a module > or a package, is rather difficult. Even in the rare case where > upstream is clear about their support plans, it doesn't mean we > have to follow that. We might want to drop the package earlier, > or the opposite -- support it on our own long after it was > abandoned upstream. I think both cases are somewhat common. > And we rarely, if ever, know this information at the branch > creation time. > That's exactly what I was worrying about. > > So, a few things. > > If we need to define these for something, I'd rather only do > it for modules rather than packages. Not only to simplify > the process for everyone but also because I kinda feel that > the module maintainer is responsible for the packages they > include. I have a hard time imagining packagers maintaining > stream branches "just in case someone wants to consume them". > They either maintain the module or only care about the release > branches. Also if you have a module with 200 components and you > derive your module's EOL from the packages' EOLs, you need to be > constantly watching all your components and their "arbitrary" > EOL dates or risk your module being dropped. I don't think > this is very user friendly. > This is an interesting idea and I think I like it. I'll try to rephrase just to confirm I got it right: You are proposing, that instead of having package stream branch owners, we would only have module owners, who would maintain the packages they build. That probably means: 1. Only modules have an EOL. 2. You maintain what you build — if multiple modules build a single package, they would collaborate on maintaining it. 3. There is no point in having a package stream branch-level EOL — these packages are not getting built outside of modules. And module maintainers should maintain them if they build them. 4. We might not need to retire package stream branches — again, these packages are note getting built outside of modules. And module maintainers should maintain them if they build them. Really interesting idea! Might require a few policy changes. > I think the rolling model would be the most commonly chosen > option. Basically "supported until I decide to kill it in > Rawhide". We could then update existing stable modules with a > more specific date, such as the date when that release goes EOL. > Maintainers should still be able to choose a date ahead of > time if they wish but I'd rather not tie it to Fedora release > numbers as that doesn't work very well for EPEL. > That's my favourite, yes, although I can see cases for the other two as well, even though they probably won't be used as often. > > Finally, I also don't see a point in really killing package > stream branches. If someone is consuming these, they are > responsible. If not, it doesn't matter that they exist. > I tend to agree, even though we would need a few changes in the policies around ownership I've mentioned above. > > Not sure how much sense this makes. > > P > > > [1] Previous thread: > > > https://lis
Re: Managing stream (arbitrary) branch and module lifecycles
On Wed, Sep 12, 2018 at 10:19 PM Matthew Miller wrote: > On Mon, Sep 03, 2018 at 03:35:50PM +0200, Adam Samalik wrote: > > There would be a policy that a module can reach its EOL in the middle of > a > > Fedora release to prevent madness. > > Can or can't? I assume you mean "can't", because "can" doesn't sound like > preventing madness. :) > Oops, you're absolutely right. :) > > > > -- > Matthew Miller > > Fedora Project Leader > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > -- Adam Šamalík --- Software Engineer Red Hat ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Wed, Sep 12, 2018 at 10:54 AM Dominik 'Rathann' Mierzejewski < domi...@greysector.net> wrote: > On Tuesday, 11 September 2018 at 21:43, Richard Shaw wrote: > [...] > > This would take care of most of the complains about people using "git > merge > > master" across release branches (even though that's the workflow > documented > > in the wiki). I know I CAN use git cherry-pick but I've never used it > > before and again, I'm not a program. Almost everything I've learned about > > git is through Fedora package maintenance and some small pull requests > for > > minor build fixes with packages I maintain. > > > > The hard part for me is maintaining EL 6/7 branches. I know there are a > lot > > of complaints about using %if conditionals in specs to have one spec file > > for all Fedora and EPEL releases and I agree when it gets to be too much > it > > makes the spec very unreadable especially by others (proven packagers) > that > > may have to step in and make changes. If stream branches could somehow > make > > this easier that would be great. > > I found the "merging upwards" workflow from gitworkflows(7) quite useful > for > Fedora for leaf packages that can be updated across the board while > retaining some small differences between branches like changelogs. > I still feel like having a single spec that builds everywhere, with a few %if conditionals should be easier to maintain than a multiple different ones. But as I said in the previous reply, if there is a really big difference between let's say Fedora and EPEL, or EPEL 6 vs. EPEL 7, we might need to think about splitting these into separate branches, but still having it appear as a single stream from the user perspective. > > Branches remain mergeable and you can easily remove any conditionals and > keep cruft limited to older branches. > > Regards, > Dominik > -- > Fedora https://getfedora.org | RPMFusion http://rpmfusion.org > There should be a science of discontent. People need hard times and > oppression to develop psychic muscles. > -- from "Collected Sayings of Muad'Dib" by the Princess Irulan > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > -- Adam Šamalík --- Software Engineer Red Hat ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Tue, Sep 11, 2018 at 10:08 PM Richard Shaw wrote: > On Tue, Sep 11, 2018 at 2:30 PM Adam Samalik wrote: > >> On Wed, Sep 5, 2018 at 3:43 PM Richard Shaw wrote: >> As a packager, what is your experience with lifecycles of your packages? >> Do you get a specific EOL date from the upstream? If not, at what >> circumstances you would retire an old version of a package? How do you >> decide when to go for a new version if there is any? >> > > I don't have too many upstreams that "sunset" a particular version > (usually the Fedora release schedule stays ahead of that) but I know others > do. I really have just two main workflows: > > 1. End user program which I keep up to date on all non-EOL Fedora releases. > 2. Libraries which I try to not make API/ABI breaking changes within a > release (sometimes changes in dependencies make this difficult). > > I see stream branches helping with both... > > In the case of #1, I would have "one branch to rule them all" instead of > having a branch per release. (also helps solve the first issue below) > Yes, that was one of the main reasons. Glad you like it. > > In the case of #2, I would have a branch per major or minor release > (whatever upstream maintains as non-ABI breaking). For instance, > OpenImageIO I have both 1.7.x and 1.8.x releases being maintained. I would > just have a branch for each. When F27 goes EOL that will be the last Fedora > release on 1.7.x so that stream branch would be able to go EOL but 1.9/2.0 > is going to be released soon so I would request a new stream branch for it > and only build/rebuild dependencies for Rawhide (unless there was a > compelling reason to include it in F29). > There is another concept in Modularity we can use here: defaults. You could have both streams 1.7 and 1.8 built for all active (non-EOL) Fedora releases, but have different default for each. So in your case, F27 could have 1.7 as a default and F28+ could have 1.8 as a default. Independently of this, you could also retire 1.7 with the end of F27 if there was no need to have it in the future releases. > > This would take care of most of the complains about people using "git > merge master" across release branches (even though that's the workflow > documented in the wiki). I know I CAN use git cherry-pick but I've never > used it before and again, I'm not a program. Almost everything I've learned > about git is through Fedora package maintenance and some small pull > requests for minor build fixes with packages I maintain. > > The hard part for me is maintaining EL 6/7 branches. I know there are a > lot of complaints about using %if conditionals in specs to have one spec > file for all Fedora and EPEL releases and I agree when it gets to be too > much it makes the spec very unreadable especially by others (proven > packagers) that may have to step in and make changes. If stream branches > could somehow make this easier that would be great. > Right now the idea is to have one branch with a particular version for all releases, therefore some %if conditionals in specs could be necessary. On the other hand, there would be just a single spec that builds everywhere. However, we might think about having more branches for one version, if there is a need to have a different source for let's say Fedora and EPEL. > > > I'm okay with having to create some sort of "control" file to say what >>> Fedora releases should be built from a stream branch but there needs to be >>> an easy template either cut and pasted from the wiki or that one of the >>> packaging tools can produce. >>> >> >> We'll do our best. :-) >> > > Thanks! > Richard > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > -- Adam Šamalík --- Software Engineer Red Hat ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Mon, Sep 03, 2018 at 03:35:50PM +0200, Adam Samalik wrote: > This is a summary of a recent thread [1]. > > Traditional branches (such as "f29") have their EOL (end of life) encoded > in the name. But what about stream branches [2] (such as "2.4" or "latest")? > > Stream branches of RPM packages would always have an EOL associated with > them. The format would be on of the following: > a) A date — mostly tied to an upstream version and its EOL. > b) A specific Fedora release — for release-specific packages. > c) Forever — rolling forward with upstream, latest development versions, > etc. > > Module streams would inherit their EOL from the packages they include — the > earliest EOL would win. This could be optionally overridden on the module > stream level by specifying one of the following: > a) A date. > b) A specific Fedora release. > > There would be a policy that a module can reach its EOL in the middle of a > Fedora release to prevent madness. > > We need a way to specify the EOL value and to manage it over time, because > it might change. For RPM stream branches, there is currently a way to > specify an EOL value when requesting the branch [3] — the actual format > might change based on this discussion. However, I'm not aware of a way to > change the value if necessary nor a process associated with that. Also, > there is currently no way to specify an EOL for modules. > > After we figure this out, we also need to make sure the build system takes > that into account (some recent progress [4]) and that the client tooling > (mostly DNF) presents that to the user. > > So... any comments to the concept? Any ideas about workflows or processes > of managing the EOL values? I went through both threads and thought I'd offer my point of view. From experience I can tell that defining the EOL, be it a module or a package, is rather difficult. Even in the rare case where upstream is clear about their support plans, it doesn't mean we have to follow that. We might want to drop the package earlier, or the opposite -- support it on our own long after it was abandoned upstream. I think both cases are somewhat common. And we rarely, if ever, know this information at the branch creation time. So, a few things. If we need to define these for something, I'd rather only do it for modules rather than packages. Not only to simplify the process for everyone but also because I kinda feel that the module maintainer is responsible for the packages they include. I have a hard time imagining packagers maintaining stream branches "just in case someone wants to consume them". They either maintain the module or only care about the release branches. Also if you have a module with 200 components and you derive your module's EOL from the packages' EOLs, you need to be constantly watching all your components and their "arbitrary" EOL dates or risk your module being dropped. I don't think this is very user friendly. I think the rolling model would be the most commonly chosen option. Basically "supported until I decide to kill it in Rawhide". We could then update existing stable modules with a more specific date, such as the date when that release goes EOL. Maintainers should still be able to choose a date ahead of time if they wish but I'd rather not tie it to Fedora release numbers as that doesn't work very well for EPEL. Finally, I also don't see a point in really killing package stream branches. If someone is consuming these, they are responsible. If not, it doesn't matter that they exist. Not sure how much sense this makes. P > [1] Previous thread: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/K4FUOQHQSRAAI3PUUGXAC6CXEN27Y2JH/ > [2] Now "stream branches", formerly "arbitrary branches": > https://fedoraproject.org/wiki/Changes/ArbitraryBranching > [3] Requesting a stream branch + specifying its EOL: > https://docs.fedoraproject.org/en-US/modularity/making-modules/adding-new-modules/#_repositories_and_stream_branches_existing_packages > [4] https://pagure.io/modularity/issue/102 > > -- > > Adam Šamalík > --- > Software Engineer > Red Hat > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archive
Re: Managing stream (arbitrary) branch and module lifecycles
On Mon, Sep 03, 2018 at 03:35:50PM +0200, Adam Samalik wrote: > There would be a policy that a module can reach its EOL in the middle of a > Fedora release to prevent madness. Can or can't? I assume you mean "can't", because "can" doesn't sound like preventing madness. :) -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Tuesday, 11 September 2018 at 21:43, Richard Shaw wrote: [...] > This would take care of most of the complains about people using "git merge > master" across release branches (even though that's the workflow documented > in the wiki). I know I CAN use git cherry-pick but I've never used it > before and again, I'm not a program. Almost everything I've learned about > git is through Fedora package maintenance and some small pull requests for > minor build fixes with packages I maintain. > > The hard part for me is maintaining EL 6/7 branches. I know there are a lot > of complaints about using %if conditionals in specs to have one spec file > for all Fedora and EPEL releases and I agree when it gets to be too much it > makes the spec very unreadable especially by others (proven packagers) that > may have to step in and make changes. If stream branches could somehow make > this easier that would be great. I found the "merging upwards" workflow from gitworkflows(7) quite useful for Fedora for leaf packages that can be updated across the board while retaining some small differences between branches like changelogs. Branches remain mergeable and you can easily remove any conditionals and keep cruft limited to older branches. Regards, Dominik -- Fedora https://getfedora.org | RPMFusion http://rpmfusion.org There should be a science of discontent. People need hard times and oppression to develop psychic muscles. -- from "Collected Sayings of Muad'Dib" by the Princess Irulan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Tue, Sep 11, 2018 at 2:30 PM Adam Samalik wrote: > On Wed, Sep 5, 2018 at 3:43 PM Richard Shaw wrote: > As a packager, what is your experience with lifecycles of your packages? > Do you get a specific EOL date from the upstream? If not, at what > circumstances you would retire an old version of a package? How do you > decide when to go for a new version if there is any? > I don't have too many upstreams that "sunset" a particular version (usually the Fedora release schedule stays ahead of that) but I know others do. I really have just two main workflows: 1. End user program which I keep up to date on all non-EOL Fedora releases. 2. Libraries which I try to not make API/ABI breaking changes within a release (sometimes changes in dependencies make this difficult). I see stream branches helping with both... In the case of #1, I would have "one branch to rule them all" instead of having a branch per release. (also helps solve the first issue below) In the case of #2, I would have a branch per major or minor release (whatever upstream maintains as non-ABI breaking). For instance, OpenImageIO I have both 1.7.x and 1.8.x releases being maintained. I would just have a branch for each. When F27 goes EOL that will be the last Fedora release on 1.7.x so that stream branch would be able to go EOL but 1.9/2.0 is going to be released soon so I would request a new stream branch for it and only build/rebuild dependencies for Rawhide (unless there was a compelling reason to include it in F29). This would take care of most of the complains about people using "git merge master" across release branches (even though that's the workflow documented in the wiki). I know I CAN use git cherry-pick but I've never used it before and again, I'm not a program. Almost everything I've learned about git is through Fedora package maintenance and some small pull requests for minor build fixes with packages I maintain. The hard part for me is maintaining EL 6/7 branches. I know there are a lot of complaints about using %if conditionals in specs to have one spec file for all Fedora and EPEL releases and I agree when it gets to be too much it makes the spec very unreadable especially by others (proven packagers) that may have to step in and make changes. If stream branches could somehow make this easier that would be great. I'm okay with having to create some sort of "control" file to say what >> Fedora releases should be built from a stream branch but there needs to be >> an easy template either cut and pasted from the wiki or that one of the >> packaging tools can produce. >> > > We'll do our best. :-) > Thanks! Richard ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Wed, Sep 5, 2018 at 3:43 PM Richard Shaw wrote: > On Mon, Sep 3, 2018 at 8:36 AM Adam Samalik wrote: > >> >> So... any comments to the concept? Any ideas about workflows or processes >> of managing the EOL values? >> > > Looking forward to this but I would say the devil is in the details. > Packagers are not necessarily programmers (I include myself in this camp) > although I've learned to fix lots of build issues. Whatever the end result > is, it needs to be well documented (wiki?) and supported by the tools. > It always is. :-) As a packager, what is your experience with lifecycles of your packages? Do you get a specific EOL date from the upstream? If not, at what circumstances you would retire an old version of a package? How do you decide when to go for a new version if there is any? And don't worry, I'll make sure it's well documented. > I'm okay with having to create some sort of "control" file to say what > Fedora releases should be built from a stream branch but there needs to be > an easy template either cut and pasted from the wiki or that one of the > packaging tools can produce. > We'll do our best. :-) > > Thanks, > Richard > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > -- Adam Šamalík --- Software Engineer Red Hat ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Managing stream (arbitrary) branch and module lifecycles
On Mon, Sep 3, 2018 at 8:36 AM Adam Samalik wrote: > > So... any comments to the concept? Any ideas about workflows or processes > of managing the EOL values? > Looking forward to this but I would say the devil is in the details. Packagers are not necessarily programmers (I include myself in this camp) although I've learned to fix lots of build issues. Whatever the end result is, it needs to be well documented (wiki?) and supported by the tools. I'm okay with having to create some sort of "control" file to say what Fedora releases should be built from a stream branch but there needs to be an easy template either cut and pasted from the wiki or that one of the packaging tools can produce. Thanks, Richard ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Managing stream (arbitrary) branch and module lifecycles
This is a summary of a recent thread [1]. Traditional branches (such as "f29") have their EOL (end of life) encoded in the name. But what about stream branches [2] (such as "2.4" or "latest")? Stream branches of RPM packages would always have an EOL associated with them. The format would be on of the following: a) A date — mostly tied to an upstream version and its EOL. b) A specific Fedora release — for release-specific packages. c) Forever — rolling forward with upstream, latest development versions, etc. Module streams would inherit their EOL from the packages they include — the earliest EOL would win. This could be optionally overridden on the module stream level by specifying one of the following: a) A date. b) A specific Fedora release. There would be a policy that a module can reach its EOL in the middle of a Fedora release to prevent madness. We need a way to specify the EOL value and to manage it over time, because it might change. For RPM stream branches, there is currently a way to specify an EOL value when requesting the branch [3] — the actual format might change based on this discussion. However, I'm not aware of a way to change the value if necessary nor a process associated with that. Also, there is currently no way to specify an EOL for modules. After we figure this out, we also need to make sure the build system takes that into account (some recent progress [4]) and that the client tooling (mostly DNF) presents that to the user. So... any comments to the concept? Any ideas about workflows or processes of managing the EOL values? [1] Previous thread: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/K4FUOQHQSRAAI3PUUGXAC6CXEN27Y2JH/ [2] Now "stream branches", formerly "arbitrary branches": https://fedoraproject.org/wiki/Changes/ArbitraryBranching [3] Requesting a stream branch + specifying its EOL: https://docs.fedoraproject.org/en-US/modularity/making-modules/adding-new-modules/#_repositories_and_stream_branches_existing_packages [4] https://pagure.io/modularity/issue/102 -- Adam Šamalík --- Software Engineer Red Hat ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
