Re: Contribute or create a channel?
Hi Attila, Attila Lendvai writes: >> > channels are a step towards this, but they are not enough in their >> > current form to successfully accommodate for such a setup. an obvious >> > thing that is missing is a way to formally express inter-channel >> > dependencies, including some form of versioning. >> >> >> Do we not have this? The manual documents a mechanism for channel >> dependencies in "(guix) Declaring Channel Dependencies". >> >> I haven't used it, but it looks like the dependencies are declared as >> channels, which can have the usual branch/commit specifications to tie >> them to specific versions. > > good point, thanks! > > i looked briefly at the code just now. it's not trivial, and it seems to > treat the guix channel specially (because i don't need to specify it as a > dependency in my channel's .guix-channel file), and i'm not sure how it > behaves when e.g. two channels depend on the same channel, but pick two > different commits... or all the other convoluted situations. The situation you describe where two channels pick different commits for a channel dependency *cannot* be made to work, at least using Guile. I think what would currently happen is that either: 1) Guix refuses to build (unlikely); 2) Guix just chooses one commit that it then uses to build all dependent channels, contrary to the requirements; 3) Guix does build each channel with the proper dependencies, but the final Guix will have profile collisions. Best, -- Josselin Poiret signature.asc Description: PGP signature
Re: Contribute or create a channel?
> > channels are a step towards this, but they are not enough in their > > current form to successfully accommodate for such a setup. an obvious > > thing that is missing is a way to formally express inter-channel > > dependencies, including some form of versioning. > > > Do we not have this? The manual documents a mechanism for channel > dependencies in "(guix) Declaring Channel Dependencies". > > I haven't used it, but it looks like the dependencies are declared as > channels, which can have the usual branch/commit specifications to tie > them to specific versions. good point, thanks! i looked briefly at the code just now. it's not trivial, and it seems to treat the guix channel specially (because i don't need to specify it as a dependency in my channel's .guix-channel file), and i'm not sure how it behaves when e.g. two channels depend on the same channel, but pick two different commits... or all the other convoluted situations. the reason i assumed it doesn't exist is that i've never seen it used by any channels that i looked at. > What are we missing? i guess it's time to experiment to be able to answer your question. FTR, it's READ-CHANNEL-METADATA and friends in guix/channels.scm note that it's not the same thing as /etc/guix/channels.scm, even though they appear similar (https://issues.guix.gnu.org/53657). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “People who have never gone to school have never developed negative attitudes toward exploring their world.” — Grace Llewellyn
Re: Contribute or create a channel?
On Tue, Mar 12 2024, Attila Lendvai wrote: > channels are a step towards this, but they are not enough in their > current form to successfully accommodate for such a setup. an obvious > thing that is missing is a way to formally express inter-channel > dependencies, including some form of versioning. Do we not have this? The manual documents a mechanism for channel dependencies in "(guix) Declaring Channel Dependencies". I haven't used it, but it looks like the dependencies are declared as channels, which can have the usual branch/commit specifications to tie them to specific versions. What are we missing? Carlo
Re: Contribute or create a channel?
Hi Attila, I know this wasn't directly sent to me, but I wanted to share a couple of thoughts anyway :-) On 2024-03-12 14:18, Attila Lendvai wrote: > my gut feeling is that new abstractions are needed that would enable > splitting the monorepo/community into less tightly coupled subgroups where > they can have their own coding standards, repos, channels, etc, and a more > federated way to maintain/integrate all the software that exists out there > into a guix system. As an end user, it is incredibly nice to have a single Guix channel to pull from. I just have to remember (or write down...) a single commit hash and that's all that's required to reproduce my setup. > channels are a step towards this, but they are not enough in their current > form to successfully accommodate for such a setup. an obvious thing that is > missing is a way to formally express inter-channel dependencies, including > some form of versioning. Although I understand where you are coming from, I am not convinced that this solves more problems than it causes. For example, a naive approach to inter-channel dependencies with versioning could reintroduce the diamond dependency problem. However, solving this by locking every single transitive dependent channel would not allow security updates across all of them simultaneously anymore. > in this hypothetical setup commit rights could be issued much more liberally > to non-core sub-repos, and more rigorous code reviews would only need to be > done when a new version of the split-out part is being incorporated back into > a new revision of the core/bootstrap chain (if e.g. assuming python is needed > for the bootstrap of the core, then the python subgroup's stuff would only > need core review when a new version of that is pointed to by the core). The same topic has come up in Nix as well [1, 2, 3], but they are using Github, so they are bound to a centralized workflow. Using Github (or any such git forge for that matter) reintroduces some of the same problems Git was designed to solve in the first place, which become most painful at scale [4]. From a technical point of view, I don't see why we couldn't split into less tightly coupled subgroups with their own repos, standards and CI merging into a single Guix "monorepo". I think there is still a lot to learn from how the kernel does its development and we could extrapolate to something that makes sense for Guix. Best wishes, Troy [1] https://discourse.nixos.org/t/wild-idea-how-about-splitting-nixpkgs-and-nixos/11487/25 [2] https://github.com/NixOS/rfcs/pull/70 [3] https://github.com/NixOS/nixpkgs/issues/50105 [4] https://blog.ffwll.ch/2017/08/github-why-cant-host-the-kernel.html OpenPGP_0xC67C9181B3893FB0.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature
Re: Contribute or create a channel?
> > the patch inflow to the guix repo is currently overwhelming the > > available capacity for review and pushing. > > > With an email like the one sent by Hartmut we can better arrange for > shepherding this large submission. (Nothing is to be gained from > repeatedly bemoaning well-known issues in the patch review processes > here and in other threads on the mailing list.) i was reflecting on why i wrote this, and what i wanted to express is that i think guix has reached a point where a monorepo is becoming a net negative, and i don't see this being discussed. my gut feeling is that new abstractions are needed that would enable splitting the monorepo/community into less tightly coupled subgroups where they can have their own coding standards, repos, channels, etc, and a more federated way to maintain/integrate all the software that exists out there into a guix system. in this hypothetical setup commit rights could be issued much more liberally to non-core sub-repos, and more rigorous code reviews would only need to be done when a new version of the split-out part is being incorporated back into a new revision of the core/bootstrap chain (if e.g. assuming python is needed for the bootstrap of the core, then the python subgroup's stuff would only need core review when a new version of that is pointed to by the core). or alternatively, simply try to split guix into a minimal core that is essential for the bootstrap, and everything else into multiple subchannels (gnome, gui stuff in general, random apps, etc). i have no impression how much that alone could shrink the monorepo part, though. channels are a step towards this, but they are not enough in their current form to successfully accommodate for such a setup. an obvious thing that is missing is a way to formally express inter-channel dependencies, including some form of versioning. sadly, i don't have any proposals beyond discussing the observable issue (i.e. the insufficient patch throughput). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Values in a free society are accepted voluntarily, not through coercion, and certainly not by law… every time we write a law to control private behavior, we imply that somebody has to arrive with a gun [to enforce it].” — Ron Paul
Re: Contribute or create a channel?
Hartmut Goebel writes: > In both cases I need some tooling to fetch the current bug-fix version > of the series in question. This can not be done using "guix refresh" > only AFAIU, as this would use the next release-series if this is > already released while our packages are not yet updated. "guix refresh" can be modified, and more specific importers / updaters can be added to better suit the needs of groups of packages. -- Ricardo
Re: Contribute or create a channel?
Am 04.03.24 um 09:32 schrieb Andreas Enge: I think that would be okay if you think it will be easier to maintain (not needing to "roll over" code from an old package at the inheritance root when it is deleted), assuming that the old packages are removed from time to time. This sounds like maintaining multiple LTS versions is desired. Anyhow Ricardo wrote: I think it would be preferable to have one LTS version in Guix. Thus the discussion about maintaining several versions is only relevant if this is what Guix wants. In both cases I need some tooling to fetch the current bug-fix version of the series in question. This can not be done using "guix refresh" only AFAIU, as this would use the next release-series if this is already released while our packages are not yet updated. Thus maintaining two LTS versions should not be too much work (except if many dependencies change incompatible). WDYT? -- Regards Hartmut Goebel | Hartmut Goebel | h.goe...@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible |
Re: Contribute or create a channel?
Am Sat, Mar 02, 2024 at 11:32:37AM +0100 schrieb Hartmut Goebel: > Maybe using one file per release (and accept duplicate code) would be a > suitable workaround. I think that would be okay if you think it will be easier to maintain (not needing to "roll over" code from an old package at the inheritance root when it is deleted), assuming that the old packages are removed from time to time. Andreas
Re: Contribute or create a channel?
Am 01.03.24 um 19:16 schrieb Saku Laesvuori: It just requires a different updating method. The different versions can just be defined as separate packages (see postgresql for an example) and the user the defines which one they want to use. They can either refer to the package variable directly in scheme (e.g. postgresql-15) or on the command line with the name@version syntax (e.g. postgresql@15). I'm aware of this way. It's fisable when there is a small number of packages like for postgres. But it is very laborious for rust packages already. I would not want to do the same for more than 200 packages. -- Regards Hartmut Goebel | Hartmut Goebel | h.goe...@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible |
Re: Contribute or create a channel?
Hi Ricardo, I think it would be preferable to have one LTS version in Guix. Okay, that's a clear statement. whether to grant you commit rights to handle these upgrades by yourself. Well, I already have commit rights :-) Anyhow the review period is a bit hindering, esp. for bug-fixes. OTOH in Guix, maintaining several version seems laborious. What makes you say this? What exactly is the obstacle here? The only way I know for handling several versions of the same package is what Saku Laesvuori described: Adding a new variable for each of these versions, possibly making the older version inherit from the new one. This means moving quite some code on every new version. Done this for quite some rust-packages and it i quite laborious and hard to automate. And for Tryton we have about 200 packages and growing. When using branches (in a channel) one would just checkout that branch, automatically apply the patches and commit. If the version is no longer supported, simply stop applying updates on that branch. Maybe using one file per release (and accept duplicate code) would be a suitable workaround. -- Regards Hartmut Goebel | Hartmut Goebel | h.goe...@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible |
Re: Contribute or create a channel?
Hi Hartmut, > I'm currently updating Tryton to version 7.0 and am wondering whether it's > better to contribute the change to Guix or to set up a > channel for Tryton. > > WDYT? I'm eager to learn about your thoughts. > > Here is why I'm wondering: > > * Tryton consists of a client, a server and about 200 module/add-on providing > business logic. > * Tryton publishes a LTS version every 2.5 years. Two LTS versions are > supported (currently 6.0 and 7.0) and bugfixes are backported > there for 5 years. I think it would be preferable to have one LTS version in Guix. If you want to commit to maintaining the packages I think it would be worth discussing whether to grant you commit rights to handle these upgrades by yourself. > OTOH in Guix, maintaining several version seems laborious. What makes you say this? What exactly is the obstacle here? > Anyhow I'm unsure whether it's worth the effort maintaining three versions > and whether I'll be able to keep three version up to date - > esp. given that I don't have much automation for this. Personally, I don't think it makes sense to commit to maintaining there separate versions. It seems like a really big committment. -- Ricardo
Re: Contribute or create a channel?
Attila Lendvai writes: >> WDYT? I'm eager to learn about your thoughts. > > the patch inflow to the guix repo is currently overwhelming the > available capacity for review and pushing. With an email like the one sent by Hartmut we can better arrange for shepherding this large submission. (Nothing is to be gained from repeatedly bemoaning well-known issues in the patch review processes here and in other threads on the mailing list.) -- Ricardo
Re: Contribute or create a channel?
> WDYT? I'm eager to learn about your thoughts. the patch inflow to the guix repo is currently overwhelming the available capacity for review and pushing. if you want an agile experience, i.e. where you can quickly fix/update this and that, then i suggest your own channel (unless you have the commit bit for the guix repo... or there's a committed maintainer who is a regular user, and as such will fast-track your patches). otherwise you'll end up using a channel anyway (i.e. your fork of the guix repo while your patches are waiting in the queue to be reviewed and pushed). PS: i don't mean to sound cynical here, just matter-of-factly. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Opportunity is missed by most people because it comes dressed in overalls and looks like work.” — Thomas A. Edison (1847–1931)
Re: Contribute or create a channel?
> I'm currently updating Tryton to version 7.0 and am wondering whether > it's better to contribute the change to Guix or to set up a channel for > Tryton. As a general rule: it is always better to contribute a change instead of maintaining a separate channel for it if the change could be accepted in Guix. > * Bugfixes happens rather often and per-module, since they are > published even for smaller fixes. Upstream promises to not contain > functional changes or change requirements. Each bugfix could be > implemented as a graft, since . I don't think it would make much sense to implement bugfixes as grafts if the package isn't depended on by a huge number of other packages. > Given this, it might be interesting to have three versions of Tryton > available: the two LTS versions and the latest version. > > Now the idea is to provide a channel which provides a branch for each > LTS version and a "main" branch for the latest release. This would allow > to checkout the respective branch and refresh the packages of the > respective version semi-automatically. > > OTOH in Guix, maintaining several version seems laborious. It just requires a different updating method. The different versions can just be defined as separate packages (see postgresql for an example) and the user the defines which one they want to use. They can either refer to the package variable directly in scheme (e.g. postgresql-15) or on the command line with the name@version syntax (e.g. postgresql@15). > Some more background-info: > > * Within each version, there is guarantee that the database schema > will not be changed. Anyhow between versions the db schema might > change, requiring manual migration steps. This is the case with postgresql, too. (which is why I chose it as the example before) - Saku signature.asc Description: PGP signature
Contribute or create a channel?
Hi, I'm currently updating Tryton to version 7.0 and am wondering whether it's better to contribute the change to Guix or to set up a channel for Tryton. WDYT? I'm eager to learn about your thoughts. Here is why I'm wondering: * Tryton consists of a client, a server and about 200 module/add-on providing business logic. * Tryton publishes a LTS version every 2.5 years. Two LTS versions are supported (currently 6.0 and 7.0) and bugfixes are backported there for 5 years. * Every 6 month a new release is crafted (x.2, x.4, x.6, x,8) which will get bugfixes for1 year. Releases typically provide new modules (which is why updating is of interest) , might change inputs and might require database updates. * Bugfixes happens rather often and per-module, since they are published even for smaller fixes. Upstream promises to not contain functional changes or change requirements. Each bugfix could be implemented as a graft, since . Given this, it might be interesting to have three versions of Tryton available: the two LTS versions and the latest version. Now the idea is to provide a channel which provides a branch for each LTS version and a "main" branch for the latest release. This would allow to checkout the respective branch and refresh the packages of the respective version semi-automatically. OTOH in Guix, maintaining several version seems laborious. Anyhow I'm unsure whether it's worth the effort maintaining three versions and whether I'll be able to keep three version up to date - esp. given that I don't have much automation for this. Some more background-info: * Within each version, there is guarantee that the database schema will not be changed. Anyhow between versions the db schema might change, requiring manual migration steps. * Debian as of now provides packages for 6.0 only (7.0 was released ) -- Regards Hartmut Goebel | Hartmut Goebel |h.goe...@crazy-compilers.com| |www.crazy-compilers.com | compilers which you thought are impossible |