Re: Functional package interface
In the code there, the function propagate-packages will indeed try and
propagate every package through the graph node.
I looked at your code, it's comprehensive. I wasn't sure such a thing
was possible with input rewriting, although I think there's still a lot
of benefit to the functional approach, like this:
- some packages are present under different names (emacs has
emacs-minimal, emacs, emacs-pgtk, emacs-wide-int...) and that is not
handled by this function.
It should handle it pretty elegantly, since those packages inherit from
emacs, if you replace emacs they'll inherit from your replacement while
keeping their unique traits.
- propagating some packages that are used in build-systems will trigger
massive rebuilds (e.g. go, emacs, python, ocaml, texlive), you'll
probably have to be smart about what you want to avoid to propagate, but
it's great because guix can already hide a package to
package-inputs-rewriting using (hidden-package {package-name}). There are some
examples in the series.
No idea how, but I guess I should implement a #:mask-hidden? argument
to handle this. I have more basic issues for now, but that's a good
point.
- this has a cost. I haven't timed it yet, but the slowdown is
noticeable, though still quite usable.
I'm doing a couple stupid things, it's improved with the last few
commits, but there's still a long way to go. If there's a cost to
syntax rule abuse, you were probably feeling that.
- you of course have to get the packages into the function from a
profile. Here's it's done in RDE's code, but that should be quite
doable in Guix too.
Yeah, the implementation model is kinda strange. Having to put extra
code in every single module in the entire repo isn't a great
prerequisite for anything. It could be the point that prevents this
from manifesting. The mile long butt-ugly list of inputs that it needs
doesn't help.
Thanks for the tips. Work is still ongoing, but recursive propagation
now works when giving one package-set another as an input, so the
intended use-case is now possible.
- sc
Re: Functional package interface
Hi Nicolas Isn't what you're trying to do already in Guix? Have a look at package-inputs-rewrite right there : https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.html I want to have this but more powerful, I'd like to do something like define an operating system with all instances of a package replaced. Maybe something like that is already possible, but I haven't figured out a way. As far as I know, with package-input-rewriting you would still have to manually add it to all the services in your operating-system definition. Or just use module-set! and manage the environment state some other way. You also only have access to the input fields of packages, this is something parameterize packages tackles too, being able to add inputs that go anywhere in the package definition. Build phases for example, to enable common compile options across multiple packages more easily. I've also been working on general package propagation in RDE, it's still a work in progress, but it's not far from it : https://lists.sr.ht/~abcdw/rde-devel/patches/49956 I'll take a look. I knew there had to be more people working on this idea, but it's not always easy to find these projects. - sc
Re: Functional package interface
On 2024-04-16 01:17, spacecadet wrote:
> Hi Nicolas
>
>> Isn't what you're trying to do already in Guix? Have a look at
>> package-inputs-rewrite right there :
>> https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.html
>
> I want to have this but more powerful, I'd like to do something like
> define an operating system with all instances of a package replaced.
> Maybe something like that is already possible, but I haven't figured
> out a way.
>
> As far as I know, with package-input-rewriting you would still have to
> manually add it to all the services in your operating-system
> definition. Or just use module-set! and manage the environment state
> some other way.
>
> You also only have access to the input fields of packages, this is
> something parameterize packages tackles too, being able to add inputs
> that go anywhere in the package definition. Build phases for example,
> to enable common compile options across multiple packages more easily.
>
>> I've also been working on general package propagation in RDE, it's still
>> a work in progress, but it's not far from it :
>> https://lists.sr.ht/~abcdw/rde-devel/patches/49956
>
> I'll take a look. I knew there had to be more people working on this idea,
> but it's not always easy to find these projects.
In the code there, the function propagate-packages will indeed try and
propagate every package through the graph node.
Four tricky points though :
- propagating some packages that are used in build-systems will trigger
massive rebuilds (e.g. go, emacs, python, ocaml, texlive), you'll
probably have to be smart about what you want to avoid to propagate, but
it's great because guix can already hide a package to
package-inputs-rewriting using (hidden-package {package-name}). There are some
examples in the series.
- some packages are present under different names (emacs has
emacs-minimal, emacs, emacs-pgtk, emacs-wide-int...) and that is not
handled by this function.
- this has a cost. I haven't timed it yet, but the slowdown is
noticeable, though still quite usable.
- you of course have to get the packages into the function from a
profile. Here's it's done in RDE's code, but that should be quite
doable in Guix too.
>
> - sc
--
Best regards,
Nicolas Graves
Re: Functional package interface
Hi spacecadet! Isn't what you're trying to do already in Guix? Have a look at package-inputs-rewrite right there : https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.html I've also been working on general package propagation in RDE, it's still a work in progress, but it's not far from it : https://lists.sr.ht/~abcdw/rde-devel/patches/49956 Best regards, Nicolas On 2024-04-14 07:02, spacecadet wrote: > Hi guix > > I started working on a "functional interface" to guix packages, as an > approach to globally modifying packages in an environment, similar to > nix overlays or the parameterization project. > Right now it's just a syntax rule that you can wrap a bunch of package > definitions in with some inputs in the style of a let. > > > (define-syntax define-public-package-set >(syntax-rules () > ((define-public-package-set pkgset >((input ...) > ...) >(package-name package-def) >...) > (begin > (define-public pkgset > (lambda* (#:optional packages > #:key (input ...) > ... > #:allow-other-keys) > (define package-name package-def) > ... > (cond ((and packages (list? packages)) >(map (lambda (package) > (cond ((eq? package 'package-name) > `(package-name . ,package-name)) > ...)) > packages)) > (packages > (cond ((eq? packages 'package-name) >package-name) > ...)) > (else > `((package-name . ,package-name) > ...) > (set-procedure-property! pkgset 'package-set? #t) > (define-public package-name > (pkgset 'package-name)) > ... > > > Tried rewriting the guile packages module with it, link to the full > code further down but here's a snip: > > > ;; existing module > (define-module (gnu packages guile) >#:use-module ((guix licenses) #:prefix license:) >#:use-module (gnu packages) > ... > > ;; package-set definition > (define-public-package-set gnu-packages-guile >((bash-minimal bash-minimal) ;; inputs > (gawk gawk) > (gmp gmp) > ... > (url-fetch url-fetch)) > >;; packages >(guile-1.8 > (package >(name "guile") > ... > > > You end up with a public "gnu-packages-guile" function that takes > keyword-argument inputs like: > > (gnu-packages-guile #:gawk (@ (gnu packages bioinformatics) bioawk)) > ;; => all the packages in gnu/packages/guile.scm, but every instance > ;; of awk is replaced with bioawk > > > Then returns an alist of all the packages in the set with the inputs > appropriately "modified". For the keys in the alist I'm using the > variable names as symbols instead of the package names like in inputs, > because I feel that's more generally appropriate in the case where a > package-set might contain something that's not a package record. > Will probably write a helper function that spits out input pairs. > > It can also take a single symbol or list of symbols as an optional > argument to produce either a single package or list of requested > packages: > > > (gnu-packages-guile 'guile-3.0) > ;; => # (gnu-packages-guile '(guile-2.2 guile2.2-json)) > ;; => (# > > Packages still get define-public'd, I assume in a way that would > preserve the existing working condition of the repository (although > there is one issue right now, more below) so this could hopefully turn > into a drop-in addition after some more work. I tried to make very few > changes necessary to the existing code to for the same reason. > > Eventually, the goal is to be able to create an operating system > definition (or manifest, or anything else) using this, with a > hypothetical case like: > > > (use-package-modules base network etc.) > > (define my-package-set >(packaget-set-append gnu-packages-base > gnu-packages-network ...)) > > (with-package-set >(my-package-set #:mesa something-else) >(operating-system > (...))) > > > Adapting more complicated package modules would probably be a task, but > I'm invested enough to keep working on it. > > There are also a few problems with the current implementation, and I > think they all boil down to one issue: > If packages A and B are both in the same module, and B is an input to > A, I can't replace B. > There may be a solution, it might be as simple as providing a dummy > input for each package in the module, I don't know, open to ideas. > > Another more minor problem is that it won't pull: > > > In ice-9/boot-9.scm: > 222:29 19 (map1 _) > 222:29 18 (map1 _) > 222:29 17 (map1 _) > 222:29 16 (map1 _) > 222:29 15 (map1 _) > 222:17 14 (map1
Functional package interface
Hi guix I started working on a "functional interface" to guix packages, as an approach to globally modifying packages in an environment, similar to nix overlays or the parameterization project. Right now it's just a syntax rule that you can wrap a bunch of package definitions in with some inputs in the style of a let. (define-syntax define-public-package-set (syntax-rules () ((define-public-package-set pkgset ((input ...) ...) (package-name package-def) ...) (begin (define-public pkgset (lambda* (#:optional packages #:key (input ...) ... #:allow-other-keys) (define package-name package-def) ... (cond ((and packages (list? packages)) (map (lambda (package) (cond ((eq? package 'package-name) `(package-name . ,package-name)) ...)) packages)) (packages (cond ((eq? packages 'package-name) package-name) ...)) (else `((package-name . ,package-name) ...) (set-procedure-property! pkgset 'package-set? #t) (define-public package-name (pkgset 'package-name)) ... Tried rewriting the guile packages module with it, link to the full code further down but here's a snip: ;; existing module (define-module (gnu packages guile) #:use-module ((guix licenses) #:prefix license:) #:use-module (gnu packages) ... ;; package-set definition (define-public-package-set gnu-packages-guile ((bash-minimal bash-minimal) ;; inputs (gawk gawk) (gmp gmp) ... (url-fetch url-fetch)) ;; packages (guile-1.8 (package (name "guile") ... You end up with a public "gnu-packages-guile" function that takes keyword-argument inputs like: (gnu-packages-guile #:gawk (@ (gnu packages bioinformatics) bioawk)) ;; => all the packages in gnu/packages/guile.scm, but every instance ;; of awk is replaced with bioawk Then returns an alist of all the packages in the set with the inputs appropriately "modified". For the keys in the alist I'm using the variable names as symbols instead of the package names like in inputs, because I feel that's more generally appropriate in the case where a package-set might contain something that's not a package record. Will probably write a helper function that spits out input pairs. It can also take a single symbol or list of symbols as an optional argument to produce either a single package or list of requested packages: (gnu-packages-guile 'guile-3.0) ;; => # (#) 619:8 3 (_ #(#(#) # 632:34 2 (_ #(#(#) # 223:20 1 (proc #(#(# In unknown file: 0 (%resolve-variable (5 (gnu packages bash) bash-minimal . ERROR: In procedure %resolve-variable: error: bash-minimal: unbound variable So I'm probably doing something bad. Seems like the first variable in the "inputs" fails to resolve somewhere? I don't even have a guess what's going wrong. My code is available at https://gitlab.vulnix.sh/spacecadet/guix Open to comments or ideas, and if anyone can give me a hand figuring out the pull issue I'd be grateful. - sc
