Re: Developing libraries for the GNU system with Guix
l...@gnu.org (Ludovic Courtès) writes: > Done in 645b9df858683dc05ffa04c9eb2fdc45ccef4a65. Feedback welcome! > > Ludo’. Excellent! I will try to use this in the future, and see how well it works for my use case.
Re: Developing libraries for the GNU system with Guix
Ricardo Wurmus skribis: > Ludovic Courtès writes: > >> Ricardo Wurmus skribis: >> >>> Ludovic Courtès writes: >>> Again to make this more convenient, I thought we could have a --with-graft option, which would work like --with-input except that it would graft the new Z onto A/B/C instead of rebuilding them. >>> >>> This is a good idea! >> >> Done in 645b9df858683dc05ffa04c9eb2fdc45ccef4a65. Feedback welcome! > > Thanks! This looks great. The only change I’d make is in the > documentation: > >“instead of rebuilding all the dependency chain, …” > > “all the … chain” sounds odd. How about “the whole dependency chain” > instead? Yes, please go ahead. :-) Ludo’.
Re: Developing libraries for the GNU system with Guix
Ludovic Courtès writes: > Ricardo Wurmus skribis: > >> Ludovic Courtès writes: >> >>> Again to make this more convenient, I thought we could have a >>> --with-graft option, which would work like --with-input except that it >>> would graft the new Z onto A/B/C instead of rebuilding them. >> >> This is a good idea! > > Done in 645b9df858683dc05ffa04c9eb2fdc45ccef4a65. Feedback welcome! Thanks! This looks great. The only change I’d make is in the documentation: “instead of rebuilding all the dependency chain, …” “all the … chain” sounds odd. How about “the whole dependency chain” instead? ~~ Ricardo
Re: Developing libraries for the GNU system with Guix
Ricardo Wurmus skribis: > Ludovic Courtès writes: > >> Again to make this more convenient, I thought we could have a >> --with-graft option, which would work like --with-input except that it >> would graft the new Z onto A/B/C instead of rebuilding them. > > This is a good idea! Done in 645b9df858683dc05ffa04c9eb2fdc45ccef4a65. Feedback welcome! Ludo’.
Re: Developing libraries for the GNU system with Guix
Ludovic Courtès writes: > Again to make this more convenient, I thought we could have a > --with-graft option, which would work like --with-input except that it > would graft the new Z onto A/B/C instead of rebuilding them. This is a good idea! ~~ Ricardo
Re: Developing libraries for the GNU system with Guix
l...@gnu.org (Ludovic Courtès) writes: > sba...@catern.com skribis: >> - Currently every dependency is located at a well known globally unique >> and globally meaningful path; add some kind of "variant package" >> construct which specifies a package which is "passed in" to the >> environment (maybe by bind-mounting it in a filesystem namespace; >> implementation specifics aren't important). To put this in a >> programming language sense, one of these packages being present would >> turn a Guix distribution from a value into a function. > > Not sure I understand. What do you mean by “passed in to the > environment”? I just mean that this would be a path that is not necessarily pointing to a directory containing the files desired by other packages depending on it. To make a complete functioning system, the path would need to be pointed at something containing the right files. (something of the right type) It could be pointed at different directories in different filesystem namespaces, and that "pointing" would happen outside the filesystem namespace when the namespace is created.
Re: Developing libraries for the GNU system with Guix
Hello! sba...@catern.com skribis: > When I am hacking on some library Z, I continuously want to test the > effects that my changes to Z have on packages A/B/C which depend on > Z. The same applies, in general, when hacking on any package Z which > other packages A/B/C depend on: While developing, I want to be able to > rapidly mutate Z and see how this affects A/B/C. A very common use case. Others have asked about it. > I am not sure how to best achieve this. Here are some solutions: > > - When you change Z, rebuild A/B/C. > This is much too slow at present. Right, but note that it’s the only way to be confident that the change in Z doesn’t break A/B/C. That said, I do understand that sometimes, you want an “I know what I’m doing” (i.e., the ABI of Z hasn’t changed) option to bypass that and test the run-time behavior of A/B/C. For cases where you do want to rebuild anyway, I was thinking of making the ‘--with-source’ option “recursive”, such that: guix build --with-source=./guile-2.0.14rc1.tar.gz guile-json would replace the source of Guile and build Guile-JSON against that. > - Use grafts to update A/B/C through binary patching. > This is also too slow, and AFAIK it can't really be sped up. A/B/C have to be really big for this to be too slow! I think grafting processes several MiB/s on my SSD laptop. Again to make this more convenient, I thought we could have a --with-graft option, which would work like --with-input except that it would graft the new Z onto A/B/C instead of rebuilding them. > - Use LD_LIBRARY_PATH to cause A/B/C to search for Z in a mutable place. > This works for C libraries, but not generically; there are equivalent > variables for some other languages but it's not a full solution. Yeah, not great. > - Before starting to hack on Z, build a new version of Z which includes > a random hash and which A/B/C depend on; then bind-mount a mutable > directory on top of that. (suggested by mark_weaver on IRC) > This is the most palatable hack, but still a hack. The inclusion of a > random hash prevents collision with other packages and the use of > bind-mounting means we aren't actually mutating the store. This > unfortunately also requires privileges: it's not usable by > unprivileged users. It is usable by unprivileged users when user namespaces are available. Under these conditions, you could use ‘call-with-container’ and bind-mount anything anywhere. Doesn’t sound too nice to me though. > Here are some not currently available possibilities: > > - Create some kind of GUIX_LIBRARY_PATH in which packages are looked up > first before looking at the compiled-in hash. > This would be an attempt to make a generic equivalent of > LD_LIBRARY_PATH. This could theoretically be implemented with variant > symlinks, if they were available: https://lwn.net/Articles/680705/ At first sight this sounds very hacky, very much against the whole idea of functional package management. > - Currently every dependency is located at a well known globally unique > and globally meaningful path; add some kind of "variant package" > construct which specifies a package which is "passed in" to the > environment (maybe by bind-mounting it in a filesystem namespace; > implementation specifics aren't important). To put this in a > programming language sense, one of these packages being present would > turn a Guix distribution from a value into a function. Not sure I understand. What do you mean by “passed in to the environment”? > - Massively speed up rebuilding A/B/C by performing incremental > builds. Not sure how exactly this could work, it's just a thought. No idea how this could work either. Thanks for raising the issue! Ludo’.
Developing libraries for the GNU system with Guix
Hi guix-devel, When I am hacking on some library Z, I continuously want to test the effects that my changes to Z have on packages A/B/C which depend on Z. The same applies, in general, when hacking on any package Z which other packages A/B/C depend on: While developing, I want to be able to rapidly mutate Z and see how this affects A/B/C. I am not sure how to best achieve this. Here are some solutions: - When you change Z, rebuild A/B/C. This is much too slow at present. - Use grafts to update A/B/C through binary patching. This is also too slow, and AFAIK it can't really be sped up. - Use LD_LIBRARY_PATH to cause A/B/C to search for Z in a mutable place. This works for C libraries, but not generically; there are equivalent variables for some other languages but it's not a full solution. - Before starting to hack on Z, build a new version of Z which includes a random hash and which A/B/C depend on; then bind-mount a mutable directory on top of that. (suggested by mark_weaver on IRC) This is the most palatable hack, but still a hack. The inclusion of a random hash prevents collision with other packages and the use of bind-mounting means we aren't actually mutating the store. This unfortunately also requires privileges: it's not usable by unprivileged users. Here are some not currently available possibilities: - Create some kind of GUIX_LIBRARY_PATH in which packages are looked up first before looking at the compiled-in hash. This would be an attempt to make a generic equivalent of LD_LIBRARY_PATH. This could theoretically be implemented with variant symlinks, if they were available: https://lwn.net/Articles/680705/ - Currently every dependency is located at a well known globally unique and globally meaningful path; add some kind of "variant package" construct which specifies a package which is "passed in" to the environment (maybe by bind-mounting it in a filesystem namespace; implementation specifics aren't important). To put this in a programming language sense, one of these packages being present would turn a Guix distribution from a value into a function. - Massively speed up rebuilding A/B/C by performing incremental builds. Not sure how exactly this could work, it's just a thought. What do you think, guix-devel? By the way, I think this is a pretty important feature overall (though it's described in fairly abstract terms in this email). I think this is one of the last missing features to make the Guix approach a clear and uncontroversial improvement over existing widespread practices.