[bug#62264] [PATCH v2 0/3] Add 'guix locate'
Hello! Here is the “camera-ready” version of the new ‘guix locate’ command (formerly ‘guix index’) that Antoine and myself have worked on. I think it’s ready to go. This version relies only on local knowledge, which means you can only find packages already in your store. In a future iteration, we’ll add an optional mechanism to download the database from elsewhere. Thoughts? Ludo’. Ludovic Courtès (3): store: Tolerate non-existent GC root directories. Add 'guix locate'. DRAFT news: Add entry for 'guix locate'. Makefile.am | 2 + doc/guix.texi | 118 etc/news.scm| 17 ++ guix/scripts/locate.scm | 657 guix/store/roots.scm| 12 +- po/guix/POTFILES.in | 1 + tests/guix-locate.sh| 72 + tests/store-roots.scm | 18 +- 8 files changed, 892 insertions(+), 5 deletions(-) create mode 100644 guix/scripts/locate.scm create mode 100755 tests/guix-locate.sh base-commit: e8f9fb3e03ea8fee0e13f13706a6b16414f74a7b -- 2.40.1
Re: 02/03: gnu: openblas: Update architectures we provide substitutes for.
Efraim Flashner writes: > On Sat, Jun 03, 2023 at 08:12:48PM +0100, Christopher Baines wrote: >> >> Efraim Flashner writes: >> >> > On Fri, Jun 02, 2023 at 11:03:42PM +0100, Christopher Baines wrote: >> >> >> >> guix-comm...@gnu.org writes: >> >> >> >> > efraim pushed a commit to branch master >> >> > in repository guix. >> >> > >> >> > commit 076688fa1e41a09f034a80e1a593bac43f1f1482 >> >> > Author: Efraim Flashner >> >> > AuthorDate: Thu Jun 1 11:06:00 2023 +0300 >> >> > >> >> > gnu: openblas: Update architectures we provide substitutes for. >> >> > >> >> > * gnu/packages/maths.scm (openblas)[arguments]: Adjust the >> >> > substitutable? >> >> > flag to only not provide substitutes when building for >> >> > powerpc-linux. >> >> > Adjust the comment accordingly. >> >> > --- >> >> > gnu/packages/maths.scm | 11 ++- >> >> > 1 file changed, 2 insertions(+), 9 deletions(-) >> >> >> >> I've been looking at why armhf-linux substitute availability has been >> >> dropping recently, and I think this change triggered a lot of >> >> rebuilds. Could this have gone to core-updates? >> >> >> >> → guix refresh -l openblas >> >> Building the following 2282 packages would ensure 5596 dependent packages >> >> are rebuilt: ... >> > >> > It's not that it's triggered rebuilds, but that it's triggered builds. >> > It's also triggered builds on powerpc64le and riscv64. Before any >> > package which had openblas as a transitive dependency wasn't built by >> > the CI because it wasn't substitutable¹. People still have the option of >> > using package transformations to use openblas tuned for the cortex a7 or >> > a15 on armhf, but in reality this just unlocks substitutes for those >> > ~5600 packages which wasn't available before. >> > >> > ¹ We saw this in the past briefly in the past when openzfs made its way >> > as a dependency to qemu and through that to Gnome. >> >> Ok, so the documentation does mention "rebuilding", and I do see that >> indeed ci.guix.gnu.org doesn't build not substitutable things. >> >> Although I think it doesn't apply recursively. Take qjson, guix refresh >> -l tells me it's dependent on openblas, and looking back at say this >> output [1] for powerpc64le-linux, that's available from both >> ci.guix.gnu.org. Which makes sense, as that derivation is substitutable, >> even though one of it's inputs isn't. >> >> 1: >> https://data.guix.gnu.org/gnu/store/fibiwzyz8s899ccpix5zs6r2pcdpxk5b-qjson-0.9.0 >> >> Maybe on the client side this works differently, and guix won't >> substitute things which have a non substitutable input? >> >> Assuming ci.guix.gnu.org was building things for armhf-linux, I think >> this would have still caused ~5596 rebuilds, and as I say, I think for >> systems like powerpc64le-linux, I think it did cause ~5596 >> rebuilds. > > I looked into it more. First I ran on master: > './pre-inst-env guix build --no-grafts --system=armhf-linux openblas -d' > /gnu/store/whi4yhiw2b0c0i3n6l8s0qfcphkvbzg4-openblas-0.3.20.drv > > Then I locally reverted the patch expanding the architectures where we > provided substitutes: > /gnu/store/1m57z8jkbf6gz7qlbw3ws4ayl0ln9602-openblas-0.3.20.drv > > Then I locally reverted the patch adjusting the make-flags: > /gnu/store/1m57z8jkbf6gz7qlbw3ws4ayl0ln9602-openblas-0.3.20.drv > > It seems I was wrong, changing the #:substitutable? flag _does_ change > the derivation of the package. I also checked the substitutes and saw > that bordeaux did (and does) have substitutes for the non-substitutable > version, showing that it was built before. I also checked for > powerpc64le, to see if perhaps cuirass worked a different way and > honored the #:substitutable flag by not building it there, and it too > has substitutes for both versions of openblas. > > I didn't check for riscv64 but I assume the case is the same with a > changing derivation. > > So obviously if I had realized this would cause ~5596 rebuilds per > affected arch I wouldn't have pushed the patch. I should've checked the > derivation before and after to make sure it didn't change. Thanks for investigating further. There's maybe a secondary issue here about why changing the substitutibility of a package affects it's outputs. That doesn't make much sense to me, but maybe there's a reason. signature.asc Description: PGP signature
Re: 01/03: gnu: sbcl: Update to 2.3.5.
Guillaume Le Vaillant writes: > Christopher Baines skribis: > >> guix-comm...@gnu.org writes: >> >>> glv pushed a commit to branch master >>> in repository guix. >>> >>> commit b019b49c74e51e42230da471f39bff9f642fbc24 >>> Author: Guillaume Le Vaillant >>> AuthorDate: Fri Jun 2 13:32:55 2023 +0200 >>> >>> gnu: sbcl: Update to 2.3.5. >>> >>> * gnu/packages/lisp.scm (sbcl): Update to 2.3.5. >>> --- >>> gnu/packages/lisp.scm | 4 ++-- >>> 1 file changed, 2 insertions(+), 2 deletions(-) >> >> Did this change need to go straight to master? >> >> → guix refresh -l sbcl >> Building the following 314 packages would ensure 1615 dependent packages are >> rebuilt: ... >> >> Above 300 but below 1800 means staging according to [1]. >> >> 1: >> https://guix.gnu.org/manual/devel/en/html_node/Submitting-Patches.html#index-rebuild-scheduling-strategy > > Currently there is no staging branch, as it got deleted because of the > changes in the branching strategy ([1]). If someone creates a staging > branch, will cuirass see that and start building it? Or could it just > stay in limbo for months? Deleting the staging branch when it's merged seems sensible to me regardless of whether you want to use that branch in the future. As far as I'm concerned, while there have been disucssions of changing the branching strategy, the guidance as written down hasn't changed yet (there are some suggested changes in [1]). The changes in [1] still keep the 300 dependent limit on pushing changes directly to master though. As for the Cuirass configuration, it might need changing. You can either ask on guix-devel/guix-sysadmin for someone to do this, or ask on guix-sysadmin to get access to Cuirass so that you can use the admin interface yourself [2]. 2: https://lists.gnu.org/archive/html/guix-devel/2023-06/msg9.html > Concerning the specific case of sbcl dependents, they build pretty fast. > On my machine, building all of them took less than one hour. So the > period during which some substitutes are missing is pretty short. > > [1] https://issues.guix.gnu.org/63459 That's still time people will be without substitutes, and because these changes in effect "jumped the queue", that's also time the build farm won't be building changes from patches or the branch(s) that it's meant to be working on (since the master branch builds have priority). signature.asc Description: PGP signature
Re: 01/03: gnu: sbcl: Update to 2.3.5.
Christopher Baines skribis: > guix-comm...@gnu.org writes: > >> glv pushed a commit to branch master >> in repository guix. >> >> commit b019b49c74e51e42230da471f39bff9f642fbc24 >> Author: Guillaume Le Vaillant >> AuthorDate: Fri Jun 2 13:32:55 2023 +0200 >> >> gnu: sbcl: Update to 2.3.5. >> >> * gnu/packages/lisp.scm (sbcl): Update to 2.3.5. >> --- >> gnu/packages/lisp.scm | 4 ++-- >> 1 file changed, 2 insertions(+), 2 deletions(-) > > Did this change need to go straight to master? > > → guix refresh -l sbcl > Building the following 314 packages would ensure 1615 dependent packages are > rebuilt: ... > > Above 300 but below 1800 means staging according to [1]. > > 1: > https://guix.gnu.org/manual/devel/en/html_node/Submitting-Patches.html#index-rebuild-scheduling-strategy Currently there is no staging branch, as it got deleted because of the changes in the branching strategy ([1]). If someone creates a staging branch, will cuirass see that and start building it? Or could it just stay in limbo for months? Concerning the specific case of sbcl dependents, they build pretty fast. On my machine, building all of them took less than one hour. So the period during which some substitutes are missing is pretty short. [1] https://issues.guix.gnu.org/63459 signature.asc Description: PGP signature
Re: 01/03: gnu: sbcl: Update to 2.3.5.
guix-comm...@gnu.org writes: > glv pushed a commit to branch master > in repository guix. > > commit b019b49c74e51e42230da471f39bff9f642fbc24 > Author: Guillaume Le Vaillant > AuthorDate: Fri Jun 2 13:32:55 2023 +0200 > > gnu: sbcl: Update to 2.3.5. > > * gnu/packages/lisp.scm (sbcl): Update to 2.3.5. > --- > gnu/packages/lisp.scm | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) Did this change need to go straight to master? → guix refresh -l sbcl Building the following 314 packages would ensure 1615 dependent packages are rebuilt: ... Above 300 but below 1800 means staging according to [1]. 1: https://guix.gnu.org/manual/devel/en/html_node/Submitting-Patches.html#index-rebuild-scheduling-strategy signature.asc Description: PGP signature
Re: branch master updated: gnu: sbcl: fix build on riscv64-linux.
guix-comm...@gnu.org writes: > This is an automated email from the git hooks/post-receive script. > > efraim pushed a commit to branch master > in repository guix. > > The following commit(s) were added to refs/heads/master by this push: > new 2a6d2fc1d8 gnu: sbcl: fix build on riscv64-linux. > 2a6d2fc1d8 is described below > > commit 2a6d2fc1d8e3434e283d6d2e559529b41a242fea > Author: Efraim Flashner > AuthorDate: Wed Jun 7 09:55:35 2023 +0300 > > gnu: sbcl: fix build on riscv64-linux. > > * gnu/packages/patches/sbcl-riscv-Make-contribs-build-again.patch: New > file. > * gnu/local.mk(dist_patch_DATA): register it. > * gnu/packages/lisp.scm (sbcl): [source]: Use it here. > --- > gnu/local.mk | 1 + > gnu/packages/lisp.scm | 3 + > .../sbcl-riscv-Make-contribs-build-again.patch | 71 > ++ > 3 files changed, 75 insertions(+) Did this change need to go straight to master? → guix refresh -l sbcl Building the following 314 packages would ensure 1615 dependent packages are rebuilt: ... To me that says to be that this would be appropriate for staging [1]. 1: https://guix.gnu.org/manual/devel/en/html_node/Submitting-Patches.html#index-rebuild-scheduling-strategy Maybe this would be OK if the patch was applied in such a way that it only affects riscv64-linux, but it isn't, so this affects all systems. signature.asc Description: PGP signature
Re: 01/03: gnu: wxwidgets: Add libxtst to inputs.
guix-comm...@gnu.org writes: > apteryx pushed a commit to branch master > in repository guix. > > commit ec9f15b158300da3a77ce02cd2267222f435e80f > Author: Maxim Cournoyer > AuthorDate: Tue Jun 6 12:12:40 2023 -0400 > > gnu: wxwidgets: Add libxtst to inputs. > > WxWidgets was already built with XTest support, but mostly by luck, via > propagation of libxtst from GTK's propagated at-spi2-core package. Make > it an > explicit input. > > * gnu/packages/wxwidgets.scm (wxwidgets) [inputs]: Add libxtst. > --- > gnu/packages/wxwidgets.scm | 1 + > 1 file changed, 1 insertion(+) Did this need to go straight to the master branch? → guix refresh -l wxwidgets Building the following 217 packages would ensure 456 dependent packages are rebuilt: ... That to me says this should go to staging. signature.asc Description: PGP signature
