bug#26441: Gnulib’s ‘test-lock’ fails to complete on machines with >= 32 cores

2017-04-12 Thread Mathieu Othacehe
Hi, > This may be fixed by Gnulib commit > 480d374e596a0ee3fed168ab42cd84c313ad3c89 (not present in On gettext, commit 1afbcb06fded2a427b761dd1615b1e48e1e853cc seems to fix the problem. I ran three consecutive tests : --8<---cut here---start->8--- mathieu@elb

bug#26441: Gnulib’s ‘test-lock’ fails to complete on machines with >= 32 cores

2017-04-12 Thread Mathieu Othacehe
YT ? Thank you, Mathieu >From e4ad9aa61fa75afa4417616de36cd25e9631fd67 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe Date: Wed, 12 Apr 2017 21:17:24 +0200 Subject: [PATCH] gnu: gettext: Fix make check issues on multi-core machines. * gnu/packages/patches/gettext-gnulib-multi-core.patch: New fi

bug#26441: Gnulib’s ‘test-lock’ fails to complete on machines with >= 32 cores

2017-04-13 Thread Mathieu Othacehe
> Maybe we can simply patch the important packages (findutils, > libunistring) and wait and see if others need the patch. Not ideal, but > I can’t think of a better way. Ok fine for me. I'll send findutils and libunistring patches for now. Thanks, Mathieu

bug#26441: [PATCH] gnu: findutils: Fix make check issues on multi-core machines.

2017-04-13 Thread Mathieu Othacehe
@@ ;;; Copyright © 2016 Efraim Flashner ;;; Copyright © 2016 Jan Nieuwenhuizen ;;; Copyright © 2017 Rene Saavedra +;;; Copyright © 2017 Mathieu Othacehe ;;; ;;; This file is part of GNU Guix. ;;; @@ -250,38 +251,43 @@ interactive means to merge two files.") (define-public find

bug#26441: [PATCH] gnu: findutils: Fix make check issues on multi-core machines.

2017-04-13 Thread Mathieu Othacehe
Hi Marius, > Please don't mix large indentation changes with functional changes. It > is really difficult to tell what this commit does. Can you split the > indentation change out in a separate commit? Sure, I'll send new patches. Mathieu

bug#26441: [PATCH 1/2] gnu: findutils: Fix make check issues on multi-core machines.

2017-04-13 Thread Mathieu Othacehe
/base.scm index 7bcfd7a2e..1ae3c8911 100644 --- a/gnu/packages/base.scm +++ b/gnu/packages/base.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2016 Efraim Flashner ;;; Copyright © 2016 Jan Nieuwenhuizen ;;; Copyright © 2017 Rene Saavedra +;;; Copyright © 2017 Mathieu Othacehe ;;; ;;; This file is part of

bug#26441: [PATCH 2/2] gnu: findutils: Reindent package definition.

2017-04-13 Thread Mathieu Othacehe
* gnu/packages/base.scm (findutils): Reindent, no functional change. --- gnu/packages/base.scm | 64 +-- 1 file changed, 32 insertions(+), 32 deletions(-) diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm index 1ae3c8911..e620d9cea 100644 -

bug#26441: [PATCH 2/3] gnu: findutils: Fix make check issues on multi-core machines.

2017-04-17 Thread Mathieu Othacehe
Mathieu Othacehe ;;; ;;; This file is part of GNU Guix. ;;; @@ -259,8 +260,13 @@ interactive means to merge two files.") (sha256 (base32 "178nn4dl7wbcw499czikirnkniwnx36argdnqgz4ik9i6zvwkm6y")) -(patches (search-pat

bug#26441: [PATCH 3/3] gnu: libunistring: Fix make check issues on multi-core machines.

2017-04-17 Thread Mathieu Othacehe
2016 Jan Nieuwenhuizen +;;; Copyright © 2017 Mathieu Othacehe ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,6 +25,7 @@ #:use-module (guix packages) #:use-module (guix download) #:use-module (guix build-system gnu) + #:use-module (gnu packages) #:use-module (gnu packages base

bug#26441: [PATCH 1/2] gnu: findutils: Fix make check issues on multi-core machines.

2017-04-17 Thread Mathieu Othacehe
Hi Marius ! > It looks like you forgot the actual patch :) Ha ! You're right :) > > Please also add it to gnu/local.mk. TIA! Ok, I'll a new batch, to be applied to core-updates. Thanks, Mathieu

bug#26441: [PATCH 1/3] build: Add two missing patches to local.mk.

2017-04-17 Thread Mathieu Othacehe
gnu/local.mk (dist_patch): Add gettext-multi-core.patch and gettext-gnulib-multi-core.patch. Commit 480da86d0969a667e8d2a564de535cb73a6a2229 ommited them. --- gnu/local.mk | 2 ++ 1 file changed, 2 insertions(+) diff --git a/gnu/local.mk b/gnu/local.mk index 51f92f088..620fb07f3 100644 --- a/gnu

bug#26791: Attempt to apply a string as a procedure during Hydra evaluation

2017-05-06 Thread Mathieu Othacehe
Hi Mark, This problem should be resolved by make clean-go && make, see here : https://lists.gnu.org/archive/html/guix-patches/2017-05/msg00152.html Mathieu Mark H Weaver writes: > The error message is: > > ERROR: In procedure GNU with Linux-Libre 4.11 (beta): > ERROR: Wrong type to apply:

bug#26987: guix pull without guile-ssh.

2017-05-18 Thread Mathieu Othacehe
Hi, If Guile-SSH is not detected, depends-on-guile-ssh? will be called here: --8<---cut here---start->8--- (guix build pull) (let* ((files (remove (if (false-if-exception (resolve-interface '(ssh session)))

bug#26987: [PATCH 0/2] Fix compilation list construction.

2017-05-19 Thread Mathieu Othacehe
hanks, Mathieu Mathieu Othacehe (2): build: pull: Fix compilation list construction. guix: modules: Export module-name->file-name. guix/build/pull.scm | 31 --- guix/modules.scm| 1 + 2 files changed, 9 insertions(+), 23 deletions(-) -- 2.13.0

bug#26987: [PATCH 2/2] build: pull: Fix compilation list construction.

2017-05-19 Thread Mathieu Othacehe
ght © 2015 Taylan Ulrich Bayırlı/Kammer +;;; Copyright © 2017 Mathieu Othacehe ;;; ;;; This file is part of GNU Guix. ;;; @@ -36,26 +37,14 @@ ;;; ;;; Code: -(define (depends-on-guile-ssh? file) - "Return true if FILE is a Scheme source file that depends, directly or -indirectly, on Guile

bug#26987: [PATCH 1/2] guix: modules: Export module-name->file-name.

2017-05-19 Thread Mathieu Othacehe
* guix/modules.scm (module-name->file-name): Export it. --- guix/modules.scm | 1 + 1 file changed, 1 insertion(+) diff --git a/guix/modules.scm b/guix/modules.scm index 8c63f21a9..26c38e8d2 100644 --- a/guix/modules.scm +++ b/guix/modules.scm @@ -23,6 +23,7 @@ #:use-module (ice-9 match) #:

bug#26987: [PATCH 2/2] build: pull: Fix compilation list construction.

2017-05-19 Thread Mathieu Othacehe
Hi Ludo, It seems better but I can't get it to work neither : > (find (match-lambda > (('ssh _ ...) #t) > (_ #f)) > -(source-module-closure file #:select? (const #t > + (source-module-closure module #:select? (const #t)

bug#26987: [PATCH 2/2] build: pull: Fix compilation list construction.

2017-05-19 Thread Mathieu Othacehe
> I'm having a hard time tring to understand ,trace output. I get why, it's because guile-ssh is not in my %load-path. So loading "ssh/session.scm" fails in "source-module-dependencies". It's a bit of a vicious circle here : if resolve-interface '(ssh session) fails, depends-on-guile-ssh? will f

bug#27007: boot-parameters are not documented

2017-05-20 Thread Mathieu Othacehe
Hi Tomáš, > My question without answer is - how can I specify bootloader menu entries now? You're right, you have to pass a now. The documentation patch is still in review, you can find it here : https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26339#489 The example has been updated : --8<-

bug#27007: boot-parameters are not documented

2017-05-22 Thread Mathieu Othacehe
Hi Ludo, Since the rework, are in (gnu bootloader grub). They can no longer be used in menu-entries field of . The is used instead. and are very similar though. What we can do here is : * Give default values as per (as proposed by Danny in doc patch review). * Rename "menu-entries" field

bug#27007: boot-parameters are not documented

2017-05-23 Thread Mathieu Othacehe
Hello, > So how does the ‘menu-entry’ example that Tomáš gave translate with this > new API? (Apologies for asking, I admit I haven’t fully adjusted to the > new API mentally. :-)) Well it's still moving :) We can ask him but I guess something like that : --8<---cut here-

bug#27007: boot-parameters are not documented

2017-05-23 Thread Mathieu Othacehe
> and those entries would be “lowered” to somewhere. > That way we’d provide the abstraction level that users may expect (“how > do I add a menu entry for my other distro?”) and at the same time reduce > the risk of mistakes (“I didn’t what to put in ‘store-device’ I put my > pet’s name there”).

bug#27007: [PATCH 1/2] bootloader: Use menu-entry to define custom bootloader entries.

2017-05-31 Thread Mathieu Othacehe
* gnu/bootloader.scm (): New variable. Export associated getters, This record is extracted from grub module. * gnu/bootloader/extlinux.scm (extlinux-configuration-file): Use menu-entry->boot-parameters to convert menu-entry records to boot-parameters. * gnu/bootloader/grub.scm (): Remove. (boot

bug#27007: [PATCH 0/2] Use menu-entry to define custom bootloader entries.

2017-05-31 Thread Mathieu Othacehe
Hi, Here's the patch to use menu-entry instead of boot-parameters to define custom bootloaders entries as discussed. The second patch is the updated documentation for bootloader rework. Thanks, Mathieu Mathieu Othacehe (2): bootloader: Use menu-entry to define custom bootloader en

bug#27007: [PATCH 2/2] doc: Adapt to multiple bootloader support.

2017-05-31 Thread Mathieu Othacehe
* doc/guix.texi (GRUB configuration): Rename to "Bootloader configuration". Adapt occurences of "GRUB" in other sections. --- doc/guix.texi | 164 -- 1 file changed, 91 insertions(+), 73 deletions(-) diff --git a/doc/guix.texi b/doc/guix

bug#27007: [PATCH 1/2] bootloader: Use menu-entry to define custom bootloader entries.

2017-06-01 Thread Mathieu Othacehe
Hi Danny, > For another future patch: Hmm, should we make this optional? I didn't have > an initrd for many years. Well I tried to run GuixSD without initrd and it's not possible yet. Besides mounting root partition, it's also used to start the initial guile script (/run/current-system/boot).

bug#26987: [PATCH 2/2] build: pull: Fix compilation list construction.

2017-06-02 Thread Mathieu Othacehe
Hi Ludo, > I believe this is fixed by commit > 20ed093977cc80ba1729c38e05ae7955a38069a6, which follows a modification > to the ‘source-module-closure’ so that callers can catch > missing-dependency errors. > > Please let me know what you think! It seems ok, thanks for fixing it ! > > After that

bug#27007: [PATCH 1/2] bootloader: Use menu-entry to define custom bootloader entries.

2017-06-02 Thread Mathieu Othacehe
Hi ! > People apparently start other Linux distributions with it :) Yes because they use initrd only to mount root partition, then kernel is starting /sbin/init provided by systemd or other init mechanism. For people who's custom kernel is able to access root partition without loading any modul

bug#27007: [PATCH 1/2] bootloader: Use menu-entry to define custom bootloader entries.

2017-06-06 Thread Mathieu Othacehe
Hi Ludo, > Do we still need ‘device-mount-point’ now? For the dual-boot use case, > I don’t see how this would be used. Nope, you're right it's not needed in . >> + (initrd (menu-entry-initrd menu-entry > > It’s weird to set ‘store-device’ and ‘store-mount-point’ here since > there’s no

bug#27007: [PATCH 2/2] doc: Adapt to multiple bootloader support.

2017-06-06 Thread Mathieu Othacehe
> What happens with other bootloaders? Do we get older boot entries? It > might be worth mentioning. On extlinux we also get older boot entries but not in a submenu. I plan to add submenu support in a new serie. I'll mention that this is true for all supported bootloaders when this will be don

bug#27007: [PATCH v2 2/2] doc: Adapt to multiple bootloader support.

2017-06-08 Thread Mathieu Othacehe
* doc/guix.texi (GRUB configuration): Rename to "Bootloader configuration". Remove device-mount-point field from menu-entry description. Adapt occurences of "GRUB" in other sections. --- doc/guix.texi | 177 -- 1 file changed, 98 insert

bug#27007: [PATCH v2 1/2] bootloader: Use menu-entry to define custom bootloader entries.

2017-06-08 Thread Mathieu Othacehe
* gnu/bootloader.scm (): New variable. Export associated getters, This record is extracted from grub module. * gnu/bootloader/extlinux.scm (extlinux-configuration-file): Use menu-entry->boot-parameters to convert menu-entry records to boot-parameters. * gnu/bootloader/grub.scm (): Remove. (boot

bug#27007: [PATCH 2/2] doc: Adapt to multiple bootloader support.

2017-06-08 Thread Mathieu Othacehe
Hi, Just sent a v2 addressing your remarks ! Thanks, Mathieu > I think it would make sense to be explicit about this in the manual. > > Ludo’.

bug#27007: [PATCH v2 2/2] doc: Adapt to multiple bootloader support.

2017-06-08 Thread Mathieu Othacehe
> In the future, I think it’ll be preferable to commit code/tests/doc > together. That is, in the commit that changes the API from being > GRUB-specific to supporting multiple bootloaders, we include not only > the code but also the doc update. This simplifies review, makes sure > each commit is

bug#30825: Building disk-image fails

2018-03-15 Thread Mathieu Othacehe
Hi Ludo & Andreas, > Could you do: > > guix gc --references > /gnu/store/vd8bx5jd868mnhm0mdj8zgjsvjakzh4y-disk-image.drv | grep > disk-image-builder > > and try to run the qemu-system-arm command that appears there with the > arguments that from gnu/build/vm.scm line 163? > > Mathieu came up

bug#31159: Reverse order for old grub.cfg entries

2018-04-17 Thread Mathieu Othacehe
Hi, This seems like a good idea ! If we decide to make it unconditional the patch attached does the job. We can also make it configurable in bootloader-configuration for instance. WDYT ? Mathieu >From 3ec63fb55a074b547724c70d560cc61776c9298e Mon Sep 17 00:00:00 2001 From: Mathieu Othac

bug#31159: Reverse order for old grub.cfg entries

2018-04-19 Thread Mathieu Othacehe
> Not really. I think we can at least send a heads-up to guix-devel and > help-guix. Ok. Would the attached patch earlier in the thread be ok for you then ? Thanks, Mathieu

bug#31159: #31159: Reverse order for old grub.cfg entries

2018-06-01 Thread Mathieu Othacehe
Hello, I tought this one was pushed and told Clément it was ok to push, my mistake! I just pushed it, so I close the ticket now. To people used to select bootloader entries with shortcuts, be aware that at next reconfigure, the entries will be reversed on all bootloaders (top of the list => mos

bug#43163: [Cuirass] /specifications fails

2020-09-02 Thread Mathieu Othacehe
Hey Ludo, > This might be related to recent Guile-JSON API changes, as the > ‘proc_args’ value for core-updates-core-updates above LGTM. > > This is with cuirass-0.0.1-42.d332955. You are right, the lists of 'proc_args' were not converted to vectors, making Guile-JSON unhappy. Fixed by b135a02b

bug#32548: Cuirass: Performance monitoring

2020-09-06 Thread Mathieu Othacehe
Hello, > As discussed earlier today on IRC with Clément, we could add performance > monitoring capabilities to Cuirass. Interesting metrics would be: > > • time of push to time of evaluation completion; > > • time of evaluation completion to time of build completion. Small update on that o

bug#32548: Cuirass: Performance monitoring

2020-09-10 Thread Mathieu Othacehe
Hello, > Agreed. We regularly push commits that are weeks or months old > (sometimes years), so there might be too many outliers when looking at > the commit time. Yes, so I used checkout time instead of commit time with af12a80599346968fb9f52edb33b48dd26852788. I also turned Evaluation 'in_p

bug#41668: Failing test: gui-installed-desktop-os-encrypted

2020-09-11 Thread Mathieu Othacehe
Hello, > Is it deterministic? Same story with “gui-installed-os”? > > I guess we can add more ‘syslog’ statements in the installer around the > place where we restart services. Or we could run the installer entirely > under strace. Now that the installation is done in a container, this one do

bug#43334: Cuirass crashes

2020-09-11 Thread Mathieu Othacehe
Hello, I've observed a few Cuirass crashes the past days. The log looks like: --8<---cut here---start->8--- 2020-09-11T12:55:35 next evaluation in 300 seconds GC Warning: Repeated allocation of very large block (appr. size 28766208): May lead to memor

bug#43334: Cuirass crashes

2020-09-11 Thread Mathieu Othacehe
Hey Chris, >> It looks like a memory allocation failed causing a Cuirass/Guile crash. > > So, I've seen this before but in a slightly different context, [1]. To > summarise, with Guile built with libgc@8 the Guix Data Service couldn't > processes Guix revisions, because the code it had Guile bui

bug#43344: "basic" system tests fail (and all the other ones) on guix master

2020-09-14 Thread Mathieu Othacehe
Hello Danny, > 1. &invoke-error: > program: "parted" > arguments: ("--script" "/dev/vda" "mklabel" "msdos" "mkpart" "primary" > "e) So it looks like the parted script failed inside the VM, while running "initialize-partition-table". This work fine both on the build farm and on m

bug#32548: Cuirass: Performance monitoring

2020-09-14 Thread Mathieu Othacehe
Hello, I just pushed support for computing and displaying metrics in Cuirass. I started with two metrics: * Builds per day * Average evaluation speed per specification. Those metrics can now be seen at: https://ci.guix.gnu.org/metrics and are updated every hour. I plan to add more metrics s

bug#32548: Cuirass: Performance monitoring

2020-09-17 Thread Mathieu Othacehe
Hello Andreas, > Congratulations, that looks like a very useful start already! > (And the number of builds has doubled since yesterday, so someone already > put it to good use!) Thanks for your feedback :) > How about also adding metrics per build machine? I have the impression, > for instance

bug#32548: Cuirass: Performance monitoring

2020-09-17 Thread Mathieu Othacehe
Hey Ludo, > As discussed on IRC, builds per day should be compared to new > derivations per day. For example, if on a day there’s 100 new > derivations and we only manage to build 10 of them, we have a problem. I added this line, and they sadly do not overlap :( > 2020-09-14T21:16:21 Failed t

bug#43533: guix-daemon fails to start in Childhurd

2020-09-21 Thread Mathieu Othacehe
Hello janneke, > 8ce6f4dc2879919c12bc76a2f4b01200af97e019 > installer: Run the installation inside a container. > > ...but I don't find the commit message quite clear about its intention > to *always* run guix-daemon in a container; it could be read as > sugessting to do so only during i

bug#43564: cuirass: Contention while registering new builds.

2020-09-22 Thread Mathieu Othacehe
Hello, The attached screenshot shows that 9 evaluations are currently "In progress" for "guix-master" specification. Evaluations 16725 to 16738 are completed, 7 hours ago and 56 minutes ago respectively. They are still shown as "In progress" because the "register" phase in "build-packages" is no

bug#43565: cuirass: Fibers scheduling blocked.

2020-09-22 Thread Mathieu Othacehe
Hello, Today between 04:04 and 10:36 no inputs were fetched. Fetching is supposed to happen every 5 minutes. This seem to be correlated to the duration of the garbage collection happening on berlin. --8<---cut here---start->8--- 2020-09-22T04:04:23 fetching i

bug#43588: cuirass: Slow SQL queries.

2020-09-24 Thread Mathieu Othacehe
Hello, I just added SQL queries logging to Cuirass. I also adapted the Guix shepherd service accordingly and deployed it on berlin. The results are already interesting: --8<---cut here---start->8--- SELECT E.id, E.status, E.timestamp, E.checkouttime, E.evalt

bug#43585: Order in manifest and command-line differs

2020-09-24 Thread Mathieu Othacehe
Hello zimoun, > $ guix package -i busybox diffutils -p /tmp/foo > $ ls -l /tmp/foo/bin > [..] > df -> /gnu/store/…-busybox-1.31.1/bin/df > diff -> /gnu/store/…-diffutils-3.7/bin/diff > diff3 -> /gnu/store/…-diffutils-3.7/bin/diff3 > dirname -> /gnu/store/…-busybox-1.31.1/bin/dirname > [..]

bug#43585: Order in manifest and command-line differs

2020-09-25 Thread Mathieu Othacehe
Hey zimoun, Oh I see, thanks for explaining! It would be nice if the `guix package -i` command could also operate from left to right then. Would you like to give it a try? Thanks, Mathieu -- https://othacehe.org

bug#43629: Cross-compiled guile-lzlib unusable, causing ‘guix substitute’ to crash on GNU/Hurd

2020-09-26 Thread Mathieu Othacehe
Hey Ludo, > checking dependency style of i586-pc-gnu-gcc... none > checking lzlib's file name... Oh, I guess the same goes for guile-zlib and all cross-compilation targets. I'll have a look later on. Thanks, Mathieu -- https://othacehe.org

bug#43564: cuirass: Contention while registering new builds.

2020-09-28 Thread Mathieu Othacehe
Hello, > As each evaluation has to register around 50k builds, there might be > some sort of "writing" contention on the database, explaining the long > build registration time. Turns out, once an evaluation is over, new builds are registered. This registration tries to insert a new build for e

bug#43564: cuirass: Contention while registering new builds.

2020-09-28 Thread Mathieu Othacehe
For future reference, here's a small addition. As I said, we are using the Cuirass SQLite database in WAL mode. This means that insertion queries are added to a separate cuirass.db-wal file. Once in a while, this file is supposed to be merged in the main cuirass.db file. This mechanism known as

bug#43629: Cross-compiled guile-lzlib unusable, causing ‘guix substitute’ to crash on GNU/Hurd

2020-09-28 Thread Mathieu Othacehe
Hello, > When you feel like it, we can have a .2 release of Guile-lzlib with this > fix. Done with af56f60ef0394b35ab7926d10a4f825c2b1245f6. Thanks, Mathieu

bug#43588: cuirass: Slow SQL queries.

2020-09-28 Thread Mathieu Othacehe
Hello, > SELECT E.id, E.status, E.timestamp, E.checkouttime, E.evaltime, B.total, > B.succeeded, B.failed, B.scheduled FROM (SELECT id, status, timestamp, > checkouttime, evaltime FROM Evaluations WHERE (id='8255' )) E LEFT JOIN > (SELECT rowid, evaluation, SUM(status=0) as succeeded, SUM(sta

bug#43564: cuirass: Contention while registering new builds.

2020-09-29 Thread Mathieu Othacehe
Hello, > In the best case of one pending evaluation, the registration duration is > reduced from 1800 seconds to 320 seconds. I think that the gain is way > larger when there are multiple pending evaluations. > > Pushed a fix as 461e07e14e1c8013343c0a2cb26c0e022e10d5e4. While this improves the

bug#43757: cuirass: Fail to fetch "guix" input.

2020-10-02 Thread Mathieu Othacehe
Hello, Running local tests, I have the following error: --8<---cut here---start->8--- 2020-10-02T11:54:09 fatal: uncaught exception 'misc-error' in 'build' fiber! 2020-10-02T11:54:09 exception arguments: (#f "Attempt to suspend fiber within continuation barr

bug#43760: cuirass: fail to install with simple configuration

2020-10-02 Thread Mathieu Othacehe
Hello Jonathan, > ice-9/boot-9.scm:1669:16: In procedure raise-exception: > Wrong type (expecting string): #f Thanks for the bug report. This should be fixed with d6a8f0a9781a90c3037f25e51d7ff32e50f7a8c1. You will then probably hit this one: https://issues.guix.gnu.org/43757 but we'll see th

bug#43760: cuirass: fail to install with simple configuration

2020-10-02 Thread Mathieu Othacehe
>> https://issues.guix.gnu.org/43757 > > I do :( At least it's reproducible. Using one input works, but using more than one input causes an exception during git fetching. It seems to be broken since at least a few months. It makes me think that we need a stronger test suite for Cuirass. Thanks

bug#43757: cuirass: Fail to fetch "guix" input.

2020-10-03 Thread Mathieu Othacehe
Hello, > ERROR: In procedure make-stack: > Attempt to suspend fiber within continuation barrier Turns out "par-map" call does not seem suspendable, even though I'm not sure why. Maybe Ludo can help here :) ? This is fixed with: 761443bca6178b4ac299a8bd368d1cac4abda5f8. Thanks, Mathieu

bug#43760: cuirass: fail to install with simple configuration

2020-10-03 Thread Mathieu Othacehe
Hello Jonathan, >> You will then probably hit this one: >> >> https://issues.guix.gnu.org/43757 > > I do :( After fixing this issue, see: 761443bca6178b4ac299a8bd368d1cac4abda5f8, I'm able to successfully evaluate and build your specification. I still need to update the Cuirass package accordi

bug#43757: cuirass: Fail to fetch "guix" input.

2020-10-03 Thread Mathieu Othacehe
Hey Ludo, > ‘par-map’ is implemented in terms of futures, and futures use their own > thread pool. What’s likely to block is ‘touch’: it essentially waits on > a condition variable, which Fibers cannot interrupt. I see, thanks for explaining. > Why not just replace ‘par-map’ with ‘map’? That

bug#43760: cuirass: fail to install with simple configuration

2020-10-03 Thread Mathieu Othacehe
> Would be great :) Done with b3f5402d2d506271857d2c987b79b741d4b0ec33. Mathieu

bug#43564: cuirass: Contention while registering new builds.

2020-10-04 Thread Mathieu Othacehe
Hello, > I'm trying locally to spawn database workers dedicated to registration, > that execute the evaluation queries in a single batch. Let's see if it > helps. I added 4 registration dedicated database workers and removed from registration all accesses to the store. Now the evaluation return

bug#43791: cuirass: Search queries are too slow.

2020-10-04 Thread Mathieu Othacehe
Hello, Search queries can take a long time to complete. This query took 658.67 seconds to complete: --8<---cut here---start->8--- SELECT * FROM ( SELECT Builds.rowid, Builds.timestamp, Builds.starttime, Builds.stoptime, Builds.log, Builds.status, Builds.job

bug#43791: cuirass: Search queries are too slow.

2020-10-05 Thread Mathieu Othacehe
> However, searching for 'hurd-barebones.qcow2%' also skips the index. That's because the index was not created in case insensitive mode. I dropped support for searching with "^" and "$" characters as this is not compatible with using an index and hence way too slow. The new search behaviour i

bug#43564: cuirass: Contention while registering new builds.

2020-10-05 Thread Mathieu Othacehe
Hey Ludo, > Isn’t that the real problem, that we’re doing one transaction per > derivation? Is it really better in term of performance to send batch of queries within a single transaction? I haven't tried it yet. I think that the real bottleneck was having N fibers fighting over 4 workers to e

bug#43826: cuirass: build query is too slow.

2020-10-06 Thread Mathieu Othacehe
Hello, When "db-get-builds" is called with a limit set to N, at least ~2N queries are executed. First the main query returns the build list, then build outputs and build products are searched. All of this should be combined in a unique query to minimize overhead. The limit should also by restri

bug#43826: cuirass: build query is too slow.

2020-10-07 Thread Mathieu Othacehe
> All of this should be combined in a unique query to minimize > overhead. The limit should also by restricted to at most 1000 builds for > instance. Fixed with cb2c4e3d8f7eda187adf6da1fc35aef838c49828. Mathieu

bug#43850: cuirass: inconsistent SQL queries execution time.

2020-10-07 Thread Mathieu Othacehe
Hello, Over the last few weeks I made sure that all Cuirass SQL queries were using indexes. As the "Builds" and "Outputs" tables can be really large, having queries covered by indexes is imperative for consistent queries duration. However, I observed that some queries have inconsistent duration

bug#43905: Acknowledgement (cuirass: uncaught exception 'read-error' in 'metrics' fiber)

2020-10-12 Thread Mathieu Othacehe
Hello Jonathan, > It happens when scamping around in the database: > update Specifications set proc_args='((subset "htop" "icedove") (systems > "x86_64-linux))'; You mean it happens only when modifying 'proc_args' while Cuirass is running? Could you share your specifications so that I can try t

bug#43850: cuirass: inconsistent SQL queries execution time.

2020-10-14 Thread Mathieu Othacehe
Hello, I pushed and deployed several patches that: - update metrics in a single transaction - register builds in a single transaction - use a single write database worker, queuing queries and submitting them by batches (in a single transaction). - optimize some SQLite parameters (decrease WAL s

bug#44000: Guile-Git cross-compiled to i586-pc-gnu gets bytestructures wrong

2020-10-15 Thread Mathieu Othacehe
Hey Ludo, > I confirm that the attached patch for Bytestructures solves the problem > (running ‘guix pull’ in my childhurd :-)). Nice catch! I guess we had the same issue when cross-compiling for armhf from x86_64. Thanks, Mathieu

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

2020-10-18 Thread Mathieu Othacehe
Hello Miguel & Brendan, > AFAIU from the code, the first partition should not be created by the > installer but it won't be removed if it was found on the disk > beforehand. Tomorrow I'll give a try at the full installation process, > I must have overlooked something if it still being created i

bug#44054: [1.2 installer]: Kernel panic when attempting to reboot after install.

2020-10-18 Thread Mathieu Othacehe
Hello Brendan, > After the install completed I pulled out the USB as per instruction on the > screen, and pressed the Reboot button. The system then Kernel panic with the > following messages: > > https://brendan.scot/p/IMG_20201018_140343.png > > Sorry I don't have it in textual form. Is the i

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

2020-10-19 Thread Mathieu Othacehe
Hello Brendan, > I feel the installer should not care what was on the drive to begin with, it > should just > format the drive how it sees fit. Why should the existing contents of the > drive > affect the outcome when we are reformatting everything anyway? That's the case except for the ESP par

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

2020-10-19 Thread Mathieu Othacehe
Hello Miguel, > Is there any other issue remaining? It doesn't modify this logic at > all. Should I apply this patch to master then? Yes, your patch LGTM. Please make sure that system tests are still working by running: --8<---cut here---start->8--- make c

bug#43850: cuirass: inconsistent SQL queries execution time.

2020-10-20 Thread Mathieu Othacehe
Hello, > this should improve the situation, even if I still observe some > inconsistent execution duration. I tried to use the two following pragma: --8<---cut here---start->8--- PRAGMA synchronous = OFF PRAGMA mmap_size = 10737418240 --8<---cut

bug#44101: Unable to use /dev/disk/by-id/ symlinks with u-boot and guix system reconfigure

2020-10-22 Thread Mathieu Othacehe
guring, the first method only is used. The attached patch tries to fallback to the second method if the first one is not defined. WDYT? Thanks, Mathieu >From 7fd5fb804317df5af5e14a6a95179acb3c8ac598 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe Date: Wed, 21 Oct 2020 10:42:50 +0200 Subject:

bug#44054: [1.2 installer]: Kernel panic when attempting to reboot after install.

2020-10-22 Thread Mathieu Othacehe
Hey, > Mathieu, are there other file system operations happening beyond that? The final "sync" call is done just after the user clicks on "Reboot", so if the flash drive is removed before clicking on "Reboot" I guess things can go wrong. I'll see if I can reproduce this issue. Thanks, Mathie

bug#43565: cuirass: Fibers scheduling blocked.

2020-10-22 Thread Mathieu Othacehe
Hey Ludo! > ‘process-build-log’ in Cuirass uses ‘read-line/non-blocking’ to read a > line from the log port of ‘build-derivations&’. If that really is > non-blocking (and I think it is), then we should be fine? > > We should attach GDB to Cuirass next time to see what’s blocking. Cuirass is cu

bug#43850: cuirass: inconsistent SQL queries execution time.

2020-10-22 Thread Mathieu Othacehe
Hello, > I have now copied the database to a tmpfs mounted directory to make sure > that those inconsistent duration are only caused by the I/O pressure on > berlin. This helps a lot. The Cuirass web service has been running smooth since two days, without any inconsistent query times. I'm cons

bug#44101: Unable to use /dev/disk/by-id/ symlinks with u-boot and guix system reconfigure

2020-10-23 Thread Mathieu Othacehe
Hello, > I don't quite understand why that would be the issue here; guix system > reconfigure works fine when /dev/mmcblkN is specified target in the > system config.scm, just not when the target is /dev/disk/by-id/... I don't think it works fine with /dev/mmcblkN. I think the bootloader config

bug#44101: Unable to use /dev/disk/by-id/ symlinks with u-boot and guix system reconfigure

2020-10-23 Thread Mathieu Othacehe
s kind of files. I'll try > to reproduce it locally. This is because the O_EXCL flag was passed, which causes open to fail if the block device is mounted. The revised patch attached should fix it. Thanks, Mathieu >From 780327ebb0db74ca4cc43d26ba7cf945d64c7d30 Mon Sep 17 00:00:00 2001

bug#43565: cuirass: Fibers scheduling blocked.

2020-10-26 Thread Mathieu Othacehe
Hey! Many thanks for your help, you rock! > But does Cuirass create file descriptors as O_NONBLOCK? This has to be > done explicitly, Fibers won’t do it for us. As it turns out, the answer > is no, in at least one important case: the connection to the daemon > (untested patch below). > > Whil

bug#43850: cuirass: inconsistent SQL queries execution time.

2020-10-27 Thread Mathieu Othacehe
Hello Chris, > I think Ricardo mentioned that the machine running Cuirass uses an SSD > for the root filesystem, so moving the database there may help? Looks like the database was already on the SSD before my tmpfs experiment. --8<---cut here---start->8--- m

bug#43565: cuirass: Fibers scheduling blocked.

2020-10-27 Thread Mathieu Othacehe
Hey, > Yeah please go ahead if you want, or let me know if you’d rather let me > apply it. I applied your patch, thanks! I'm closing this one, because there's nothing much that can be done right now. Thanks, Mathieu

bug#43850: cuirass: inconsistent SQL queries execution time.

2020-10-27 Thread Mathieu Othacehe
> I don't really get why I/O pressure on /dev/sdb could impact /dev/sda. Turns out /tmp is mounted on /dev/sda, so all the building and ISO production are first written on /dev/sda before being copied to the store in /dev/sdb. Reducing the build activity of berlin, as Ludo proposed should help

bug#43850: cuirass: inconsistent SQL queries execution time.

2020-11-01 Thread Mathieu Othacehe
Hey Maxim, > Perhaps we should run 'vacuum' when invoking 'guix gc' or at some other > key places (where lots of data gets removed from the DB). There's also > the auto_vacuum PRAGMA, which is not enabled currently: Vacuuming periodically seems important, however, it didn't resulted in noticea

bug#43879: Problem with graphical installer

2020-11-01 Thread Mathieu Othacehe
Hello Miguel, > + ;; TRANSLATORS: The ~{ and ~} format specifiers are used to iterate the > list > + ;; of device names of the user partitions that will be formatted. > + (run-confirmation-page (format #f (G_ "We are about to write the > configured \ > +partition table to the disk and forma

bug#44370: Cuirass fails building (some) packages from a custom-packages input

2020-11-01 Thread Mathieu Othacehe
Hello David, > ice-9/boot-9.scm:1667:16: In procedure raise-exception: > ERROR: > 1. &evaluation-error: > name: "my-pkgs" > id: Thanks for the bug report. Could you please send the associated log trace in /var/log/cuirass/evaluations/xxx.gz? Mathieu

bug#44370: Cuirass fails building (some) packages from a custom-packages input

2020-11-02 Thread Mathieu Othacehe
Hello David, > error: specifications->manifest: unbound variable This error is caused by the evaluation of "packages/manifest.scm" that misses some includes. > guix/inferior.scm:247:2: ERROR: > 1. &inferior-exception: > arguments: (quit 1) > inferior: #< pid: pipe socket: #

bug#43565: cuirass: Fibers scheduling blocked.

2020-11-02 Thread Mathieu Othacehe
Hey, > Yeah please go ahead if you want, or let me know if you’d rather let me > apply it. I finally reverted this patch that causes the following error: --8<---cut here---start->8--- 2020-11-02T11:05:08 fatal: uncaught exception 'wrong-type-arg' in 'build'

bug#44390: Cuirass reports succesful builds as failed in /var/log/cuirass.log and cuirass-web reports them as scheduled

2020-11-02 Thread Mathieu Othacehe
Hello David, > /gnu/store/4ijsrmkb02z2s18bgkw5kyvf4kpwsdyg-python-pydotplus-2.0.2 > 2020-11-02T15:42:20 success: 0, fail: 10 > 2020-11-02T15:42:34 heap: 44.49 MiB; threads: 11; file descriptors: 4 Thanks for the bug report. This bug is the result of some recent optimizations. The database reque

bug#44370: Cuirass fails building (some) packages from a custom-packages input

2020-11-02 Thread Mathieu Othacehe
Hey Ludo, > Now, in Guix proper, the ‘license’ field should always be either a > or a list of records. The original code would > “enforce” that (by failing hard if a package doesn’t follow the rule > :-)), which I think is good. You are right, missed that. > Now, why is David’s use case inf

bug#44399: Cuirass problem with building from manifest file

2020-11-02 Thread Mathieu Othacehe
Hello David, > (("/gnu/store/ps30mn7xq0ymcrdzlwim8pkv1czgv2zf-profile.drv") 0) > ERROR: In procedure read: > In procedure scm_lreadr: #:16:634: Unknown # object: #\< This is probably a consequence of me breaking things today. I restored the original handling of package licenses with b5f2a035.

  1   2   3   4   5   6   7   >