bug#43818: [PATCH] packages: Fix a bug in %patch-path.

[Ludovic Courtès]

Hi,

Maxim Cournoyer  skribis:

> Fixes .
>
> Having the Guix root trailed by a slash in the GUILE_LOAD_PATH would
> previously cause %patch-path to not return an entry with the patches
> sub-directory, leading to errors such as:
>
>While executing meta-command:
>ERROR:
>  1. :
>  format: "~a: patch not found\n"
>  arguments: ("icecat-use-older-reveal-hidden-html.patch")
>
> * gnu/packages.scm (strip-trailing-slash): New procedure.
> (%patch-path): Use it to strip any trailing slash from the %load-path entries.
> ---
>  gnu/packages.scm | 9 -
>  1 file changed, 8 insertions(+), 1 deletion(-)

How can this be reproduced, outside Geiser?  I tried:

  ./pre-inst-env sh -c 'GUILE_LOAD_PATH=$PWD/:$GUILE_LOAD_PATH guix build 
icecat -n'

with an explicit trailing slash, but it doesn’t trigger the problem.

Thanks,
Ludo’.

bug#44194: Cuirass ignoring proc_args on Berlin

[Ludovic Courtès]

Hi Marius,

Marius Bakke  skribis:

> Cuirass is currently evaluating all of 'core-updates':
>
>   https://ci.guix.gnu.org/jobset/core-updates-core-updates
>
> This is despite (subset . core) in "proc_args".

It should be (subset . "core"), with quotes.

I’ve changed it now, let’s see!

Ludo’.

bug#43850: cuirass: inconsistent SQL queries execution time.

[Ludovic Courtès]

Hi,

Mathieu Othacehe  skribis:

>> I have now copied the database to a tmpfs mounted directory to make sure
>> that those inconsistent duration are only caused by the I/O pressure on
>> berlin.
>
> This helps a lot. The Cuirass web service has been running smooth since
> two days, without any inconsistent query times.

Interesting.

> I'm considering using a tmpfs backed database for good. The problem is
> that we would need a save/restore mechanism in case Berlin
> reboots.

Hmm sounds risky, no?

I wonder if we could instead ensure no I/O-intensive workload runs that
machine.  I’m thinking in particular of the derivations that produce
ISO/qcow images that are not offloaded but maybe should.

WDYT?  Do you think that’d be enough?  Or is tmpfs our only hope?

Thanks,
Ludo’.

bug#44104: [patch] guix publish: provide server's signing key.

[Ludovic Courtès]

Hi,

"Amar M. Singh"  skribis:

> From fe0098082289a2350002da4e2438c41f6385b3e2 Mon Sep 17 00:00:00 2001
> From: Amar Singh 
> Date: Wed, 21 Oct 2020 02:02:02 +0530
> Subject: [PATCH] guix publish: Provide server's signing key.
>
> Published at $(guix-publish-uri)/signing-key.pub
>
> Knock yourselves out! Best in industry.
>
> Signed-off-by: Amar Singh 

I tweaked the commit log and committed it.

Thanks!

Ludo’.

bug#39260: bug#41409: “guix pull” should show a progress bar for channel updates

[Ludovic Courtès]

Ludovic Courtès  skribis:

> And now the Guix side of it:
>
>   https://issues.guix.gnu.org/43968

Merged in 298f9d29d6c26e408a90d08d147d926aa6f81ab3, closing!

Ludo’.

bug#44196: [PATCH 4/3] system: Fix dependency for grub.cfg generation.

[Miguel Ángel Arruga Vivas]

One extra bit :-)

* The keymap was depending on the exported package grub, not on the one
provided by the bootloader configuration, so e.g. systems that use
grub-efi also ended up with grub in their dependency graph.  This patch
solves that issue.

>From ae2d83f91d522b5b0edfe1abbe88dec0a797235c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
 
Date: Sun, 25 Oct 2020 10:13:46 +0100
Subject: [PATCH v3 4/5] system: Fix dependency for grub.cfg generation.

* gnu/bootloader/grub.scm (eye-candy)[font-file]: Use the bootloader
package provided with the configuration.
---
 gnu/bootloader/grub.scm | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm
index b610ecbfe5..8636e9c690 100644
--- a/gnu/bootloader/grub.scm
+++ b/gnu/bootloader/grub.scm
@@ -191,9 +191,11 @@ fi~%"
  (symbol->string (assoc-ref colors 'bg)
 
   (define font-file
-(normalize-file (file-append grub "/share/grub/unicode.pf2")
-store-mount-point
-store-directory-prefix))
+(let* ((bootloader (bootloader-configuration-bootloader config))
+   (grub (bootloader-package bootloader)))
+  (normalize-file (file-append grub "/share/grub/unicode.pf2")
+  store-mount-point
+  store-directory-prefix)))
 
   (define image
 (normalize-file (grub-background-image config)
-- 
2.28.0

bug#25305: bug#37851: Grub installation only checks for encrypted /boot folder

[Miguel Ángel Arruga Vivas]

Hello!

Ludovic Courtès  writes:
> Does that cause GRUB to mount all the LUKS partitions it was aware of at
> installation time, or does it cause it to scan all the partitions in
> search of a LUKS signature?
>
> In the latter case that wouldn’t be great, but in the former case it
> sounds like we could go ahead (well, with a comment above explaining
> what this does.  :-)).

Sorry for this huge delay, but I have this patch for this.  It
includes a test case, even though I have been suffering a lot until I
noticed that OCR was returning garbage and I was trying to be too
specific, so I've left it as basic as I could.

I add Mathieu to the loop to bring more eyes over it, I'm open to any
suggestion. :-)

Happy hacking!
Miguel

PS: It should apply on top of master too, but I tested it on top of some
other grub.cfg fixes, I'll send a new version if there is any problem
with this.

From d40f0a26afef194e7e68906ba793ca0ffac6da5f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
 
Date: Sun, 25 Oct 2020 16:31:17 +0100
Subject: [PATCH v3 5/5] system: Allow separated /boot and encrypted root.

* gnu/bootloader/grub.scm (grub-configuration-file): New parameter
store-crypto-devices.
[crypto-devices]: New helper function.
[builder]: Use crypto-devices.
* gnu/machine/ssh.scm (roll-back-managed-host): Use
boot-parameters-store-crypto-devices to provide its contents to the
bootloader configuration generation process.
* gnu/tests/install.scm (%encrypted-root-not-boot-os,
%encrypted-root-not-boot-os): New os declaration.
(%encrypted-root-not-boot-installation-script): New script, whose contents
were initially taken from %encrypted-root-installation-script.
(%test-encrypted-root-not-boot-os): New test.
* gnu/system.scm (define-module): Export
operating-system-bootoader-crypto-devices and
boot-parameters-store-crypto-devices.
(): Add field store-crypto-devices.
(read-boot-parameters): Parse store-crypto-devices field.
[uuid-sexp->uuid]: New helper function extracted from
device-sexp->device.
(operating-system-bootloader-crypto-devices): New function.
(operating-system-bootcfg): Use
operating-system-bootloader-crypto-devices to provide its contents to
the bootloader configuration generation process.
(operating-system-boot-parameters): Add store-crypto-devices to the
generated boot-parameters.
(operating-system-boot-parameters-file): Likewise to the file with
the serialized structure.
* guix/scripts/system.scm (reinstall-bootloader): Use
boot-parameters-store-crypto-devices to provide its contents to the
bootloader configuration generation process.
* tests/boot-parameters.scm (%default-store-crypto-devices): New
variable.
(%grub-boot-parameters, test-read-boot-parameters): Use
%default-store-crypto-devices.
(tests store-crypto-devices): New tests.
---
 gnu/bootloader/grub.scm   |  19 ++-
 gnu/machine/ssh.scm   |   3 ++
 gnu/system.scm|  57 -
 gnu/tests/install.scm | 103 ++
 guix/scripts/system.scm   |   2 +
 tests/boot-parameters.scm |  29 ++-
 6 files changed, 208 insertions(+), 5 deletions(-)

diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm
index 8636e9c690..c6e7d3fd6d 100644
--- a/gnu/bootloader/grub.scm
+++ b/gnu/bootloader/grub.scm
@@ -4,7 +4,7 @@
 ;;; Copyright © 2017 Leo Famulari 
 ;;; Copyright © 2017, 2020 Mathieu Othacehe 
 ;;; Copyright © 2019, 2020 Jan (janneke) Nieuwenhuizen 
-;;; Copyright © 2019 Miguel Ángel Arruga Vivas 
+;;; Copyright © 2019, 2020 Miguel Ángel Arruga Vivas 
 ;;; Copyright © 2020 Maxim Cournoyer 
 ;;; Copyright © 2020 Stefan 
 ;;;
@@ -361,6 +361,7 @@ code."
   (locale #f)
   (system (%current-system))
   (old-entries '())
+  store-crypto-devices
   store-directory-prefix)
   "Return the GRUB configuration file corresponding to CONFIG, a
  object, and where the store is available at
@@ -413,6 +414,21 @@ menuentry ~s {
   (string-join (map string-join '#$modules)
"\n  module " 'prefix))
 
+  (define (crypto-devices)
+(define (crypto-device->cryptomount dev)
+  (if (uuid? dev)
+  #~(format port "cryptomount -u ~a~%"
+;; cryptomount only accepts UUID without the hypen.
+#$(string-delete #\- (uuid->string dev)))
+  ;; Other type of devices aren't implemented.
+  #~()))
+(let ((devices (map crypto-device->cryptomount store-crypto-devices))
+  ;; XXX: Add luks2 when grub 2.06 is packaged.
+  (modules #~(format port "insmod luks~%")))
+  (if (null? devices)
+  devices
+  (cons modules devices
+
   (define (sugar)
 (let* ((entry (first all-entries))
(device (menu-entry-device entry))
@@ -469,6 +485,7 @@ keymap ~a~%" 

bug#40558: (no subject)

[Ricardo Wurmus]

elaexuo...@wilsonb.com writes:

> With the patch to texlive-amsfonts the above typesets just fine;
> however, metafont ends up generating cmmi10.657pk and cmr10.657pk font
> files. Is this expected? Typsetting it from the texlive installation
> of my foreign distro doesn't call out to metafont at all.

This is a problem with the modular TeX Live packages.  The pk files are
bitmap fonts.  I found that adding texlive-cm-super to the union helps
in that the bitmap variants of the CM fonts will not be generated any
more.  More font packages may be needed in the union to prevent TeX from
falling back to bitmap fonts in other cases.

In any case, that’s unrelated to Jelle’s patch, which looks fine to me.

-- 
Ricardo

bug#44196: [PATCH 3/3 v3] system: Do not depend on locale folder generated by grub-install.

[Miguel Ángel Arruga Vivas]

This version doesn't modify grub package at all, and it only generates
the folder when needed, and fixes a bug in Grub 2.04 that I didn't
noticed until I started playing with the installer:
https://savannah.gnu.org/bugs/?57889

Happy hacking!
Miguel
>From 770532b931c312edb9f9a075d86dc132b804dc0b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
 
Date: Sat, 24 Oct 2020 20:36:21 +0200
Subject: [PATCH v3 3/5] system: Do not depend on locale folder generated by
 grub-install.

* gnu/bootloader/grub.scm (define-module): Use (guix packages).
(grub-locale-folder): New computed folder.
(grub-configuration-file)[locale-config]: Use grub-locale-folder only when a
locale is needed.
---
 gnu/bootloader/grub.scm | 50 -
 1 file changed, 39 insertions(+), 11 deletions(-)

diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm
index f1479024e6..b610ecbfe5 100644
--- a/gnu/bootloader/grub.scm
+++ b/gnu/bootloader/grub.scm
@@ -25,6 +25,7 @@
 
 (define-module (gnu bootloader grub)
   #:use-module (guix build union)
+  #:use-module (guix packages)
   #:use-module (guix records)
   #:use-module (guix store)
   #:use-module (guix utils)
@@ -142,6 +143,25 @@ file with the resolution provided in CONFIG."
 (image->png image #:width width #:height height))
(_ #f)
 
+(define (grub-locale-folder grub)
+  "Generate a folder with the locales from GRUB."
+  (define builder
+#~(begin
+(use-modules (ice-9 ftw))
+
+(define (copy-mo lang-path)
+  (let ((file (string-append lang-path "/LC_MESSAGES/guix.mo"))
+(mo   (string-append (basename lang-path) ".mo")))
+(when (file-exists? file)
+  (copy-file file mo
+
+(let* ((locale (string-append #$grub "/share/locale"))
+   (langs  (scandir locale)))
+  (mkdir #$output)
+  (chdir #$output)
+  (for-each copy-mo langs
+  (computed-file "grub-boot-locale" builder))
+
 (define* (eye-candy config store-device store-mount-point
 #:key store-directory-prefix port)
   "Return a gexp that writes to PORT (a port-valued gexp) the 'grub.cfg' part
@@ -402,18 +422,26 @@ menuentry ~s {
  #:port #~port)))
 
   (define locale-config
-#~(let ((locale #$(and locale
-   (locale-definition-source
-(locale-name->definition locale)
-(when locale
-  (format port "\
+(let* ((entry (first all-entries))
+   (device (menu-entry-device entry))
+   (mount-point (menu-entry-device-mount-point entry))
+   (bootloader (bootloader-configuration-bootloader config))
+   (grub (bootloader-package bootloader))
+   (locale-dir (grub-locale-folder grub)))
+
+  #~(let ((locale #$(and locale
+ (locale-definition-source
+  (locale-name->definition locale
+  (locale-dir #$(and locale (grub-locale-folder grub
+  (when locale
+(format port "\
 # Localization configuration.
-if search --file --set boot_partition /grub/grub.cfg; then
-set locale_dir=(${boot_partition})/grub/locale
-else
-set locale_dir=/boot/grub/locale
-fi
-set lang=~a~%" locale
+search --file --set ~a
+set locale_dir=~a
+set lang=~a~%"
+(string-append locale-dir "/e...@quot.mo")
+locale-dir
+locale)
 
   (define keyboard-layout-config
 (let* ((layout (bootloader-configuration-keyboard-layout config))
-- 
2.28.0

bug#42941: qgis: plugin installation fails with python error

[Jonathan Brielmaier]

Fixed in 542dc0ee4f2385c4d71f65fe3a36f4fe9eb68b9a.

bug#44243: problems with cyrillic in games

[Adam Kandur via Bug reports for GNU Guix]

hi everyone! i cannot type cyrillic chars in games (for example in xonotic). it 
refuses to print out any chars after i change layout

bug#44244: meson build-system has no cross-compilation support

[Danny Milosavljevic]

Hi,

meson build-system has no cross-compilation support.

$ ./pre-inst-env guix build --target=arm-linux-gnuhf gobject-introspection
guix build: error: 
/home/dannym/src/guix-master/guix/gnu/packages/glib.scm:419:2: 
gobject-introspection@1.62.0: build system `meson' does not support cross builds

I need that for debugging https://github.com/spk121/guile-gi/issues/96 .

It would be really good if we had support for meson cross-builds.


pgpIoedp1nI0b.pgp
Description: OpenPGP digital signature

bug#24066: icecat "mailto" handler does not work - and cannot be reconfigured by user

[Jonathan Brielmaier]

This issue annoys me, so I searched for a workaround which I found :)

In Icecat go to about:preferences -> General -> Applications (only a
header): Set mailto to `Use other` and set it to
`/run/current-system/profile/bin/icedove`.

$ grep mailto
/gnu/store/3zxx08qfl9q6d9s80p58521y9x3ni3p1-icedove-78.3.3/share/applications/icedove.desktop
MimeType=x-scheme-handler/mailto;

That seems correct. xdg-open mailto:f...@gnu.org works for me always, so
it really seems a Icecat/Firefox thing...

bug#43565: cuirass: Fibers scheduling blocked.

[Ludovic Courtès]

Hello!

Mathieu Othacehe  skribis:

>> But does Cuirass create file descriptors as O_NONBLOCK?  This has to be
>> done explicitly, Fibers won’t do it for us.  As it turns out, the answer
>> is no, in at least one important case: the connection to the daemon
>> (untested patch below).
>>
>> While GC is running, Cuirass typically sends ‘build-derivations’ RPCs
>> and they block until the GC lock is released.  That can lead to the
>> situation above: a bunch of threads blocked in ‘read’ from their daemon
>> socket, waiting for the RPC reply.  OTOH, ‘build-derivations’ RPCs are
>> made from a fresh thread created by ‘build-derivations&’.
>
> While I agree not opening file descriptors with O_NONBLOCK is an issue,
> build-derivations is called in a separate thread. Blocking this separate
> thread should not block the fibers.

Agreed.

> Now the question is why there's no fetching while the GC is running? The
> answer is that "latest-repository-commit" called by "fetch-input" will
> block the only fiber dedicated to fetching. Having multiple fibers
> trying to fetch wouldn't solve anything because fetching requires some
> building from the daemon.

Exactly: when the GC lock is taken, ‘latest-repository-commit’ makes an
‘add-to-store’ RPC, and that RPC blocks.  Thus the whole fetch fiber is
blocked.

The patch should address this case.  That said, nothing useful happens
anyway when the GC lock is held, so it wouldn’t have any practical
effect.

I believe there are other cases where RPCs can be slow, for example when
there’s contention on the sqlite database.  Perhaps that could help a
bit there although again, it’s a situation where nothing useful can
happen.

> Long story short, I think we can apply your patch that can be useful to
> prevent fibers talking directly to the daemon to block, even though it
> won't help for this particular hang, that will only be fixed the GC time
> will be reduced to something more acceptable.

Yeah please go ahead if you want, or let me know if you’d rather let me
apply it.

Thanks!

Ludo’.

bug#44030: guix import pypi foo@1.2.3 breaks

[zimoun]

Hi,

On Sun, 25 Oct 2020 at 23:50, Lulu  wrote:
> I just sent in a patch to fix this for the pypi importer, although
> ideally we'd want versioning support for all importers with a uniform
> syntax.

The uniform syntax is ’@’.  However, it is importer by importer (case by
case) since the external API is different.  AFAIU.


Speaking about “guix import pypi”, the “--recursive” options still
ungracefully fails; even with you patch.


> I poked and prodded at the RubyGems API to see if it can do what PyPI
> can. It's unfortunately much more limited: API v1 provides a method
> for querying all versions of a gem [1] and v2 provides a method for
> querying specific versions of a gem [2], although neither of them
> provides information about dependency versions, so we can't rely on it
> for recursive imports (since it would try to import latest versions of
> all dependencies). There's a method that fetches dependency and
> versioning info for all versions of all given gems but (probably due
> to the verbosity of JSON/YAML) it's in binary Ruby serialisation
> format [3]. I can try to hack a parser that surgically extracts
> dependency info from a given gem version. What do you think?
>
> ELPA runs into a similar problem in that it provides tarballs/files
> for older versions but dependency info is only provided in the repo
> file. (MELPA tries to directly peg all packages to their respective
> Git repos' trunks.)
>
> I presume it's much simpler with the gnu importer, as it's only a
> matter of pointing the FTP fetch in the right direction, although I
> couldn't confirm it, as the gnu importer doesn't work for me since my
> ISP blocks PGP keyserver ports.
>
> I need to take a closer look at CRAN, CPAN, TeXLive and opam.

Hum?  You are discussing this bug [1], right?  So please could you
discuss overthere.

1: 


All the best,
simon

bug#44030: [PATCH] guix: import: Add versioning syntax to pypi importer.

[zimoun]

Hi,

Thank you for working on this.

On Sun, 25 Oct 2020 at 23:04, Lulu  wrote:
> Ah, heck, I used an old revision of the file when I generated the diff.
> Sorry about that. Here's the properly working patch:

That’s fine.  Usually to ease the reading, it is a good habit to
increase the version of the patch, (see the option ’reroll-count’ of
git-format-patch). 


However, the 2 patches are not proper commit, right?  You can locally
commit your changes and send them.  Basically, it means the same thing
but including the commit message.  Here it looks like:

--8<---cut here---start->8---
import: pypi: Allow ’@’ for specifying the version.

Fixes .

* guix/import/pypi.json (pypi-fetch): Allow ’@’ for specifying the
  version.
--8<---cut here---end--->8---

Well, I let you find more inspiration in previous commit messages. ;-)


> diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
> index 15116e349d..1ec1ecbfa1 100644
> --- a/guix/import/pypi.scm
> +++ b/guix/import/pypi.scm
> @@ -118,13 +118,15 @@
>  
>  (define (pypi-fetch name)
>"Return a  record for package NAME, or #f on failure."
> -  (and=> (json-fetch (string-append "https://pypi.org/pypi/; name "/json"))
> - json->pypi-project))
> +  ;; Convert @ in package name to / to access the correct URL.
> +  (let ((versioned-name (string-join (string-split name #\@) "/")))
> +(and=> (json-fetch (string-append "https://pypi.org/pypi/; 
> versioned-name "/json"))
> +   json->pypi-project)))

If you feel adventurous, you could try to add a test in the file
“tests/pypi.scm”. :-)

>  ;; For packages found on PyPI that lack a source distribution.
>  (define-condition-type  
>missing-source-error?
> -  (package  missing-source-error-package))
> +  (package missing-source-error-package))

Why is this line modified?


>  (define (latest-source-release pypi-package)
>"Return the latest source release for PYPI-PACKAGE."
> @@ -371,7 +373,7 @@ be extracted in a temporary directory."
>   (invoke "tar" "xf" archive "-C" dir)))
> (let ((requires.txt-files
>(find-files dir (lambda (abs-file-name _)
> - (string-match "\\.egg-info/requires.txt$"
> +(string-match "\\.egg-info/requires.txt$"

Idem.


All the best,
simon

bug#43960:

[zimoun]

Hi,

On Sun, 25 Oct 2020 at 23:33, Miguel Ángel Arruga Vivas  
wrote:

> I'm hitting this while testing some patches too.

Well, I do not know if it is related to the initial bug.


> [env]$ make check-system TESTS=encrypted-root-and-boot-os
> [...]
> successfully built 
> /gnu/store/32qhcfz3amwq9zisqnagdvrlpl3lr82a-guix-system-tests-modules.drv
> building 
> /gnu/store/dsq1i6hfhqnx25gsa5m00q9a7d1vshql-module-import-compiled.drv...
> [ 1/88] Loading './gnu/build/accounts.scm'...
> [ 2/88] Loading './gnu/build/file-systems.scm'...
> ;;; WARNING: loading compiled file 
> /gnu/store/y1gzzlm1da2i5j60nws3iqrgnck6bhnn-guix-1.1.0-30.875c01f/lib/guile/3.0/site-ccache/guix/records.go
>  failed:
> ;;; In procedure load-thunk-from-memory: incompatible bytecode version
> ;;; WARNING: loading compiled file 
> /gnu/store/y1gzzlm1da2i5j60nws3iqrgnck6bhnn-guix-1.1.0-30.875c01f/lib/guile/3.0/site-ccache/guix/combinators.go
>  failed:
> [...]
>
> It compiles the modules afterwards (my test fails too, but it shouldn't
> be related to this).
>
> How could I check what's happenning?

Sometimes, I have remarked that “guix environment -C guix” then
“./bootstrap && ./configure” and “make” plus some tweaks; days later, I
re-rerun “guix environment -C guix” and directly run some “make foo” and
there some “incompatible” happens.  Therefore, I re-run “./bootstrap &&
./configure”.  And « days later » means that I did “guix pull” between
the 2 “guix environment”.

I do not know.


All the best,
simon

bug#43565: cuirass: Fibers scheduling blocked.

[Mathieu Othacehe]

Hey!

Many thanks for your help, you rock!

> But does Cuirass create file descriptors as O_NONBLOCK?  This has to be
> done explicitly, Fibers won’t do it for us.  As it turns out, the answer
> is no, in at least one important case: the connection to the daemon
> (untested patch below).
>
> While GC is running, Cuirass typically sends ‘build-derivations’ RPCs
> and they block until the GC lock is released.  That can lead to the
> situation above: a bunch of threads blocked in ‘read’ from their daemon
> socket, waiting for the RPC reply.  OTOH, ‘build-derivations’ RPCs are
> made from a fresh thread created by ‘build-derivations&’.

While I agree not opening file descriptors with O_NONBLOCK is an issue,
build-derivations is called in a separate thread. Blocking this separate
thread should not block the fibers.

For instance, the following program:

--8<---cut here---start->8---
(use-modules (fibers)
 (ice-9 threads))

(run-fibers
 (lambda ()
   (spawn-fiber
(lambda ()
  (call-with-new-thread
   (lambda ()
 (read (car (pipe)))
   (spawn-fiber
(lambda ()
  (while #t
(format #t "alive~%")
(sleep 1)
 #:hz 10
 #:drain? #t)
--8<---cut here---end--->8---

keeps displaying "alive" even if the spawned thread is blocking. I guess
that's also what's happening in Cuirass because the log shows that some
fibers are scheduled while the GC is running.

Now the question is why there's no fetching while the GC is running? The
answer is that "latest-repository-commit" called by "fetch-input" will
block the only fiber dedicated to fetching. Having multiple fibers
trying to fetch wouldn't solve anything because fetching requires some
building from the daemon.

Long story short, I think we can apply your patch that can be useful to
prevent fibers talking directly to the daemon to block, even though it
won't help for this particular hang, that will only be fixed the GC time
will be reduced to something more acceptable.

Thanks,

Mathieu

bug#31302: `guix import json` doesn't handle inputs with the form 'package:output'

[zimoun]

Dear,

On Sun, 25 Oct 2020 at 20:12, Lulu  wrote:

> So, it seems like this bug was fixed somewhere along the line.

Yep, the bug is fixed.  Probably by these:

7cef499bb0 * import/json: json->code: Handle files with more than one 
definition.
c893432320 * import/json: Use json->code.
16dd764691 * import/json: Add json->scheme-file.


You can close the bug by sending to <31302-d...@debbugs.gnu.org> with
the explanations.


Thanks,
simon

bug#31302: `guix import json` doesn't handle inputs with the form 'package:output'

[Lulu]

The version hash of Guix I tried this on was 
`26f6bd0403ed20c1d26eca9d4f28a0da519a56ac'.

I used the example JSON file for the `hello' package provided in
"(guix)Invoking guix import". Then, based on the given example of glib, I tried
running `guix import json' with `glib', `glib:bin', `glib:doc', `glib:out'
separately appended to the `native-inputs' array of `hello.json', in order. It
produced an appropriate SXML structure in the `native-inputs` field each time:
("glib" (@ (gnu packages glib) glib-with-documentation))
for standalone `glib' and
("glib" (@ (gnu packages glib) glib-with-documentation) "bin")
for the rest, which appears to fit the package definition syntax.

When I tried with `glib:foo', where `foo' is an erroneous output, it yielded the
following error:
guix import: error: package `glib@2.62.6' lacks output `foo'.

So, it seems like this bug was fixed somewhere along the line.

--
Lulu

bug#44175: [optimization] Grafting is too slow

[Lars-Dominik Braun]

Hi,

> Yes, you’re right of course.  But I think in the example above Lars was
> reporting the run time of the ‘guix’ command when the graft itself is
> already built.  Just the extra code to compute the graft derivation (not
> actually building them) leads to x2 wall-clock time or so.
yes, Ludo is right. The numbers I reported are with grafts already built, warm 
caches and
local socket communication. So we’re only seeing the CPU-bound computation
here. Sorry for the confusion. I should have been more specific in the other
issue.

Cheers,
Lars



signature.asc
Description: PGP signature

bug#44030: guix import pypi foo@1.2.3 breaks

[Lulu]

I just sent in a patch to fix this for the pypi importer, although
ideally we'd want versioning support for all importers with a uniform
syntax.

I poked and prodded at the RubyGems API to see if it can do what PyPI
can. It's unfortunately much more limited: API v1 provides a method
for querying all versions of a gem [1] and v2 provides a method for
querying specific versions of a gem [2], although neither of them
provides information about dependency versions, so we can't rely on it
for recursive imports (since it would try to import latest versions of
all dependencies). There's a method that fetches dependency and
versioning info for all versions of all given gems but (probably due
to the verbosity of JSON/YAML) it's in binary Ruby serialisation
format [3]. I can try to hack a parser that surgically extracts
dependency info from a given gem version. What do you think?

ELPA runs into a similar problem in that it provides tarballs/files
for older versions but dependency info is only provided in the repo
file. (MELPA tries to directly peg all packages to their respective
Git repos' trunks.)

I presume it's much simpler with the gnu importer, as it's only a
matter of pointing the FTP fetch in the right direction, although I
couldn't confirm it, as the gnu importer doesn't work for me since my
ISP blocks PGP keyserver ports.

I need to take a closer look at CRAN, CPAN, TeXLive and opam.

--
Lulu

[1]: https://guides.rubygems.org/rubygems-org-api/#gem-version-methods
[2]: https://guides.rubygems.org/rubygems-org-api-v2/
[3]: https://guides.rubygems.org/rubygems-org-api/#misc-methods

bug#44030: [PATCH] guix: import: Add versioning syntax to pypi importer.

[Lulu]

Use @ as the unified versioning syntax, as per crate and
hackage importers, plus minor spacing fixes.


diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 15116e349d..add2a7ed7a 100644
--- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -118,13 +118,15 @@
 
 (define (pypi-fetch name)
   "Return a  record for package NAME, or #f on failure."
-  (and=> (json-fetch (string-append "https://pypi.org/pypi/; name "/json"))
- json->pypi-project))
+  ;; Convert @ in package name to / to access the correct URL.
+  (let ((versioned-name (string-join (string-split name #\@) "/")))
+(and=> (json-fetch (string-append "https://pypi.org/pypi/; name "/json"))
+   json->pypi-project)))
 
 ;; For packages found on PyPI that lack a source distribution.
 (define-condition-type  
   missing-source-error?
-  (package  missing-source-error-package))
+  (package missing-source-error-package))
 
 (define (latest-source-release pypi-package)
   "Return the latest source release for PYPI-PACKAGE."
@@ -371,7 +373,7 @@ be extracted in a temporary directory."
  (invoke "tar" "xf" archive "-C" dir)))
(let ((requires.txt-files
   (find-files dir (lambda (abs-file-name _)
-   (string-match "\\.egg-info/requires.txt$"
+(string-match "\\.egg-info/requires.txt$"
   abs-file-name)
  (match requires.txt-files
(()

bug#44030: [PATCH] guix: import: Add versioning syntax to pypi importer.

[Lulu]

Ah, heck, I used an old revision of the file when I generated the diff.
Sorry about that. Here's the properly working patch:


diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 15116e349d..1ec1ecbfa1 100644
--- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -118,13 +118,15 @@
 
 (define (pypi-fetch name)
   "Return a  record for package NAME, or #f on failure."
-  (and=> (json-fetch (string-append "https://pypi.org/pypi/; name "/json"))
- json->pypi-project))
+  ;; Convert @ in package name to / to access the correct URL.
+  (let ((versioned-name (string-join (string-split name #\@) "/")))
+(and=> (json-fetch (string-append "https://pypi.org/pypi/; versioned-name 
"/json"))
+   json->pypi-project)))
 
 ;; For packages found on PyPI that lack a source distribution.
 (define-condition-type  
   missing-source-error?
-  (package  missing-source-error-package))
+  (package missing-source-error-package))
 
 (define (latest-source-release pypi-package)
   "Return the latest source release for PYPI-PACKAGE."
@@ -371,7 +373,7 @@ be extracted in a temporary directory."
  (invoke "tar" "xf" archive "-C" dir)))
(let ((requires.txt-files
   (find-files dir (lambda (abs-file-name _)
-   (string-match "\\.egg-info/requires.txt$"
+(string-match "\\.egg-info/requires.txt$"
   abs-file-name)
  (match requires.txt-files
(()

bug#44230: qjackctl cannot start jack2 unless it has been built with jack2 as an input

[Nathan Dehnel]

See https://github.com/rncbc/qjackctl/issues/96 for the error messages.

I have just confirmed this by building qjackctl with jack2 as an input,
verifying the settings are unchanged, and starting jackdbus from qjackctl.