bug#50031: gcc-core-mesboot1 may not be deterministic

[Thiago Jung Bauermann via Bug reports for GNU Guix]

Hello Ludo,

Em segunda-feira, 20 de setembro de 2021, às 04:11:13 -03, Ludovic Courtès 
escreveu:
> Hi Thiago,
> 
> Thiago Jung Bauermann  skribis:
> > Using Guix from both the ‘master’ branch¹ and from the
> > ‘core-updates-frozen’ branch², the following command fails:
> > 
> > $ guix build --check \
> > 
> > -e '(@@ (gnu packages commencement) gcc-core-mesboot1)’
> > ⋮
> > 
> > guix build: erro: derivation
> > `/gnu/store/qbnxfv7v7288iisl44kccz68k0pv9qdi-gcc-core-mesboot1-4.6.4.d
> > rv' may not be deterministic: output
> > `/gnu/store/rn3qvn67nraicabvlrx1rhw6nsjrpgpx-gcc-core-mesboot1-4.6.4'
> > differs
> 
> This was discussed in .  There’s the
> beginning of a patch there that needs to be adapted to avoid depending
> on xz at this early stage, I think.

Thank you for mentioning that discussion. I should have found that before 
filing this issue. Sorry for the duplicate  report.

-- 
Thanks,
Thiago

bug#50696: [core-updates-frozen] Wrong output hashes computed since cb06f7c61e4b839

[Ludovic Courtès]

Hi Marius,

Marius Bakke  skribis:

> On the 'core-updates-frozen' branch, attempting to pull 'master' gives a
> backtrace along the lines of ...
>
> Computing Guix derivation for 'x86_64-linux'... |Backtrace:

[...]

>   1. :
>   message: "derivation 
> `/gnu/store/szv5fh043rrqq01xpycp7xci4y7myqfk-guile-3.0.7.tar.xz.drv' has 
> incorrect output 
> `/gnu/store/qwjhql8lqw40fqhaqw40jwqs0hc0248w-guile-3.0.7.tar.xz', should be 
> `/gnu/store/0zliqv8lcw13f9iasy14kmwsqj3d309w-guile-3.0.7.tar.xz'"
>   status: 1
>
> I bisected it down to cb06f7c61e4b:
>
> commit cb06f7c61e4b8393abf38f1f5891e03c33d53b9b
> Author: Ludovic Courtès 
> Date:   Thu Sep 9 23:22:10 2021 +0200
>
> base32: Provide an open-coded 'bit-field'.
>
> This improves the throughput of 'bytevector->base32-string' a bit.
>
> * guix/base32.scm (bit-field): New macro.

The problem is that at -O1 with Guile 3.0.7 (which is the default for
imported modules on ‘core-updates-frozen’ since
331a09654eb7e9f6212b7e8469077fa7393e8b11), that code gets miscompiled:

--8<---cut here---start->8---
$ rm guix/base32.go
$ guild compile -O1 guix/base32.scm -o guix/base32.go
wrote `guix/base32.go'
$ ./pre-inst-env guile
GNU Guile 3.0.7
Copyright (C) 1995-2021 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,use(guix base32)
scheme@(guile-user)> (nix-base32-string->bytevector 
"0wqd8sjmxfskrflaxywc7gqw7sfawrfvdxd9skxawzfgyy0pzdz6")
$1 = #vu8(230 180 126 128 240 207 124 168 250 212 169 244 186 92 228 202 232 
206 240 60 140 248 170 168 200 83 184 90 164 64 13 112)
scheme@(guile-user)> (bytevector->nix-base32-string $1)
$2 = "000d0080l00k00h0l00c0h00w00a0hq0lh090hl0hh0gwh06"
--8<---cut here---end--->8---

Long story short: the “baseline compiler” (-O0 and -O1) would
erroneously translate (ash x N) to (ash x (- N)) when N is a literal:

--8<---cut here---start->8---
scheme@(guix base32)> (compile '(lambda (x) (ash x -2)) #:to 'value #:env 
(current-module) #:optimization-level 1)
$46 = #
scheme@(guix base32)> (compile '(lambda (x) (ash x -2)) #:to 'value #:env 
(current-module) #:optimization-level 2)
$47 = #
scheme@(guix base32)> ($47 4)
$48 = 1
scheme@(guix base32)> ($46 4)
$49 = 16
--8<---cut here---end--->8---

Fixed upstream:

  
https://git.savannah.gnu.org/cgit/guile.git/commit/?id=74abae04aa68dc18676bd84c4a73324b0613475d

However I’m not sure how to work around the bug, other than by reverting
the change.  I’ll think a bit more about it.

Thanks,
Ludo’.

bug#50705: emacs-guix fails to list packages

[Simon Streit]

Hello,

I just pulled Guix up to 6eded1a04186e3118b293486b038c994e05efedf, and
unfortunately emacs-guix fails to list any installed packages of a given
profile.

Part of the output from it's internal REPL follows as such:
--8<---cut here---start->8---
  14 (eval (#:21:16 ()>) 
#)
In emacs-guix/packages.scm:
   777:23 13 (package/output-sexps _ output _ _ (synopsis installed output 
version name id package-id known-status # #))
In srfi/srfi-1.scm:
   460:18 12 (fold # _ (#< name: …> …))
In emacs-guix/packages.scm:
359:2 11 (_ #< name: "syncthing-gtk" version: 
"0.9.4.4-1.c46fbd8" output: "out" item: "/gnu/store/w4…> …)
   351:22 10 (packages-by-name _ _ _)
   281:48  9 (_ _ _)
In unknown file:
   8 (force #>)
In gnu/packages.scm:
   239:33  7 (fold-packages # # _ # #)
In guix/discovery.scm:
   153:11  6 (all-modules _ #:warn _)
In srfi/srfi-1.scm:
   460:18  5 (fold # _ (("/home/ss2/.config/guix/…" . #)))
In guix/discovery.scm:
   143:19  4 (_ _ ())
In srfi/srfi-1.scm:
   691:23  3 (filter-map # _ . _)
In guix/discovery.scm:
   118:22  2 (_ . _)
In guix/ui.scm:
324:2  1 (report-unbound-variable-error _ #:frame _)
In ice-9/boot-9.scm:
  1685:16  0 (raise-exception _ #:continuable? _)
--8<---cut here---end--->8---

But if I where to remove this package “syncthing-gtk”, the error would
move on to the next package.

Then I rolled back to commit 337b7f5a13b8fd8b5ee320fd5a850ede1ad63b6d,
wich is a commit before 7bae88b5b9dcacad4dcd11b353b486dc2f8a78e2, where
a new checkout of guix was added to the packages tree.  Unfortunately
the error changes over to:
--8<---cut here---start->8---
   2 (eval (#:19:16 ()>) 
#)
In current input:
19:17  1 (_)
In ice-9/boot-9.scm:
  1685:16  0 (raise-exception _ #:continuable? _)
--8<---cut here---end--->8---

Hope this is only a small error.


Kind regards
Simon

bug#50483: [R] Package r-shiny: Shiny server logic inactive

[Charles via Bug reports for GNU Guix]

Thanks simon, I will take into account your suggestions and submit to patches.

bug#50483: [R] Package r-shiny: Shiny server logic inactive

[zimoun]

Hi Charles,

Thanks for the patch.

On Sat, 18 Sep 2021 at 05:14, Charles  wrote:

> Here is a patch to update the packages to use the up to date
> minifier. Is the the correct thread to put this or should I start a
> new thread in the patches mailing list?

I think the bestis to send this patch to guix-patc...@gnu.org.  See
commit message and suggestions below.


> From 57c3801034e77120e4030d0e5421a02343e9280c Mon Sep 17 00:00:00 2001
> From: Charles 
> Date: Sat, 18 Sep 2021 00:05:55 -0500
> Subject: [PATCH] gnu: Update some packages using old JavaScript minifier.

Subject: [PATCH] gnu: Switch to (gnu package uglifyjs) minifier.

Fixes 

> * gnu/packages/minetest.scm (minetest-basic-trains): New variable.
> * gnu/packages/bioinformatics.scm (bismark): Update uglifyjs input.
> * gnu/packages/ci.scm (laminar): Update uglifyjs input.
> * gnu/packages/cran.scm (r-shiny r-shinytree r-shinydashboard r-colourpicker
> r-threejs r-flexdashboard r-networkd3 r-dygraphs): Update uglifyjs input.
> * gnu/packages/javascript.scm (): Update uglifyjs input.

Missing package   --^


I have not compiled, so I assume this removal:

> -  #:use-module (gnu packages lisp-xyz)

does not trigger warning when compiling Guix, right?


> diff --git a/gnu/packages/ci.scm b/gnu/packages/ci.scm
> index b7ea77ca31..a6d296923f 100644
> --- a/gnu/packages/ci.scm
> +++ b/gnu/packages/ci.scm
> @@ -38,7 +38,7 @@
>#:use-module (gnu packages guile)
>#:use-module (gnu packages guile-xyz)
>#:use-module (gnu packages gnupg)
> -  #:use-module (gnu packages lisp-xyz)
> +  #:use-module (gnu packages uglifyjs)

It seems the module are sorted by alphabetic order.

> diff --git a/gnu/packages/cran.scm b/gnu/packages/cran.scm
> index f80256c901..81dd78ac94 100644
> --- a/gnu/packages/cran.scm
> +++ b/gnu/packages/cran.scm
> @@ -84,7 +84,7 @@
>#:use-module (gnu packages java)
>#:use-module (gnu packages javascript)
>#:use-module (gnu packages libevent)
> -  #:use-module (gnu packages lisp-xyz)
> +  #:use-module (gnu packages uglifyjs)

Idem.

> diff --git a/gnu/packages/javascript.scm b/gnu/packages/javascript.scm
> index 81de5b67a6..e107dd6126 100644
> --- a/gnu/packages/javascript.scm
> +++ b/gnu/packages/javascript.scm
> @@ -26,7 +26,7 @@
>#:use-module (gnu packages)
>#:use-module (gnu packages base)
>#:use-module (gnu packages compression)
> -  #:use-module (gnu packages lisp-xyz)
> +  #:use-module (gnu packages uglifyjs)

Idem.


Otherwise, LGTM. Thanks.


Cheers,
simon

bug#38450: Kdenlive file dialog broken

[Mathieu Othacehe]

Hello,

> In kdenlive, I can't add any media ("clips") to a project because the
> file choosing dialog is not working.

This cannot be reproduced on master, closing.

Thanks,

Mathieu

bug#50676: [core-updates-frozen] Image production is broken.

[Mathieu Othacehe]

Hey,

The duplicated system derivation appears to be a grafted version of
the first one.

Running:

--8<---cut here---start->8---
./pre-inst-env guix system image gnu/system/examples/bare-bones.tmpl -t qcow2 
--no-grafts
--8<---cut here---end--->8---

produces an image that boots fine in QEMU.

Mathieu

bug#50697: [core-updates-frozen] ca-certificate-bundle generation is broken.

[Guillaume Le Vaillant]

Mathieu Othacehe  skribis:

> Hello,
>
> On core-updates-frozen, the ca-certificate-bundle derivation produces an
> empty output. That's because nss-certs only contains .crt files that are
> ignored by ca-certificate-bundle procedure.
>
> The following patches should fix the situation.
>
> Thanks,
>
> Mathieu
>
> From 18248cc817952c690694707cc965283dad1933c2 Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe 
> Date: Mon, 20 Sep 2021 10:26:30 +
> Subject: [PATCH 1/2] gnu: certdata2pem: Produce pem files.
>
> Create files with pem extension instead of crt.
>
> [...]

Hi,

With this patch I think the 'install-keystore' phase of icedtea-7 will
also have to be updated to search for the ".pem" files instead of the
".crt" ones.


signature.asc
Description: PGP signature

bug#50697: [core-updates-frozen] ca-certificate-bundle generation is broken.

[Mathieu Othacehe]

Hello,

On core-updates-frozen, the ca-certificate-bundle derivation produces an
empty output. That's because nss-certs only contains .crt files that are
ignored by ca-certificate-bundle procedure.

The following patches should fix the situation.

Thanks,

Mathieu
>From 18248cc817952c690694707cc965283dad1933c2 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe 
Date: Mon, 20 Sep 2021 10:26:30 +
Subject: [PATCH 1/2] gnu: certdata2pem: Produce pem files.

Create files with pem extension instead of crt.

* gnu/packages/certs.scm (certdata2pem)[arguments]<#:phases>{fix-extension}:
New phase.
---
 gnu/packages/certs.scm | 5 +
 1 file changed, 5 insertions(+)

diff --git a/gnu/packages/certs.scm b/gnu/packages/certs.scm
index 37e3fa6786..c9d8e7b76c 100644
--- a/gnu/packages/certs.scm
+++ b/gnu/packages/certs.scm
@@ -104,6 +104,11 @@ port forwarding to your local machine.")
   (arguments
`(#:phases (modify-phases %standard-phases
 (delete 'configure)
+(add-before 'build 'fix-extension
+  (lambda _
+(substitute* "certdata2pem.c"
+  (("\\.crt")
+   ".pem"
 (replace 'build
   (lambda _
 (invoke ,(cc-for-target) "certdata2pem.c"
-- 
2.33.0

>From 751a89919aa7a7bdf963a85112346153bfd03407 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe 
Date: Mon, 20 Sep 2021 10:27:55 +
Subject: [PATCH 2/2] gnu: nss-certs: Copy pem files.

* gnu/packages/certs.scm (nss-certs)[arguments]<#:phases>{install}: Copy pem
files instead of crt files.
---
 gnu/packages/certs.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/certs.scm b/gnu/packages/certs.scm
index c9d8e7b76c..39c3ebd128 100644
--- a/gnu/packages/certs.scm
+++ b/gnu/packages/certs.scm
@@ -164,9 +164,9 @@ that was originally contributed to Debian.")
(call-with-output-file "blacklist.txt" (const #t)))
  ;; Extract selected single certificates from blob.
  (invoke "certdata2pem")
- ;; Copy .crt files into the output.
+ ;; Copy .pem files into the output.
  (for-each (cut install-file <> certsdir)
-   (find-files "." ".*\\.crt$")))
+   (find-files "." ".*\\.pem$")))
(invoke "openssl" "rehash" certsdir)))
 (synopsis "CA certificates from Mozilla")
 (description
-- 
2.33.0

bug#50696: [core-updates-frozen] Wrong output hashes computed since cb06f7c61e4b839

[Marius Bakke]

Hello,

On the 'core-updates-frozen' branch, attempting to pull 'master' gives a
backtrace along the lines of ...

Computing Guix derivation for 'x86_64-linux'... |Backtrace:
In ice-9/boot-9.scm:
152:2 19 (with-fluid* _ _ _)
152:2 18 (with-fluid* _ _ _)
In ./guix/store.scm:
  2108:24 17 (run-with-store # 
# ?)
In ./guix/self.scm:
   1313:2 16 (_ #)
In ./guix/packages.scm:
  1548:17 15 (_ #)
  1177:16 14 (cache! # # 
("x86_64-linux" . #t) #)
  1495:22 13 (thunk)
  1428:25 12 (bag->derivation # #< 
name: "guile-3.0.7" system: "x86_64-l?> ?)
In srfi/srfi-1.scm:
   586:17 11 (map1 (("source" #) ?))
In ./guix/packages.scm:
   1239:5 10 (failure)
In ice-9/boot-9.scm:
  1752:10  9 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
In ./guix/packages.scm:
  1244:18  8 (_)
In ./guix/store.scm:
  2108:24  7 (run-with-store # 
# ?)
   1945:8  6 (_ #)
/In ./guix/gexp.scm:
   275:18  5 (_ #)
In ./guix/packages.scm:
   1603:5  4 (_ #)
In ./guix/store.scm:
  1980:38  3 (_ #)
In ./guix/derivations.scm:
   955:17  2 (derivation # 
"guile-3.0.7.tar.xz" "builtin:download" () #:system ?)
In ./guix/store.scm:
   1058:9  1 (_ # 
"guile-3.0.7.tar.xz.drv" #vu8(68 101 114 105 118 101 40 91 ?) ?)
In ice-9/boot-9.scm:
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
ERROR:
  1. :
  message: "derivation 
`/gnu/store/szv5fh043rrqq01xpycp7xci4y7myqfk-guile-3.0.7.tar.xz.drv' has 
incorrect output 
`/gnu/store/qwjhql8lqw40fqhaqw40jwqs0hc0248w-guile-3.0.7.tar.xz', should be 
`/gnu/store/0zliqv8lcw13f9iasy14kmwsqj3d309w-guile-3.0.7.tar.xz'"
  status: 1

I bisected it down to cb06f7c61e4b:

commit cb06f7c61e4b8393abf38f1f5891e03c33d53b9b
Author: Ludovic Courtès 
Date:   Thu Sep 9 23:22:10 2021 +0200

base32: Provide an open-coded 'bit-field'.

This improves the throughput of 'bytevector->base32-string' a bit.

* guix/base32.scm (bit-field): New macro.

It can be reproduced with a two-dimensional time machine:

  $ guix time-machine --branch=core-updates-frozen -- time-machine \
  --commit=cb06f7c61e4b8393abf38f1f5891e03c33d53b9b -- build hello

...or by pulling 'core-updates-frozen' and attempting to pull 'master'.

The commit was merged to 'core-updates-frozen' in 5c3cb22c9b281066, so
pulling core-updates-frozen -> core-updates-frozen also fails in a
similar manner (albeit on a different derivation).


signature.asc
Description: PGP signature

bug#44187: Channel clones lack SWH fallback

[zimoun]

Hi,

On Sat, 18 Sept 2021 at 23:10, Ludovic Courtès  wrote:
> zimoun  skribis:

> > and after more than 12h, the status is still: «SWH vault: Processing...»
> > and nothing is complete.
>
> Did it eventually succeed?  We obviously have no guarantee as to how
> long it might take to cook a bundle.

No, I stopped.  And I reported to #swh-devel.  It might be something
wrong on their side.
Yeah, cook a bundle could be long... especially with large repo as
Guix (lot of commits and couple of files).
I think it is ok to let the code as it is now.


> >> *Third, and this answers the asterisk above, we must keep in mind that
> >> this is content-addressibility *with SHA1*.  Generating a chosen-prefix
> >> collision is becoming affordable³, so users absolutely need an additional
> >> mechanism to authenticate code they fetched.
>
> [...]
>
> > How a chosen-prefix attack could work here?  I understand why the second
> > preimage attack is an issue.  But I miss how the SHA-1 chosen-prefix attack
> > could be exploited here to compromise the user, because this hash is 
> > provided
> > by this very same user.
>
> I think you’re right, it’s rather second-preimage attacks that would be
> a serious problem.  My point is: as time passes, assuming that a SHA1
> resolves to a single revision on SWH is becoming more and more
> questionable.

Well, SHA-1 is 2^160 (~10^48.2) and compared to 10^50 which is the
estimated number of atoms in Earth.  Speaking about
content-addressability, SHA-1 seems fine.  However, for security, yeah
time flies. :-)


> >>   swh: Support downloads of bare Git repositories.
> >>   git: 'update-cached-checkout' can fall back to SWH when cloning.
> >>   git: 'reference-available?' recognizes 'tag-or-commit'.
>
> I’ve pushed this after adding the warning as you suggested:
>
>   dce2cf311b * git: 'reference-available?' recognizes 'tag-or-commit'.
>   05f44c2d85 * git: 'update-cached-checkout' can fall back to SWH when 
> cloning.
>   6ec81c31c0 * swh: Support downloads of bare Git repositories.

Cool!  I would deserve a --news entry. ;-)

Cheers,
simon

bug#50566: [core-updates-frozen] Grub fails to decrypt LUKS partition (cryptomount not found)

[Mathieu Othacehe]

Hey,

>>#:strip-flags '("--strip-debug" "--enable-deterministic-archives")

Pushed as 650e85d85514c5fae06adf97ae615643a41bbbd8 :).

> That one works as well. I tested it :)

Thanks for the suggestion Ludo and for the testing Jonathan.

Mathieu

bug#50676: [core-updates-frozen] Image production is broken.

[Mathieu Othacehe]

Hey Ludo!

> What command did you use?

The same one as you :)

> drops me in the GRUB rescue shell right from the start, but I think
> that’s because the command surprisingly builds an EFI image, as can be
> seen from the generated genimage.cfg:

Yep, that would be because of the Grub stripping issue. Regarding the
EFI, both the qcow2 and the efi-raw image types produce "hybrid" images
that have "grub" installed in the MBR-gap and "grub-efi" in the ESP
partition. This way, those images can be used both on legacy BIOS based
machines as well as on more modern UEFI machines.

Now the image types names can be confusing, and we could rename efi-raw
and qcow2 to pc-hybrid-raw and pc-hybrid-qcow2 respectively.

WDYT?

Thanks,

Mathieu

bug#50566: [core-updates-frozen] Grub fails to decrypt LUKS partition (cryptomount not found)

[Jonathan Brielmaier]

Am 20.09.21 um 00:47 schrieb Ludovic Courtès:

‘master’ uses ‘--strip-debug’ whereas ‘core-updates-frozen’ uses
‘--strip-unneeded’ (from commit
e0f31baacc6ad30096a332b69433c85f5830bb2c).

Instead of #:strip-binaries? #f, we could try:

   #:strip-flags '("--strip-debug" "--enable-deterministic-archives")


That one works as well. I tested it :)

bug#50568: Missing source code

[Ludovic Courtès]

Hi Florian,

"pelzflorian (Florian Pelz)"  skribis:

> On Tue, Sep 14, 2021 at 10:38:55AM +0200, Ludovic Courtès wrote:
>> I wonder if there’s a way to construct generic web.archive.org URLs that
>> we could use as a fallback in (guix build download)?
> On Thu, Sep 16, 2021 at 09:45:26PM +0200, Ludovic Courtès wrote:
>> How ’bout the attached patch?  Works for me.
>
> Thank you for implementing.  I guix pull with the patch applied, add the IP
> from
>
> guix environment --ad-hoc bind:utils -- dig web.archive.org
>
> to /etc/hosts, set my DNS server to a non-existing 10.0.0.50, disable
> IPv6, then “sudo herd stop nscd”, re-connect Network Manager (ping
> gnu.org fails now) and LC_ALL=en_US.utf8 guix install hello.  It
> fails.  Did I misunderstand the patch’s purpose or test in a wrong
> way?  This is the output:

The thing is that this code is used by the internal ‘guix
perform-download’ command, invoked by ‘guix-daemon’ when performing a
source download.  Thus, since you were still running the old daemon, you
were using code lacking the Internet Archive fallback.

To test it, you can first start the new daemon, along these lines:

  sudo herd stop guix-daemon
  sudo -E ./pre-inst-env guix-daemon --build-users-group=guixbuild

Another option is to call ‘url-fetch’ manually at the REPL.

Thanks for testing!

Ludo’.

bug#50031: gcc-core-mesboot1 may not be deterministic

[Ludovic Courtès]

Hi Thiago,

Thiago Jung Bauermann  skribis:

> Using Guix from both the ‘master’ branch¹ and from the ‘core-updates-frozen’
> branch², the following command fails:
>
> $ guix build --check \
> -e '(@@ (gnu packages commencement) gcc-core-mesboot1)’
> ⋮
> guix build: erro: derivation 
> `/gnu/store/qbnxfv7v7288iisl44kccz68k0pv9qdi-gcc-core-mesboot1-4.6.4.drv' may 
> not be deterministic: output 
> `/gnu/store/rn3qvn67nraicabvlrx1rhw6nsjrpgpx-gcc-core-mesboot1-4.6.4' differs
>
>
> During the stripping phase, there are many of these warnings, which may be
> related:
>
> starting phase `strip'
> stripping binaries in 
> "/gnu/store/rn3qvn67nraicabvlrx1rhw6nsjrpgpx-gcc-core-mesboot1-4.6.4/lib" 
> with "strip" and flags ("--strip-debug" "--enable-deterministic-archives")
> strip: unrecognized option `--enable-deterministic-archives'

This was discussed in .  There’s the
beginning of a patch there that needs to be adapted to avoid depending
on xz at this early stage, I think.

Thanks,
Ludo’.