Re: guix environment --profile with --ad-hoc

[Lars-Dominik Braun]

Hi Pierre,

> Instead, I'd like the following behaviour:
> […]
hm, I feel that’s unnecessarly complex with lots of if’s and else’s. If
I could design the frontend from scratch, I’d have one command that does
profile/environment manipulation (because they’re essentially the same)
and one that can set them up for use. In that world you’d do

$ guix environment $(guix profile -m manifest.scm -i additional-package)

to get a temporary profile/environment and

# add a few packages
$ guix profile -p /path -m manifest.scm -i additional-package
/path
# remove one
$ guix profile -p /path -r additional-package
/path
# enter env
$ guix environment /path
$ ^D
# enter using container
$ guix environment -C --user=joeuser /path
$ ^D

to get a persistent environment and entering it without fiddling with -r
vs. -p vs. package options. In this case `guix environment` does not
know anything about packages and can only operate on existing profiles.

Maybe it’s possible to do profile inheritance like this too:

$ guix profile -p /new/profile -i /old/profile additional-package
/new/profile

which would be the union of /old/profile and {additional-package}.

Cheers,
Lars

gnutls package may be vulnerable to CVE-2021-20232

[Léo Le Bouter]

CVE-2021-20232  12.03.21 20:15
A flaw was found in gnutls. A use after free issue in
client_send_params in lib/ext/pre_shared_key.c may lead to memory
corruption and other potential consequences.

It is not certain whether 3.6.x series are affected as packaged in GNU
Guix. I asked the upstream at <
https://gitlab.com/gnutls/gnutls/-/issues/1151#note_528567535>. Let's
wait for an answer, or then apply/backport this commit (
https://gitlab.com/gnutls/gnutls/-/commit/75a937d97f4fefc6f9b08e3791f151445f551cb3
) to 3.6.x series.

A rather low impact vulnerability upstream says, but I would be careful
there as an experienced exploit writer could find reliable ways to
exploit it in my opinion.

Let's patch this as soon as possible!


signature.asc
Description: This is a digitally signed message part

libupnp package vulnerable to CVE-2021-28302

[Léo Le Bouter]

CVE-2021-28302  12.03.21 16:15
A stack overflow in pupnp 1.16.1 can cause the denial of service
through the Parser_parseDocument() function. ixmlNode_free() will
release a child node recursively, which will consume stack space and
lead to a crash.

Upstream did not provide a patch yet, see <
https://github.com/pupnp/pupnp/issues/249>.

I suggest we wait for the patch to be made and then update, to be
monitored.


signature.asc
Description: This is a digitally signed message part

regression: “guix pack” Docker images no longer work on AWS

[Ricardo Wurmus]

Hi Guix,

I’m using “guix pack”-generated Docker images on AWS ECS.  On June 17,
2020 I generated and uploaded an image that works fine.  According to
AWS this image has this manifest type:

application/vnd.docker.distribution.manifest.v2+json

Today I generated a new image that does not work.  It cannot be opened
by ECS; it cannot find /bin/sh, which exists.  The manifest type is now
recognized as

application/vnd.oci.image.manifest.v1+json

It also now detects an “Artifact media type” and prints it as

application/vnd.oci.image.config.v1+json

I’d very much like to push a newly generated image.  Is there a way to
generate the image as it was done in the summer of 2020?

Note that both these images appear to work “fine” with a local Docker
installation (i.e. when run with “docker run”).  (When running /bin/sh
in the container interactively it appears to freeze whenever I try to
actually run a command, but perhaps I’m just holding it wrong…)

-- 
Ricardo

Re: CVEs missing from the NIST database

[Mark H Weaver]

Hi Ludovic,

Ludovic Courtès  writes:

> In this case, I noticed that ‘guix lint -c cve cairo’ wouldn’t report
> CVE-2020-35492 and found that
>  is 404.
>
> Likewise, this command:
>
>wget -qO - 
> "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2020.json.gz; | \
>  gunzip | grep CVE-202-35492
>
> turns up nothing.
>
> It could be that this CVE is still “pending” (I think that happens
> sometimes).  Do you know more about this one?

I was looking in Debian's cairo package for fixes for other CVEs (namely
the ones that "guix lint -c cve cairo" *did* report), and noticed that
they included a fix for CVE-2020-35492.  I didn't investigate further.

While we're on the subject on issues with the CVE database, or possibly
with our linter, "guix lint -c cve" now erroneously reports:

--8<---cut here---start->8---
gnu/packages/gnome.scm:8434:2: gnome-shell@3.34.5: probably vulnerable to 
CVE-2019-3820
gnu/packages/gnome.scm:6452:2: gvfs@1.40.2: probably vulnerable to 
CVE-2019-12447, CVE-2019-12448, CVE-2019-12449
--8<---cut here---end--->8---

All of these are incorrect.

* CVE-2019-3820 was fixed long before GNOME 3.34 came out, and I've
  verified that the commit that fixes it is included in
  gnome-shell-3.34.5:

commit f0a7395b3006360905ccdc642982f9fc67378927
Author: Ray Strode 
Date:   Wed Jan 23 15:59:15 2019 -0500

shellActionModes: disable POPUP keybindings in unlock screen

* CVE-2019-12447, CVE-2019-12448, and CVE-2019-12449 are fixed in
  gvfs-1.40.2, according to its NEWS file:

--8<---cut here---start->8---
Major changes in 1.40.2
===
* daemon: Only accept EXTERNAL authentication (CVE-2019-12795)
* daemon: Check that the connecting client is the same user (CVE-2019-12795)
* admin: Ensure correct ownership when moving to file:// uri (CVE-2019-12449)
* admin: Use fsuid to ensure correct file ownership (CVE-2019-12447)
* admin: Allow changing file owner (CVE-2019-12447)
* admin: Add query_info_on_read/write functionality (CVE-2019-12448)
* afc: Remove assumptions about length of device UUID to support new devices
* gmountsource: Fix deadlocks in synchronous API
* afp: Fix afp backend crash when no username supplied
* Translation updates
--8<---cut here---end--->8---

  Mark

Re: Release on April 18th?

[Chris Marusich]

Chris Marusich  writes:

> Subject: [PATCH] syscalls: mount: Fix a matching bug.

In the patch message, "mount" should be "mounts".  Sorry for the typo!

-- 
Chris


signature.asc
Description: PGP signature

Re: Release on April 18th?

[Chris Marusich]

Hi,

Vincent Legoll  writes:

> I rebuilt guix on core-updates with gcc-8 succesfully
> I'll now try the same above wip-ppc64le.

Awesome!  Thank you for doing this.  I'm sure there will be some bumps,
and the sooner we can fix them, the easier it will be to integrate
later.

I'm still working on getting you access to ppc64le hardware or a VM.

Andreas Enge  writes:

> Am Fri, Mar 12, 2021 at 12:33:18AM -0800 schrieb Chris Marusich:
>> The proc man page has this to say about column 7:
>>   (7) optional fields: zero or more fields of the form "tag[:value]"
>
> And it goes on like this:
> (8)  separator: the end of the optional fields is marked
>by a single hyphen.
>
> So it looks like it is enough to search for a hyphen surrounded by spaces.
> The hyphen is apparently there also when there is no optional field (7),
> as can be seen from your examples and also my own system, where both occur
> in parallel:
> 34 26 253:0 /gnu/store /gnu/store ro,noatime - ext4 /dev/mapper/cryptroot rw
> 51 26 253:0 /var/lib/docker /var/lib/docker rw,relatime shared:1 - ext4 
> /dev/mapper/cryptroot rw
>
> Alternatively, one can also count from the back to get the fields labelled
> (9) to (11) (which may be (8) to (10) in case there are no optional fields).
> (I was momentarily confused by "(12) super option*s*"; but these are
> apparently separated by commas and not whitespace.)

That's true.  How about the following patch?  It fixes the issue for me
on my systems, and I manually tried out the new match pattern with some
lines from Leo's output, and it seems to behave correctly.  If nobody
has concerns, I'd like to apply it to master directly, since the tests
are already failing there on some systems, like mine.  Here's the patch:

From 5417f68ee5892f6a587895105854cadf29eb7297 Mon Sep 17 00:00:00 2001
From: Chris Marusich 
Date: Thu, 11 Mar 2021 23:19:30 -0800
Subject: [PATCH] syscalls: mount: Fix a matching bug.

On some systems, the columns in /proc/self/mountinfo look like this:

23 28 0:21 / /proc rw,nosuid,nodev,noexec,relatime shared:11 - proc proc rw

Before this change, the mount procedure was written with the assumption that
the type and source could always be found in columns 8 and 9, respectively.
However, the proc(5) man page explains that there can be zero or more optional
fields starting at column 7 (e.g., "shared:11" above), so this assumption is
false in some situations.

* guix/build/syscalls.scm (mount): Update the match pattern to use ellipsis to
match zero or more optional fields followed by a single hyphen.  Remove the
trailing ellipsis, since multiple ellipses are not allowed in the same level.
The proc(5) man page indicates that there are no additional columns, so it is
probably OK to match an exact number of columns at the end like this.
---
 guix/build/syscalls.scm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/guix/build/syscalls.scm b/guix/build/syscalls.scm
index 552343a481d..726c8e86d06 100644
--- a/guix/build/syscalls.scm
+++ b/guix/build/syscalls.scm
@@ -621,8 +621,9 @@ current process."
   (if (eof-object? line)
   (reverse result)
   (match (string-tokenize line)
+;; See the proc(5) man page for a description of the columns.
 ((id parent-id major:minor root mount-point
- options _ type source _ ...)
+ options _ ... "-" type source _)
  (let ((devno (string->device-number major:minor)))
(loop (cons (%mount (octal-decode source)
(octal-decode mount-point)
-- 
2.26.2


Efraim Flashner  writes:

>> Something about the way in which we are searching for the patch is off,
>> but I don't have time just now to investigate.  Efraim, if you can
>> figure it out, that'd be nice, but I'll look into it more tomorrow.
>> It's probably something simple and related to commit 2712703.
>
> Leo told me about it yesterday and I pushed a second commit that fixed
> it. We needed to make sure the patch file was included as an input,
> that's why it got #f instead of a string. In any case, commit
> 710cfc330a7ed06934a193583b159fbdf07bf2fe takes care of it; it's the
> combination of 2712703 and the squash commit.
>
> I'm now running make guix-binary.powerpc64le-linux.tar.xz and so far
> it's made it past the initial building stages, we're on to building the
> grafts now.

Thank you.  This fixed the patch-related problem for me, too.  I'm
currently running "make guix-binary.powerpc64le-linux.tar.xz", also, on
my Debian ppc64le machine.  We'll see how far it gets - fingers crossed!

-- 
Chris


signature.asc
Description: PGP signature

Re: CVEs missing from the NIST database

[Leo Famulari]

On Fri, Mar 12, 2021 at 04:31:59PM +0100, Ludovic Courtès wrote:
> It could be that this CVE is still “pending” (I think that happens
> sometimes).  Do you know more about this one?

I found some references from other distros:

https://access.redhat.com/security/cve/cve-2020-35492
https://security-tracker.debian.org/tracker/CVE-2020-35492

... and the upstream bug report:

https://gitlab.freedesktop.org/cairo/cairo/-/issues/437

My impression of the process around reporting and registering CVE IDs is
that it's somewhat decentralized now, so there can be lack of
coordination between reporters and "canonical" authorities like NIST.


signature.asc
Description: PGP signature

Re: guix environment --profile with --ad-hoc

[Pierre Neidhardt]

Hi again,

There is possibly a design flaw that prevents it from being completely
super useful :)  It'd be great if the command would be more "do-what-I-mean"
when both package options and --profile are passed.

Currently, I get

--8<---cut here---start->8---
$ guix environment -l ./build-scripts/guix.scm -r my-root -p my-profile

guix environment: error: '--profile' cannot be used with package options
--8<---cut here---end--->8---

Instead, I'd like the following behaviour:

- If "my-root" does not exist, load guix.scm and create link the
  resulting environment to my-root.

- If "my-root" exists, do _not_ load guix.scm, instead load the profile
  as per my-profile.

Optional: isn't --root superfluous when --profile is specified?  Seems
like it, so we could just do

--8<---cut here---start->8---
$ guix environment -l ./build-scripts/guix.scm -p my-profile
--8<---cut here---end--->8---

to get the aforementioned behaviour.

What do you think?
Lars?

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

CVEs missing from the NIST database

[Ludovic Courtès]

Hi Mark,

guix-comm...@gnu.org skribis:

> commit bc16eacc99e801ac30cbe2aa649a2be3ca5c102a
> Author: Mark H Weaver 
> AuthorDate: Fri Mar 12 05:24:36 2021 -0500
>
> gnu: cairo: Fix CVE-2018-19876 and CVE-2020-35492.
> 
> * gnu/packages/patches/cairo-CVE-2018-19876.patch,
> gnu/packages/patches/cairo-CVE-2020-35492.patch: New files.
> * gnu/local.mk (dist_patch_DATA): Add them.
> * gnu/packages/gtk.scm (cairo)[replacement]: New field.
> (cairo/fixed): New variable.
> (cairo-xcb): Use package/inherit.

Since there are lot of CVEs getting fixed in Guix these days (thanks
folks!), I’m trying to see how helpful (guix cve) is for those.

In this case, I noticed that ‘guix lint -c cve cairo’ wouldn’t report
CVE-2020-35492 and found that
 is 404.

Likewise, this command:

   wget -qO - "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2020.json.gz; 
| \
 gunzip | grep CVE-202-35492

turns up nothing.

It could be that this CVE is still “pending” (I think that happens
sometimes).  Do you know more about this one?

Thanks,
Ludo’.

Release 1.2.1: timeline

[zimoun]

Hi,

The plan is to release on the April, 18th.  It is a target date.

This 1.2.1 release will mainly contain bunch of bug fixes and package
updates.  More, remove of Python 2 when possible.

Releasing is a good occasion to take the time to ungraft and test the
installers.


Ungrafting should break packages.  It is somehow unexpected but it could
happen that’s why it needs some tests and care.  Leo wrote how to help
here:

   

and the branch wip-next-release already contains fixes.  Please check it
out and feedback is welcome.

We are planning an «ungraftathon» the last week-end of March (27-28).
Please roam on #guix if you want to help.

Python 2 removal needs some love.  Maxim described how they is doing
here:

   

and a set of candidates is listed there:

   


That’s said, you have in your tree any package update, it is the good
time to submit them. :-)

In addition, any bug fix is welcome.  Closing old ones are very welcome! :-)


A draft of the timeline is:

 - until April 1rst: fixes, check substitute availability, etc.
 - as soon as possible: start to build wip-next-release
 - merge branch wip-next-release when ready
 - on Monday 12th April, string freeze
 - couple of days after, branch the release and write the materials
   (ChangeLog and posts)
 - release

The architecture armf will not be included.  The branch core-updates
will not be merged.  Once this release is done, we could write a
timeline for the next core-updates merge and list what should be
included in the next release (1.2.2 or 1.3).  Somehow, all this is an
“experiment” for a webpage detailing the different timelines.

Does it make sense?


Cheers,
simon

Re: Release on April 18th?

[Efraim Flashner]

On Fri, Mar 12, 2021 at 12:33:18AM -0800, Chris Marusich wrote:
> Hi Efraim and Ludo,
> 
> Efraim Flashner  writes:
> 
> > My plan was absolutely to merge master into core-updates after and then
> > integrate all the changes into their affected packages. I'd also make
> > sure to bump gcc to 8 (assuming we don't go straight to 9).
> 
> Sounds good.  If we can get powerpc64le-linux working on master, I'd be
> very happy.  We can simultaneously try that while still working on
> wip-ppc64le (based on core-updates).
> 
> I tried out the wip-ppc64le-for-master branch.  I can build it manually
> on my Debian ppc64le system, but "make check" fails.  There are two test
> failures.
> 
> The first test failure is tests/syscalls.scm.  It seems that Ludo's
> recently added mount procedure (7e9d9f2 syscalls: Add 'mounts' and the
>  record type.) does not work on my system.  In fact, it does not
> work on my Debian ppc64le system, nor does it work on my x86_64 Fedora
> system.  In both cases, the code makes the incorrect assumption that the
> type and source are located in columns 8 and 9, like so:
> 
> --8<---cut here---start->8---
> (define (mounts)
>   "Return the list of mounts ( records) visible in the namespace of the
> current process."
>   (define (string->device-number str)
> (match (string-split str #\:)
>   (((= string->number major) (= string->number minor))
>(+ (* major 256) minor
> 
>   (call-with-input-file "/proc/self/mountinfo"
> (lambda (port)
>   (let loop ((result '()))
> (let ((line (read-line port)))
>   (if (eof-object? line)
>   (reverse result)
>   (match (string-tokenize line)
> ((id parent-id major:minor root mount-point
>  options _ _ type source _ ...)
>  (let ((devno (string->device-number major:minor)))
>(loop (cons (%mount (octal-decode source)
>(octal-decode mount-point)
>devno type options)
>result)))
> --8<---cut here---end--->8---
> 
> However, on my systems, the correct columns are 9 and 10.  Here's the
> first few lines of output from Fedora:
> 
> --8<---cut here---start->8---
> $ cat /proc/self/mountinfo 
> 22 97 0:21 / /sys rw,nosuid,nodev,noexec,relatime shared:2 - sysfs sysfs rw
> 23 97 0:5 / /proc rw,nosuid,nodev,noexec,relatime shared:24 - proc proc rw
> 24 97 0:6 / /dev rw,nosuid shared:20 - devtmpfs devtmpfs 
> rw,size=3923828k,nr_inodes=980957,mode=755
> ...
> --8<---cut here---end--->8---
> 
> And here it is from Debian:
> 
> --8<---cut here---start->8---
> 22 28 0:20 / /sys rw,nosuid,nodev,noexec,relatime shared:7 - sysfs sysfs rw
> 23 28 0:21 / /proc rw,nosuid,nodev,noexec,relatime shared:11 - proc proc rw
> 24 28 0:5 / /dev rw,nosuid,noexec,relatime shared:2 - devtmpfs udev 
> rw,size=15625664k,nr_inodes=244151,mode=755
> ...
> --8<---cut here---end--->8---
> 
> On these systems, the 7th column is an optional field, like "shared:7".
> The proc man page has this to say about column 7:
> 
>   (7) optional fields: zero or more fields of the form "tag[:value]"
> 
> So presumably there can be even more than just one optional field.  In
> any case, Leo Le Bouter kindly checked his own x86_64 system, where he
> observed different output:
> 
> --8<---cut here---start->8---
> $ cat /proc/self/mountinfo 
> 23 27 0:21 / /proc rw,relatime - proc none rw
> 24 27 0:5 / /dev rw,relatime - devtmpfs none 
> rw,size=7972408k,nr_inodes=1993102,mode=755
> 25 27 0:22 / /sys rw,relatime - sysfs none rw
> --8<---cut here---end--->8---
> 
> As you can see, in this case there is no optional column, so the mount
> procedure works fine on Leo's system.  But it fails on mine.
> 
> Ludo, do you have an opinion on how to fix this?  It seems like we can't
> assume the number of columns will always be the same, so I guess we'll
> have to somehow ignore the optional columns more intelligently, if we
> want to keep using string-tokenize to do this.
> 
> The second test failure is tests/pack.scm.  There are two contributing
> factors to this test failure.
> 
> The first contributing factor was commit 8f52ea2 on
> wip-ppc64le-for-master.  It fixes an issue that is not present on
> master, and for that reason it actually causes a problem if it's
> included.  I have removed it from the branch in Savannah; please update
> your local copy.

Ooops, I guess it was like the findutils-boot0 patch, necessary for
core-updates but not needed for master.

> The second contributing factor is this bug:
> 
> https://issues.guix.gnu.org/47018
> 
> However, we can work around it by 

Guile-zlib 0.1.0 released

[Ludovic Courtès]

Hi!

I’m pleased to announce the second release of Guile-zlib:

  git clone https://notabug.org/guile-zlib/guile-zlib
  cd guile-zlib
  git checkout v0.1.0  # or 429b3c65ee8946d7ced2353efb8710047fd29c57
  git tag -v v0.1.0

The ‘git tag -v’ command checks the authenticity of your checkout.
You may need to retrieve the signing key first:

  gpg --keyserver pool.sks-keyservers.net \
  --recv-keys 3CE464558A84FDC69DB40CFB090B11993D9AEBB5

Guile-zlib provides Guile 3.0/2.x bindings to the zlib compression
library.

Changes since the previous release (excerpt from the ‘NEWS’ file):

  ** New procedures: ‘make-zlib-input-port’, ‘make-zlib-output-port’

 These procedures and the companion ‘call-with-zlib-*’ procedures
 provide an alternative to the ‘make-gzip-*’ and ‘call-with-gzip-*’
 procedures; unlike those, they operate on all types of ports, not
 just file ports.

 These procedures also support multiple formats: raw “deflate”,
 “zlib”, and “gzip”.

  ** New ‘compress’ and ‘uncompress’ procedures

 These procedures support compression/decompression of individual
 bytevectors.

You can report bugs and send patches to  or on the
web site:

  https://notabug.org/guile-zlib/guile-zlib

If you like Guile-zlib, you may also like these:

  https://notabug.org/guile-lzlib/guile-lzlib
  https://notabug.org/guile-zstd/guile-zstd

:-)

Ludo’.


signature.asc
Description: PGP signature

Re: guix environment --profile with --ad-hoc

[Ricardo Wurmus]

Pierre Neidhardt  writes:

> It seems to be some thing more than just Guix commands being slow.
> On my machine
>
>
> --8<---cut here---start->8---
> $ time guix environment --help >/dev/null
>
> real  0m0.345s
> --8<---cut here---end--->8---
>
> Which is 2s less than
>
> --8<---cut here---start->8---
> $ time guix environment --profile=/my/profile -- exit
>
> real  0m2.372s
> --8<---cut here---end--->8---
>
> So `guix environment --profile' seems to be pretty slow still.

https://elephly.net/downies/guix-env-perf.svg is a timechart recording
of “guix environment --profile=$HOME/.guix-profile -- exit”.  It shows
that 0.5s Guix is busy with itself, and then enters a back and forth
with the daemon.  The little thing at the end is where “exit” is invoked
(and fails).

strace reveals more.  For example, some time is lost searching libraries
and other files.  (At least the excessive library searches should
already have been mitigated by changes on the core-updates branch.)
This includes reading .go files, .scm files, reading TLS certs for
guile-gnutls, etc.

Then we finally connect to the daemon.  We read the profile manifest,
and then go on another search for Guile files, this time related to
package modules.  For every module it looks something like this:

--8<---cut here---start->8---
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/3.0/gnu/packages/commencement.scm",
 0x7ffdcfe812e0) = -1 ENOENT (No such file or directory)
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/site/3.0/gnu/packages/commencement.scm",
 0x7ffdcfe812e0) = -1 ENOENT (No such file or directory)
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/site/gnu/packages/commencement.scm",
 0x7ffdcfe812e0) = -1 ENOENT (No such file or directory)
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/gnu/packages/commencement.scm",
 0x7ffdcfe812e0) = -1 ENOENT (No such file or directory)
6455  stat("/home/rekado/dev/gx/branches/master/gnu/packages/commencement.scm", 
{st_mode=S_IFREG|0644, st_size=159995, ...}) = 0
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/lib/guile/3.0/ccache/gnu/packages/commencement.go",
 0x7ffdcfe810f0) = -1 ENOENT (No such file or directory)
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/lib/guile/3.0/site-ccache/gnu/packages/commencement.go",
 0x7ffdcfe810f0) = -1 ENOENT (No such file or directory)
6455  stat("/home/rekado/dev/gx/branches/master/gnu/packages/commencement.go", 
{st_mode=S_IFREG|0644, st_size=727379, ...}) = 0
6455  openat(AT_FDCWD, 
"/home/rekado/dev/gx/branches/master/gnu/packages/commencement.go", 
O_RDONLY|O_CLOEXEC) = 14
--8<---cut here---end--->8---

I’m guessing that we could avoid some of that searching (and thus
condense a whole lot of stat calls) by constraining things a little
ahead of time.  We know the location of Guile modules in advance, so
perhaps we can come up with a way to avoid having to go on
GUILE_LOAD_PATH and GUILE_LOAD_COMPILED_PATH for everything, e.g. by
recording GUIX_ROOT and using that.

Same for patches:

--8<---cut here---start->8---
…
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/3.0/binutils-boot-2.20.1a.patch",
 0x7ffdcfe80ef0) = -1 ENOENT (No such file or directory)
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/site/3.0/binutils-boot-2.20.1a.patch",
 0x7ffdcfe80ef0) = -1 ENOENT (No such file or directory)
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/site/binutils-boot-2.20.1a.patch",
 0x7ffdcfe80ef0) = -1 ENOENT (No such file or directory)
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/binutils-boot-2.20.1a.patch",
 0x7ffdcfe80ef0) = -1 ENOENT (No such file or directory)
6455  
stat("/home/rekado/dev/gx/branches/master/gnu/packages/patches/binutils-boot-2.20.1a.patch",
 {st_mode=S_IFREG|0644, st_size=6924, ...}) = 0
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/3.0/glibc-boot-2.16.0.patch",
 0x7ffdcfe80ef0) = -1 ENOENT (No such file or directory)
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/site/3.0/glibc-boot-2.16.0.patch",
 0x7ffdcfe80ef0) = -1 ENOENT (No such file or directory)
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/site/glibc-boot-2.16.0.patch",
 0x7ffdcfe80ef0) = -1 ENOENT (No such file or directory)
6455  
stat("/gnu/store/m5iprcg6pb5ch86r9agmqwd8v6kp7999-guile-3.0.5/share/guile/glibc-boot-2.16.0.patch",
 0x7ffdcfe80ef0) = -1 ENOENT (No such file or directory)
6455  

Re: guix environment --profile with --ad-hoc

[Pierre Neidhardt]

Lars-Dominik Braun  writes:

>> My only complaint is that it's still a bit slow:
>> Is there anything we can do to speed this up?
> yeah, true. I’m think it’s still computing and building derivations, for
> example using manifest->derivation for `prof-drv` and then again
> built-derivations for `prof-drv`. Maybe if we make these conditional
> it’ll be faster…?

Seems to make sense!

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: guix environment --profile with --ad-hoc

[Lars-Dominik Braun]

Hi Pierre,

> My only complaint is that it's still a bit slow:
> Is there anything we can do to speed this up?
yeah, true. I’m think it’s still computing and building derivations, for
example using manifest->derivation for `prof-drv` and then again
built-derivations for `prof-drv`. Maybe if we make these conditional
it’ll be faster…?

Lars



signature.asc
Description: PGP signature

Re: Release on April 18th?

[Andreas Enge]

Hello,

Am Fri, Mar 12, 2021 at 12:33:18AM -0800 schrieb Chris Marusich:
> The proc man page has this to say about column 7:
>   (7) optional fields: zero or more fields of the form "tag[:value]"

And it goes on like this:
(8)  separator: the end of the optional fields is marked
   by a single hyphen.

So it looks like it is enough to search for a hyphen surrounded by spaces.
The hyphen is apparently there also when there is no optional field (7),
as can be seen from your examples and also my own system, where both occur
in parallel:
34 26 253:0 /gnu/store /gnu/store ro,noatime - ext4 /dev/mapper/cryptroot rw
51 26 253:0 /var/lib/docker /var/lib/docker rw,relatime shared:1 - ext4 
/dev/mapper/cryptroot rw

Alternatively, one can also count from the back to get the fields labelled
(9) to (11) (which may be (8) to (10) in case there are no optional fields).
(I was momentarily confused by "(12) super option*s*"; but these are
apparently separated by commas and not whitespace.)

Andreas

Re: guix environment --profile with --ad-hoc

[zimoun]

Hi Pierre,

On Thu, 11 Mar 2021 at 21:04, Pierre Neidhardt  wrote:

> My only complaint is that it's still a bit slow:
>
> --8<---cut here---start->8---
> time guix environment -C --profile=/my/profile -- exit
>
> real  0m2.372s
> --8<---cut here---end--->8---
>
> Same without `-C`.
>
> In comparison,
>
> --8<---cut here---start->8---
> time env -i $(which bash) --norc --noprofile -c 'source nyxt/etc/profile && 
> exit'
>
>
> real  0m0.004s
> --8<---cut here---end--->8---

You are not comparing apple to apple. :-)


> Is there anything we can do to speed this up?

We already had this discussion. :-)

See the thread: 

Compared to the previous discussion, the --profile is a good improvement
since it by-passes the computations of derivations and profiles and the
time of the container part still is less than 0.2s on my machine, which
is acceptable, as you are saying. :-)


Therefore, IIUC, your question is: how to speed up the call of Guix
commands?  Right?

For instance, cold cache:

--8<---cut here---start->8---
$ sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
$ time guix environment --help >/dev/null

real0m1.730s
user0m0.845s
sys 0m0.198s
--8<---cut here---end--->8---


BTW, what about “guix run” which is faster, right?


Cheers,
simon

Re: Release on April 18th?

[Chris Marusich]

Hi Efraim and Ludo,

Efraim Flashner  writes:

> My plan was absolutely to merge master into core-updates after and then
> integrate all the changes into their affected packages. I'd also make
> sure to bump gcc to 8 (assuming we don't go straight to 9).

Sounds good.  If we can get powerpc64le-linux working on master, I'd be
very happy.  We can simultaneously try that while still working on
wip-ppc64le (based on core-updates).

I tried out the wip-ppc64le-for-master branch.  I can build it manually
on my Debian ppc64le system, but "make check" fails.  There are two test
failures.

The first test failure is tests/syscalls.scm.  It seems that Ludo's
recently added mount procedure (7e9d9f2 syscalls: Add 'mounts' and the
 record type.) does not work on my system.  In fact, it does not
work on my Debian ppc64le system, nor does it work on my x86_64 Fedora
system.  In both cases, the code makes the incorrect assumption that the
type and source are located in columns 8 and 9, like so:

--8<---cut here---start->8---
(define (mounts)
  "Return the list of mounts ( records) visible in the namespace of the
current process."
  (define (string->device-number str)
(match (string-split str #\:)
  (((= string->number major) (= string->number minor))
   (+ (* major 256) minor

  (call-with-input-file "/proc/self/mountinfo"
(lambda (port)
  (let loop ((result '()))
(let ((line (read-line port)))
  (if (eof-object? line)
  (reverse result)
  (match (string-tokenize line)
((id parent-id major:minor root mount-point
 options _ _ type source _ ...)
 (let ((devno (string->device-number major:minor)))
   (loop (cons (%mount (octal-decode source)
   (octal-decode mount-point)
   devno type options)
   result)))
--8<---cut here---end--->8---

However, on my systems, the correct columns are 9 and 10.  Here's the
first few lines of output from Fedora:

--8<---cut here---start->8---
$ cat /proc/self/mountinfo 
22 97 0:21 / /sys rw,nosuid,nodev,noexec,relatime shared:2 - sysfs sysfs rw
23 97 0:5 / /proc rw,nosuid,nodev,noexec,relatime shared:24 - proc proc rw
24 97 0:6 / /dev rw,nosuid shared:20 - devtmpfs devtmpfs 
rw,size=3923828k,nr_inodes=980957,mode=755
...
--8<---cut here---end--->8---

And here it is from Debian:

--8<---cut here---start->8---
22 28 0:20 / /sys rw,nosuid,nodev,noexec,relatime shared:7 - sysfs sysfs rw
23 28 0:21 / /proc rw,nosuid,nodev,noexec,relatime shared:11 - proc proc rw
24 28 0:5 / /dev rw,nosuid,noexec,relatime shared:2 - devtmpfs udev 
rw,size=15625664k,nr_inodes=244151,mode=755
...
--8<---cut here---end--->8---

On these systems, the 7th column is an optional field, like "shared:7".
The proc man page has this to say about column 7:

  (7) optional fields: zero or more fields of the form "tag[:value]"

So presumably there can be even more than just one optional field.  In
any case, Leo Le Bouter kindly checked his own x86_64 system, where he
observed different output:

--8<---cut here---start->8---
$ cat /proc/self/mountinfo 
23 27 0:21 / /proc rw,relatime - proc none rw
24 27 0:5 / /dev rw,relatime - devtmpfs none 
rw,size=7972408k,nr_inodes=1993102,mode=755
25 27 0:22 / /sys rw,relatime - sysfs none rw
--8<---cut here---end--->8---

As you can see, in this case there is no optional column, so the mount
procedure works fine on Leo's system.  But it fails on mine.

Ludo, do you have an opinion on how to fix this?  It seems like we can't
assume the number of columns will always be the same, so I guess we'll
have to somehow ignore the optional columns more intelligently, if we
want to keep using string-tokenize to do this.

The second test failure is tests/pack.scm.  There are two contributing
factors to this test failure.

The first contributing factor was commit 8f52ea2 on
wip-ppc64le-for-master.  It fixes an issue that is not present on
master, and for that reason it actually causes a problem if it's
included.  I have removed it from the branch in Savannah; please update
your local copy.

The second contributing factor is this bug:

https://issues.guix.gnu.org/47018

However, we can work around it by simply not running guix-daemon when
running the test.  When guix builds guix, it'll be done in the build
environment, so these problematic tests will probably be skipped.

Finally, I tried running make guix-binary.powerpc64le-linux.tar.xz just
to see how far it would get, anyway.  I was quickly greeted with this
failure when building glibc-intermediate:

--8<---cut 

Re: Release on April 18th?

[Vincent Legoll]

Hello,

I rebuilt guix on core-updates with gcc-8 succesfully
I'll now try the same above wip-ppc64le.

-- 
Vincent Legoll