How to use Guix with sssd, not nscd, on a foreign distro?

[Chris Marusich]

Hi,

The Guix manual recommends running nscd:

https://guix.gnu.org/manual/en/html_node/Application-Setup.html

However, Fedora intends to remove it:

https://fedoraproject.org/wiki/Changes/RemoveNSCD

The document says:

"The hosts cache will automatically be replaced by the one provided by
systemd-resolved. However, in order to restore caching functionality for
other caches provided by nscd, the system administrator will need to
install and/or configure sssd (by enabling sssd with authconfig, and
editing /etc/sssd/sssd.conf to enable it to work with nss)."

This poses a problem for people who use Fedora, like myself.  I tried to
set up sssd on my Fedora machine, but I couldn't get it to work.

Let's take a step back.  Why does the Guix manual recommend using nscd?
The Guix manual explains why in the link above.  To rephrase the manual,
my understanding is that if nscd is available, then a program using
glibc will "try to use nscd" first.  However, if nscd is not available,
then the program will attempt to dlopen shared objects, and in some
cases the program might dlopen an incompatible shared object from the
foreign distro (e.g., libnss_*.so files on Fedora, which may be
incompatible with the glibc used by a program that Guix built).

The Fedora document explains that at least the hosts cache will be
handled by systemd-resolved.  Can I expect Guix-built programs to "try
to use systemd" when resolving host names, or is additional
configuration likely to be required?

Regarding sssd specifically, how can I arrange for a Guix-built program
to "try to use sssd" first?  I know that the specific steps for how to
do this on Fedora might be different from other systems.  For example,
maybe on Fedora there is some fancy authselect/authconfig thing you can
do to configure everything more easily.  That's fine, and I will figure
out what to do at a higher level as needed.  However, for now I just
want to know the very basics: fundamentally, what configuration needs to
exist in order to ensure that Guix-built programs will "use" sssd, in
the same way that they would "use" nscd?  I want to avoid the kind of
problems that the manual discusses, but I want to do it with sssd.

I believe this will require changes to /etc/nsswitch.conf, as well as
configuration for sssd.  Anything else?  I have never written sssd
configuration files, and the sssd manual was not very approachable for
me, so I'm starting from essentially zero knowledge about sssd.  It
seems rather complex.  Has anyone tried setting up sssd and configuring
nsswitch to use it, in order to avoid the kinds of issues that the Guix
manual discusses?

Any tips would be welcome.  Maybe I should just switch back to Guix
System and avoid this headache, but I think there are lots of people out
there who use Fedora, so it would be good for Guix adoption if we can
have a recommended solution ready for people who are curious to try Guix
on Fedora.

-- 
Chris

PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836


signature.asc
Description: PGP signature

Re: License of your contributions to the blog at guix.gnu.org

[Chris Marusich]

Hi,

Ludovic Courtès  writes:

> Hello,
>
> I am emailing you on behalf of the GNU Guix project because you are the
> author or coauthor of one or more articles to the blog at
> .
>
> With a few exceptions, these articles do not have a clear license, which
> we would like to fix.  We propose to dual-license all the articles under
> CC-BY-SA 4.0 and GFDL version 1.3 or later, with no Invariant Sections,
> no Front-Cover Texts, and no Back-Cover Texts.
>
> Do you agree with the proposed licensing terms for your contributions to
> the blog?
>
> If you do, please reply to this message to say so, keeping
> guix-devel@gnu.org Cc’d (if you already replied in the previous thread,
> you do not need to reply again).
>
> If you would prefer different licensing terms, or if you have any
> questions, feel free to ask them publicly on guix-devel@gnu.org or
> privately with guix-maintain...@gnu.org.
>
> The clarified license will allow us and others to reuse material in the
> manual, cookbook, and in other free cultural works.  See
> 
> for the initial discussion.
>
> Thanks in advance!
>
> Ludo’.

I agree with the proposed licensing terms mentioned above.

-- 
Chris

PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836


signature.asc
Description: PGP signature

Re: version-1.4.0 branch updated

[Chris Marusich]

Hi Maxim,

I've CC'd Mathieu on this email because he sent an email recently asking
for help with the 1.4.0 release, and Ludo because he was working on bug
52752.

Maxim Cournoyer  writes:

> Hi Maxime,
>
> Maxime Devos  writes:
>
>> Efraim Flashner schreef op di 11-01-2022 om 12:39 [+0200]:
>>> On Mon, Jan 10, 2022 at 02:57:09PM -0500, Maxim Cournoyer wrote:
>>> > If nobody has another world rebuilding change deemed necessary in time
>>> > for the release, I suggest we enable substitutes on the branch soon, and
>>> > then get busy trying to get 'make dist' to succeed so that we can issue
>>> > a first RC.
>>
>> The fix for a non-deterministic test suite failure in Guile:
>> https://issues.guix.gnu.org/48389#11
>
> That one slipped :-/.  Sorry!  I should have checked my emails more
> carefully before turning on the switch in Cuirass yesterday.

On aarch64-linux, on master, bug 52943 prevents the "guix" package from
building successfully.  I've committed a fix for this on the master
branch in commit 24c3485bb3ffc05e687ef6513ac287b8d3048bab.  However, I
haven't yet updated the "guix" package, since Ludo mentioned here that
he wanted to update the "guix" package to include a different fix (for
bug 52752), but I'm not sure that he's done that yet:

  https://issues.guix.gnu.org/52752#2

I figured we could coordinate and update the "guix" package at the right
time, all at once.

It would be good to ensure that the "guix" package builds successfully
on aarch64-linux for the 1.4.0 release.  Therefore, I'd like to get the
fix for 52943 included in the version-1.4.0 branch, but I don't want to
step on anyone's toes.  What can I do to get the fix included in the
release?

-- 
Chris

PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836


signature.asc
Description: PGP signature

Re: guix fails to build on aarch64

[Chris Marusich]

Hi Pierre,

Thank you very much for the details!  I have added this information to
the existing bug report:

https://issues.guix.gnu.org/52943

Please direct further discussion regarding this bug there, so that
people looking at the bug report will see an accurate record of the
investigation so far.

-- 
Chris

PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836


signature.asc
Description: PGP signature

Re: Release v1.4 (or 2.0): process and schedule ?

[Chris Marusich]

Hi Maxim,

Maxim Cournoyer  writes:

> Hello Chris,
>
> Chris Marusich  writes:
>
>> Maxim Cournoyer  writes:
>>
>>> About the current status, I'm nearing on pushing a version-1.4.0 branch
>>> which is based on master with a few more (core-ish) updates.  There's
>>> still a few days ahead of that, so if you manage to get many of this
>>> kind of problems fixed & merged in master they can easily be included in
>>> the next release.
>>
>> There is a problem that currently prevents "guix pull" from succeeding
>> for powerpc64le-linux on master.  I'd like to resolve it before the
>> release if possible.  The problem is here, including a patch to fix it:
>>
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=52940
>>
>> [...]
>
> Since the version-1.4.0 branch I've been polishing locally still hasn't
> been published nor built, there's still time to apply the patch without
> any consideration for world rebuilds to that branch (at this point we
> want the changes to be low risk ones though, but that sounds like one).

With Ludo's feedback, I was able to fix 52940 without rebuilding the
world in commit 195bb1fb9d55d8e5187d669c63a3cde747fc5f64 on master.  Can
you try merging master into your version-1.4.0 branch?

At this point in time, I'm unaware of any issues that would be
show-stoppers for for powerpc64le-linux.  As a reminder, issues
important to powerpc64le-linux can be seen by searching for issues
tagged with the "guix" user's powerpc64le-linux Debbugs usertag:

https://debbugs.gnu.org/cgi-bin/pkgreport.cgi?tag=powerpc64le-linux;users=guix

I'll keep my eye out for other potential powerpc64le-linux issues and
see what I can do to help in the days/weeks to come.

Now that the "guix" package is building successfully again on master, I
can finally build up-to-date versions of cuirass and
guix-build-coordinator.  Although it isn't directly related to the
release, I think I will resume my efforts to add a new powerpc64le-linux
node to the build farm.

-- 
Chris

PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836


signature.asc
Description: PGP signature

Re: Release v1.4 (or 2.0): process and schedule ?

[Chris Marusich]

Maxim Cournoyer  writes:

> About the current status, I'm nearing on pushing a version-1.4.0 branch
> which is based on master with a few more (core-ish) updates.  There's
> still a few days ahead of that, so if you manage to get many of this
> kind of problems fixed & merged in master they can easily be included in
> the next release.

There is a problem that currently prevents "guix pull" from succeeding
for powerpc64le-linux on master.  I'd like to resolve it before the
release if possible.  The problem is here, including a patch to fix it:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=52940

This patch is relatively simple, but it will rebuild many packages for
all architectures because it modifies guix/build/gremlin.scm, which is
used by the gnu-build-system.  How can I integrate this change into the
1.4.0 release?

Normally I would commit such a change to core-updates.  However, if I do
that, then the change probably won't make it into master or the
version-1.4.0 branch in time.  Is there an opportunity to put the change
somewhere so that it will make it into the release?  I'm not sure.

Here's my proposal.  Since the bug may very well be benign, I could
apply a simple work-around to master that just skips the failing test on
powerpc64le-linux.  I could then apply the actual fix to core-updates.
Later, after master has been merged to core-updates, I could re-enable
the test on core-updates.  This would allow "guix pull" to succeed for
powerpc64le-linux on master without rebuilding the world, and the
correct fix would still be applied on core-updates for a later release.

Do you think that would work?

-- 
Chris

PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836


signature.asc
Description: PGP signature

Re: bug#48941: [powerpc64le-linux] libfaketime CLOCK_MONOTONIC test hangs

[Chris Marusich]

Thiago Jung Bauermann  writes:

> GDB uses the shell to launch the debugged program. That is probably where
> ‘/bin/sh’ is entering the picture here. I don’t know whether that has any 
> relation to your foreign distro’s libc being used.
>
> The output of `help run` in GDB mentions that the shell is specified by the 
> ‘$SHELL’ environment variable. Perhaps you have that set?
>
> One way to see if this is the problem is to use the GDB command
> `set startup-with-shell off` to make it launch the debugged program without 
> the shell.

Thank you for this suggestion.  When I ran "set startup-with-shell off",
gdb successfully launched the program.

It seems something in the Guix-built software is trying to run /bin/sh.
Perhaps there is stray hard-coded path in GDB or something that needs to
be fixed...  In any case, I'm glad I can continue debugging!

>> The x86-64 systems I have that run pure Guix don't have any
>> /lib*/ directories. You might try running gdb with
>> LD_LIBRARY_PATH=/gnu/store/kmblbljiygayhlc5gb02an9imhy90ws9-glibc-2.33/l
>> ib to have the Guix libc.so picked up before the other one. HTH
>
> Another alternative worth trying is the ‘--container’ option to ‘guix 
> environment’, to completely isolate GDB from the foreign distro. You might 
> want to add the coreutils package to the ‘--ad-hoc’ list so that you can 
> get amenities such as an ls command. :-)

When I tried "--container", gdb successfully launched the program.  So
it's another viable work-around for the gdb issue.

As for the actual libfaketime bug, when it hung I pressed Control+C, and
here's the backtrace at that point:

--8<---cut here---start->8---
(gdb) backtrace
#0  0x77eccba0 in __futex_abstimed_wait_common64 
(futex_word=0x77aaf1c0, expected=, clockid=, 
abstime=0x0, private=, cancel=) at 
../sysdeps/nptl/futex-internal.c:74
#1  0x77eb9934 in __pthread_clockjoin_ex (threadid=140737348563184, 
thread_return=0x7fffe2d0, clockid=, abstime=, 
block=) at pthread_join_common.c:102
#2  0x77eb9684 in __pthread_join (threadid=, 
thread_return=) at pthread_join.c:24
#3  0x10001784 in main (argc=1, argv=0x7fffe718) at timetest.c:146
--8<---cut here---end--->8---

Here's the entire session in which I built the problematic test and
debugged it, including a full back trace with local variables:

--8<---cut here---start->8---
[0] [env] marusich@suzaku:/tmp/guix-build-libfaketime-0.9.9.drv-0
$ ~/guix-core-updates/pre-inst-env guix environment libfaketime --ad-hoc 
gcc-toolchain@10.3.0:debug gdb
[0] [env] marusich@suzaku:/tmp/guix-build-libfaketime-0.9.9.drv-0
$ cd source/
[0] [env] marusich@suzaku:/tmp/guix-build-libfaketime-0.9.9.drv-0/source
$ make FAKETIME_COMPILE_CFLAGS='-DFORCE_MONOTONIC_FIX -g' CC=gcc 
PREFIX=/tmp/guix-build-libfaketime-0.9.9.drv-0/myprefix clean
make  -C src clean
make[1]: Entering directory '/tmp/guix-build-libfaketime-0.9.9.drv-0/source/src'
make[1]: Leaving directory '/tmp/guix-build-libfaketime-0.9.9.drv-0/source/src'
make  -C test clean
make[1]: Entering directory 
'/tmp/guix-build-libfaketime-0.9.9.drv-0/source/test'
make[1]: Leaving directory '/tmp/guix-build-libfaketime-0.9.9.drv-0/source/test'
[0] [env] marusich@suzaku:/tmp/guix-build-libfaketime-0.9.9.drv-0/source
$ make FAKETIME_COMPILE_CFLAGS='-DFORCE_MONOTONIC_FIX -g' CC=gcc 
PREFIX=/tmp/guix-build-libfaketime-0.9.9.drv-0/myprefix
make  -C src all
make[1]: Entering directory '/tmp/guix-build-libfaketime-0.9.9.drv-0/source/src'
gcc -o libfaketime.o -c -std=gnu99 -Wall -Wextra -Werror -Wno-nonnull-compare 
-DFAKE_PTHREAD -DFAKE_STAT -DFAKE_UTIME -DFAKE_SLEEP -DFAKE_TIMERS 
-DFAKE_INTERNAL_CALLS -fPIC 
-DPREFIX='"'/tmp/guix-build-libfaketime-0.9.9.drv-0/myprefix'"' 
-DLIBDIRNAME='"'/lib/faketime'"' -DFORCE_MONOTONIC_FIX -g   libfaketime.c
gcc -o libfaketime.so.1 -Wl,-soname,libfaketime.so.1  -lpthread 
-Wl,--version-script=libfaketime.map -shared libfaketime.o -ldl -lm -lrt
gcc -o libfaketimeMT.o -c -std=gnu99 -Wall -Wextra -Werror -Wno-nonnull-compare 
-DFAKE_PTHREAD -DFAKE_STAT -DFAKE_UTIME -DFAKE_SLEEP -DFAKE_TIMERS 
-DFAKE_INTERNAL_CALLS -fPIC 
-DPREFIX='"'/tmp/guix-build-libfaketime-0.9.9.drv-0/myprefix'"' 
-DLIBDIRNAME='"'/lib/faketime'"' -DFORCE_MONOTONIC_FIX -g  
-DPTHREAD_SINGLETHREADED_TIME libfaketime.c
gcc -o libfaketimeMT.so.1 -Wl,-soname,libfaketimeMT.so.1  -lpthread 
-Wl,--version-script=libfaketime.map -shared libfaketimeMT.o -ldl -lm -lrt
gcc -o faketime -std=gnu99 -Wall -Wextra -Werror -Wno-nonnull-compare 
-DFAKE_PTHREAD -DFAKE_STAT -DFAKE_UTIME -DFAKE_SLEEP -DFAKE_TIMERS 
-DFAKE_INTERNAL_CALLS -fPIC 
-DPREFIX='"'/tmp/guix-build-libfaketime-0.9.9.drv-0/myprefix'"' 
-DLIBDIRNAME='"'/lib/faketime'"' -DFORCE_MONOTONIC_FIX -g   faketime.c  
-lpthread -Wl,--version-script=libfaketime.map -lrt
make[1]: Leaving directory '/tmp/guix-build-libfaketime-0.9.9.drv-0/source/src'
[0] [env] 

Re: bug#48941: [powerpc64le-linux] libfaketime CLOCK_MONOTONIC test hangs

[Chris Marusich]

Hi Kaelyn,

Thank you for the reply.

Kaelyn  writes:

> Are you using Guix on a foreign distro? This line looks like your
> distro's normal libc.so was being used and it was from glibc-2.31 or
> older. The x86-64 systems I have that run pure Guix don't have any
> /lib*/ directories. You might try running gdb with
> LD_LIBRARY_PATH=/gnu/store/kmblbljiygayhlc5gb02an9imhy90ws9-glibc-2.33/lib
> to have the Guix libc.so picked up before the other one. HTH

Yes, I'm using Guix on a foreign distro.  It isn't clear to me what is
trying to access the /lib directories.  That's what I'm trying to figure
out.  In a --pure environment using only Guix-built tools, one would
expect that I would not have to set LD_LIBRARY_PATH.  However, if I
can't figure it out, that's another trick I can try.

-- 
Chris


signature.asc
Description: PGP signature

Re: bug#48941: [powerpc64le-linux] libfaketime CLOCK_MONOTONIC test hangs

[Chris Marusich]

Hi,

I need a little help figuring out how to use gdb in Guix for bug 48941:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=48941

Here's the situation.  A libfaketime test hangs forever.  Upstream
suggested I debug it.  I'm trying to, but gdb errors out.  What am I
doing wrong?  It's probably something simple, but I can't see what.

I'll describe what I've done.  First, I started a build like so:

  ./pre-inst-env guix build --keep-failed libfaketime

While the problematic test hung, I found the PID of the test and killed
it.  This caused the build to fail, leaving the build environment for me
to play around in.

I entered a pure environment that contains all the things I need to
debug the test (gcc 10.3.0 is currently the default gcc on
core-updates):

  ./pre-inst-env guix environment --pure libfaketime --ad-hoc 
gcc-toolchain@10.3.0 gcc-toolchain@10.3.0:debug gdb

In the pure environment, I confirmed I can build and run the hanging
test via the following commands (I added -g in order to get debug
symbols):

  make FAKETIME_COMPILE_CFLAGS='-DFORCE_MONOTONIC_FIX -g' CC=gcc 
PREFIX=/tmp/guix-build-libfaketime-0.9.9.drv-0/myprefix
  make FAKETIME_COMPILE_CFLAGS='-DFORCE_MONOTONIC_FIX -g' CC=gcc 
PREFIX=/tmp/guix-build-libfaketime-0.9.9.drv-0/myprefix test

OK, so I can trigger the hang.  Great!  Next step, fire up GDB:

--8<---cut here---start->8---
[0] [env] marusich@suzaku:/tmp/guix-build-libfaketime-0.9.9.drv-0/source/test
$ gdb ./timetest
GNU gdb (GDB) 10.2
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "powerpc64le-unknown-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./timetest...
(gdb) 
--8<---cut here---end--->8---

The debug symbols provided by gcc-toolchain@10.3.0:debug are under
$GUIX_ENVIRONMENT/lib/debug.  This is the value of GUIX_ENVIRONMENT:

--8<---cut here---start->8---
$ echo $GUIX_ENVIRONMENT
/gnu/store/32fjhp30k34fh0g9f1gmgcj8pc5wldq6-profile
--8<---cut here---end--->8---

By the way, this directory corresponds to glibc 2.33:

--8<---cut here---start->8---
$ realpath /gnu/store/32fjhp30k34fh0g9f1gmgcj8pc5wldq6-profile/lib/debug
/gnu/store/8akrlhc25d7xvi85gzvginw0vdi4zyg4-glibc-2.33-debug/lib/debug
--8<---cut here---end--->8---

Let's tell GDB where to find those debug symbols:

  (gdb) set debug-file-directory 
/gnu/store/32fjhp30k34fh0g9f1gmgcj8pc5wldq6-profile/lib/debug

Let's also tell GDB to set the environment variables that upstream
recommended when running the test program:

--8<---cut here---start->8---
(gdb) set environment LD_PRELOAD=../src/libfaketime.so.1
(gdb) set environment FAKETIME=-10d
(gdb) set environment NO_FAKE_STAT=1
--8<---cut here---end--->8---

Now run it:

--8<---cut here---start->8---
(gdb) run
Starting program: /tmp/guix-build-libfaketime-0.9.9.drv-0/source/test/timetest 
/bin/sh: /lib/powerpc64le-linux-gnu/libc.so.6: version `GLIBC_2.33' not found 
(required by ../src/libfaketime.so.1)
/bin/sh: /lib/powerpc64le-linux-gnu/libc.so.6: version `GLIBC_2.32' not found 
(required by 
/gnu/store/kmblbljiygayhlc5gb02an9imhy90ws9-glibc-2.33/lib/libpthread.so.0)
/bin/sh: /lib/powerpc64le-linux-gnu/libc.so.6: version `GLIBC_2.32' not found 
(required by 
/gnu/store/kmblbljiygayhlc5gb02an9imhy90ws9-glibc-2.33/lib/libdl.so.2)
/bin/sh: /lib/powerpc64le-linux-gnu/libc.so.6: version `GLIBC_2.32' not found 
(required by 
/gnu/store/kmblbljiygayhlc5gb02an9imhy90ws9-glibc-2.33/lib/librt.so.1)
During startup program exited with code 1.
(gdb) 
--8<---cut here---end--->8---

Huh?  What happened?  I've double checked that I'm using gdb provided by
Guix:

--8<---cut here---start->8---
$ type -P gdb
/gnu/store/32fjhp30k34fh0g9f1gmgcj8pc5wldq6-profile/bin/gdb
--8<---cut here---end--->8---

I also tried running gdb by invoking it via that absolute file name, and
it still errored out in the same way.

I'm operating in a --pure environment.  All the tools are provided by
Guix.  I'm surprised that /bin/sh and /lib are even mentioned above.

If anyone can 

Re: bug#49337: Guix pull fails on my Talos II

[Chris Marusich]

Hi Tobias,

Were you able to "guix pull" successfully?

-- 
Chris


signature.asc
Description: PGP signature

Re: Debbugs Usertags - which user to use?

[Chris Marusich]

Maxim Cournoyer  writes:

> Hello,
>
> Chris Marusich  writes:
>
> [...]
>
>> Yes, my understanding is that the user "guix" would also work.
>>
>> I think it would be fine to use the user "guix" instead of
>> "guix-devel@gnu.org".  Perhaps it would have the added benefit of
>> eliminating any possible risk of accidentally spamming the email list,
>> although so far that hasn't happened.
>>
>> To give others time to voice their opinion, I'll plan to update the
>> manual to suggest using the "guix" user about 2 weeks from now.  At that
>> time, I'll also re-tag the handful of issues currently tagged with the
>> powerpc64le-linux usertag associated with the guix-devel@gnu.org user.
>
> Sounds good to me!
>
> Thank you,
>
> Maxim

Done in 586136d12745eeddccd05d80dbd21959595b45d1.

-- 
Chris


signature.asc
Description: PGP signature

Re: bug#49337: Guix pull fails on my Talos II

[Chris Marusich]

Hi,

FYI, I was able to successfully "guix pull" from 83f8b6d (Apr 09) to
c33e200 (Jul 02) just now on my own POWER9 machine (a Blackbird).

-- 
Chris


signature.asc
Description: PGP signature

Re: bug#49337: Guix pull fails on my Talos II

[Chris Marusich]

Hi Tobias,

Tobias Platen  writes:

> /gnu/store/vc2j3816jx9z4vgrw6zmk357xrka3zy0-texlive-latex-amsmath-51265.drv 
> wird erstellt …
> \ „build“-PhaseBacktrace:
>   13 (primitive-load 
> "/gnu/store/y5sniqlw2x5aaixyk4bw162v2a7bwdpl-compute-guix-derivation")
> In ice-9/eval.scm:
> 155:9 12 (_ _)
> 159:9 11 (_ #(#(#(#(#(#(#(#(#(#(#(#(#(#(#(#(# ?) ?) 
> ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?))
> In ice-9/boot-9.scm:
> 152:2 10 (with-fluid* _ _ _)
> 152:2  9 (with-fluid* _ _ _)
> In ./guix/store.scm:
>   2090:24  8 (run-with-store # _ 
> #:guile-for-build _ #:system _ #:target _)
>1927:8  7 (_ _)
> In ./guix/gexp.scm:
>275:18  6 (_ _)
>1156:2  5 (_ _)
>1022:2  4 (_ _)
> 868:4  3 (_ _)
> In ./guix/store.scm:
>   1975:12  2 (_ #)
>1377:5  1 (map/accumulate-builds # _ 
> _)
>   1388:15  0 (_ # _ _)
>
> ./guix/store.scm:1388:15: ERROR:
>   1. :
>   message: "build of 
> `/gnu/store/bbvxcaa31h006wg2kp1ysly55hkm56hn-po4a-0.63.drv' failed"
>   status: 100
> guix pull: error: You found a bug: the program 
> '/gnu/store/y5sniqlw2x5aaixyk4bw162v2a7bwdpl-compute-guix-derivation'
> failed to compute the derivation for Guix (version: 
> "6243ad3812f8c689599a19f0e8b9719ba14461f2"; system: "powerpc64le-linux";
> host version: "1.3.0"; pull-version: 1).
> Please report it by email to .

Thanks for the report!  I'll try reproducing it locally, but in the
meantime, if you could provide the following information, it would be
helpful.

What is the output of "guix describe" on your Talos II?

When you ran "guix pull", did you just run "guix pull" or did you pass
some options in?  I.e., were you trying to pull to a specific branch or
commit?  If you are pulling using a channel file (e.g.,
~/.config/guix/channels.scm), could you share it?

Does "guix build po4a" successfully build po4a?  Does "guix build -d
po4a" show the same derivation as above
(/gnu/store/bbvxcaa31h006wg2kp1ysly55hkm56hn-po4a-0.63.drv)?

-- 
Chris


signature.asc
Description: PGP signature

Debbugs Usertags - which user to use? (Was: Re: GNU Guix 1.4.0 in September?)

[Chris Marusich]

Hi Maxim,

Maxim Cournoyer  writes:

> Hello Chris,
>
> Chris Marusich  writes:
>
> [...]
>
>> I intend to try to get POWER9 working on core-updates before the next
>> release.  Hopefully the recent upgrade to GCC 10 will make it easier.
>
> Neat!
>
>> Anyone who wants to help with powerpc64le-linux is, of course, more than
>> welcome to coordinate and lend a hand!  I will be tagging bug reports
>> specific to powerpc64le-linux with the "powerpc64le-linux" usertag, for
>> the user "guix-devel@gnu.org".  You can see the open bugs tagged thusly
>> here:
>
> I'm rather late to the party, but may I suggest simply using 'guix' as
> the user?  It was made possible to do so as the result of me bothering
> the very nice GNU people maintaining emacs-debbugs / GNU debbugs
> instance (there used to be a limitation for 5 letters+ projects so that
> 'emacs' would be usable as a user).
>
> C-h f debbugs-gnu-usertags also has the docstring "List all user tags
> for USERS, which is \(\"emacs\"\) by default." and later the input text
> "Package name(s) or email address: ", which suggest 'guix' should be
> valid.
>
> Thanks for this nice initiatives of yours!
>
> Maxim

Yes, my understanding is that the user "guix" would also work.

I think it would be fine to use the user "guix" instead of
"guix-devel@gnu.org".  Perhaps it would have the added benefit of
eliminating any possible risk of accidentally spamming the email list,
although so far that hasn't happened.

To give others time to voice their opinion, I'll plan to update the
manual to suggest using the "guix" user about 2 weeks from now.  At that
time, I'll also re-tag the handful of issues currently tagged with the
powerpc64le-linux usertag associated with the guix-devel@gnu.org user.

-- 
Chris


signature.asc
Description: PGP signature

Re: Debbugs user tags

[Chris Marusich]

Ludovic Courtès  writes:

>> * doc/contributing.texi (Contributing): Update the short description of the
>> "Tracking Bugs and Patches" chapter in the menu.
>> (Tracking Bugs and Patches): Split this section into three new subsections,
>> titled "Debbugs", "Debbugs User Interfaces", and "Debbugs Usertags".  Of
>> these, only the "Debbugs Usertags" is actually new.
>
> Wonderful!

I'm glad you like it!  :-)

>> +++ b/doc/contributing.texi
>> @@ -26,7 +26,7 @@ choice.
>>  * Packaging Guidelines::Growing the distribution.
>>  * Coding Style::Hygiene of the contributor.
>>  * Submitting Patches::  Share your work.
>> -* Tracking Bugs and Patches::   Using Debbugs.
>> +* Tracking Bugs and Patches::   Keeping it all organized.
>
> You probably have to adjust it in a second place (yeah…).

Other references to "Tracking Bugs and Patches" do exist, but they don't
mention "Using Debbugs," so I don't think they need to be updated.

>> +@menu
>> +* Debbugs:: The official bug and patch tracker.
>> +* Debbugs User Interfaces:: Ways to interact with Debbugs.
>> +* Debbugs Usertags:: Tag reports with custom labels.
>> +@end menu
>
> ‘M-x texinfo-make-menu’ or similar should indent it for you.  :-)

You learn something new every day!

>> +@node Debbugs
>> +@subsection Debbugs
>
> Maybe rename to “The Issue Tracker”, so as to not require knowledge of
> what Debbugs is when skimming through the table of contents?

Sounds good.  I just changed the first subsection, so now it looks like
this:

--8<---cut here---start->8---
@menu
* The Issue Tracker::   The official bug and patch tracker.
* Debbugs User Interfaces:: Ways to interact with Debbugs.
* Debbugs Usertags::Tag reports with custom labels.
@end menu
--8<---cut here---end--->8---

This seems fine, since the first subsection introduces Debbugs.

>> +Debbugs provides a feature called ``usertags'' that allows any user to
>
> @dfn{usertags}

OK.

>> +In Guix, we are experimenting with usertags to keep track of
>> +architecture-specific issues.  To facilitate collaboration, all our
>> +usertags are associated with the single user @code{guix-devel@@gnu.org}.
>> +The following usertags currently exist for that user:
>> +
>> +@table @code
>> +
>> +@item powerpc64le-linux
>> +The purpose of this usertag is to make it easy to find the issues that
>> +matter most for the @code{powerpc64le-linux} system type.  Please assign
>> +this usertag to issues or patches that affect @code{powerpc64le-linux}
>> +but not other system types.  In addition, you may use it to identify
>> +issues that for some reason are particularly important for the
>> +@code{powerpc64le-linux} system type, even if the issue affects other
>> +system types, too.
>
> Maybe we can already add ‘reproducibility’.
>
> Otherwise LGTM, thank you!
>
> Ludo’.

I added this text:

--8<---cut here---start->8---
@item reproducibility
For issues related to reproducibility.  For example, it would be
appropriate to assign this usertag to a bug report for a package that
fails to build reproducibly.
--8<---cut here---end--->8---

I've gone ahead and committed these changes in
3c86372e36cecaed1ad55675ce32a14b972406bf.  I verified that the Texinfo
manual could be built, and that when viewing it in Info, it looks as
expected.

Please let me know if you'd like any further changes; I'd be happy to
make them.

-- 
Chris


signature.asc
Description: PGP signature

Re: Authenticating maintenance.git

[Chris Marusich]

Chris Marusich  writes:

> Hi Ludo,
>
> Ludovic Courtès  writes:
>
>> It looks like you’re missing a local ‘keyring’ branch for that repo, no?
>>
>> I think you need to run:
>>
>>   git fetch
>>   git branch --track keyring
>
> This works, basically.  Thank you!

Although I was now able to run the pre-push hook, it seems unaware of my
PGP key.  I tried making the attached change to the README and testing a
push via "git push -n origin", and it complained about the signature:

--8<---cut here---start->8---
$ git push -n origin
Authenticating commits 8a7e10b to 413b8f1 (1 new commits)...
[##]guix
 git: error: could not authenticate commit 
413b8f1c6d9ca2160d7aa8d80db181a6f39d3d82: key CBF5 9755 CBE7 E7EF EF18  3FB1 
DD40 9A15 D822 469D is missing
error: failed to push some refs to 
'git.savannah.gnu.org:/srv/git/guix/maintenance.git'
--8<---cut here---end--->8---

However, the signature looks good to me:

--8<---cut here---start->8---
$ git verify-commit 413b8f1c6d9ca2160d7aa8d80db181a6f39d3d82
gpg: Signature made Tue 22 Jun 2021 05:54:13 PM PDT
gpg:    using RSA key CBF59755CBE7E7EFEF183FB1DD409A15D822469D
gpg: Good signature from "Chris Marusich " [ultimate]
--8<---cut here---end--->8---

GnuPG reports it can find the keys:

--8<---cut here---start->8---
$ gpg --list-keys 'CBF5 9755 CBE7 E7EF EF18  3FB1 DD40 9A15 D822 469D'
pub   rsa4096 2016-02-19 [SC] [expires: 2021-08-13]
  CBF59755CBE7E7EFEF183FB1DD409A15D822469D
uid   [ultimate] Chris Marusich 
sub   rsa4096 2016-02-19 [E] [expires: 2021-08-13]
--8<---cut here---end--->8---

This happens even if I update guix with "guix pull".  Any idea what the
problem might be?

-- 
Chris
From 413b8f1c6d9ca2160d7aa8d80db181a6f39d3d82 Mon Sep 17 00:00:00 2001
From: Chris Marusich 
Date: Tue, 22 Jun 2021 17:51:07 -0700
Subject: [PATCH] README: Clarify that pre-push hook needs keyring.

* README: Explain that the pre-push hook requires the existence of a
local keyring branch, and add a "git branch" command to show how to
create one.
---
 README | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/README b/README
index 338895b..71dc641 100644
--- a/README
+++ b/README
@@ -1,8 +1,10 @@
 This repository is meant to contain documents and tools by Guix hackers
 and maintainers that do not fit in the Guix repository.
 
-If you’re a committer, please install this pre-push hook:
+If you’re a committer, please create a local keyring branch that
+tracks origin/keyring and install this pre-push hook:
 
+git branch --track keyring origin/keyring
 cat > .git/hooks/pre-push <

signature.asc
Description: PGP signature

Re: Debbugs user tags

[Chris Marusich]

Ludovic Courtès  writes:

>> Anyone who wants to help with powerpc64le-linux is, of course, more than
>> welcome to coordinate and lend a hand!  I will be tagging bug reports
>> specific to powerpc64le-linux with the "powerpc64le-linux" usertag, for
>> the user "guix-devel@gnu.org".  You can see the open bugs tagged thusly
>> here:
>>
>> https://debbugs.gnu.org/cgi-bin/pkgreport.cgi?tag=powerpc64le-linux;users=guix-devel@gnu.org
>>
>> For details on usertags, start here:
>>
>> https://lwn.net/Articles/150658/
>
> Should we add some text (and conventions?) to “Tracking Bugs and
> Patches” in the manual about usertags?  Sounds like it could be useful.

How's this?

-- 
Chris
From f640132745b26b19bd163bc67482e8aea041881b Mon Sep 17 00:00:00 2001
From: Chris Marusich 
Date: Tue, 22 Jun 2021 19:44:18 -0700
Subject: [PATCH] Document the use of Debbugs usertags.

* doc/contributing.texi (Contributing): Update the short description of the
"Tracking Bugs and Patches" chapter in the menu.
(Tracking Bugs and Patches): Split this section into three new subsections,
titled "Debbugs", "Debbugs User Interfaces", and "Debbugs Usertags".  Of
these, only the "Debbugs Usertags" is actually new.
---
 doc/contributing.texi | 61 ++-
 1 file changed, 60 insertions(+), 1 deletion(-)

diff --git a/doc/contributing.texi b/doc/contributing.texi
index e612ea7b23..6a287fe6a4 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -26,7 +26,7 @@ choice.
 * Packaging Guidelines::Growing the distribution.
 * Coding Style::Hygiene of the contributor.
 * Submitting Patches::  Share your work.
-* Tracking Bugs and Patches::   Using Debbugs.
+* Tracking Bugs and Patches::   Keeping it all organized.
 * Commit Access::   Pushing to the official repository.
 * Updating the Guix Package::   Updating the Guix package definition.
 * Translating Guix::Make Guix speak your native language.
@@ -1223,6 +1223,18 @@ for more information.  You can install @command{git send-email} with
 @node Tracking Bugs and Patches
 @section Tracking Bugs and Patches
 
+This section describes how the Guix project tracks its bug reports and
+patch submissions.
+
+@menu
+* Debbugs:: The official bug and patch tracker.
+* Debbugs User Interfaces:: Ways to interact with Debbugs.
+* Debbugs Usertags:: Tag reports with custom labels.
+@end menu
+
+@node Debbugs
+@subsection Debbugs
+
 @cindex bug reports, tracking
 @cindex patch submissions, tracking
 @cindex issue tracking
@@ -1234,6 +1246,9 @@ email to @email{bug-guix@@gnu.org}, while patch submissions are filed
 against the @code{guix-patches} package by sending email to
 @email{guix-patches@@gnu.org} (@pxref{Submitting Patches}).
 
+@node Debbugs User Interfaces
+@subsection Debbugs User Interfaces
+
 A web interface (actually @emph{two} web interfaces!) are available to
 browse issues:
 
@@ -1271,6 +1286,50 @@ For example, to list all open issues on @code{guix-patches}, hit:
 @xref{Top,,, debbugs-ug, Debbugs User Guide}, for more information on
 this nifty tool!
 
+@node Debbugs Usertags
+@subsection Debbugs Usertags
+
+@cindex usertags, for debbugs
+@cindex Debbugs usertags
+Debbugs provides a feature called ``usertags'' that allows any user to
+tag any bug with an arbitrary label.  Bugs can be searched by usertag,
+so this is a handy way to organize bugs.@footnote{The list of usertags
+is public information, and anyone can modify any user's list of
+usertags, so keep that in mind if you choose to use this feature.}
+
+For example, to view all the bug reports (or patches, in the case of
+guix-patches) tagged with the usertag @code{powerpc64le-linux} for the
+user @code{guix-devel@@gnu.org}, open a URL like the following in a web
+browser:
+@url{https://debbugs.gnu.org/cgi-bin/pkgreport.cgi?tag=powerpc64le-linux;users=guix-devel@@gnu.org}
+
+For more information on how to use usertags, please refer to the
+documentation for Debbugs or the documentation for whatever tool you use
+to interact with Debbugs.
+
+In Guix, we are experimenting with usertags to keep track of
+architecture-specific issues.  To facilitate collaboration, all our
+usertags are associated with the single user @code{guix-devel@@gnu.org}.
+The following usertags currently exist for that user:
+
+@table @code
+
+@item powerpc64le-linux
+The purpose of this usertag is to make it easy to find the issues that
+matter most for the @code{powerpc64le-linux} system type.  Please assign
+this usertag to issues or patches that affect @code{powerpc64le-linux}
+but not other system types.  In addition, you may use it to identify
+issues that for some reason are particularly important for the
+@code{powerpc64le-linux} system type, even if the issue affects other
+system types, too.
+
+@end table
+
+If you'r

Re: Authenticating maintenance.git

[Chris Marusich]

Hi Ludo,

Ludovic Courtès  writes:

> It looks like you’re missing a local ‘keyring’ branch for that repo, no?
>
> I think you need to run:
>
>   git fetch
>   git branch --track keyring

This works, basically.  Thank you!

Details: When master is currently checked out, that "git branch" command
actually creates a local branch named "keyring" that tracks my local
"master" branch, which is probably not what you meant I should do.  In
the end, "git branch --track keyring origin/keyring" worked for me: it
created a local branch named "keyring" that tracks remote branch
"origin/keyring".  After that, I was able to run the pre-push hook
without issue!

-- 
Chris


signature.asc
Description: PGP signature

Re: Freezing ‘core-updates’ soon?

[Chris Marusich]

Ludovic Courtès  writes:

> Hello Guix!
>
> What about finally merging that ‘core-updates’ branch?  :-)
>
> The main things to decide on are:
>
>   • Upgrading to GCC 10?  I know Marius has spent time looking at it and
> fixing build failures caused by the upgrade.  Can we do it?
>
>   • Reduced binary seeds—anything new?  My understanding is that the
> reduced binary seed bootstrap now works on ARM, but that we were
> waiting on a Mes release to merge those bits.  Janneke, Danny?
>
>   • Simplified package inputs—I’ll keep working on that, and most of the
> work can be done without a world rebuild, so it’s not a blocker IMO.
>
> Anything else?  Any patches pending review?
>
> It would be great to freeze within a week (that is, stop world-rebuild
> changes) with the goal of merging within a six weeks (end of July).
>
> Thoughts?
>
> Ludo’.

I'm taking some time to test out powerpc64le-linux on core-updates.  At
the very least, I'd like to be able to "guix pull" on core-updates and
use the pulled guix to build some basic stuff.  Last time I tested this
was when we were using gcc-8, so I expect some things are different now.

I would also like to go go back and remove some of the conditionals that
we added in order to support powerpc64le-linux on master.  Then we could
delete the wip-ppc64le branch for good.

-- 
Chris


signature.asc
Description: PGP signature

Re: GNU Guix 1.4.0 in September?

[Chris Marusich]

Maxim Cournoyer  writes:

> Perhaps we can aim for the next release mid-September (core-updates).
> I'm not too sure of the status of core-updates right now, but last time
> I worked on it was in a rather good state.

I intend to try to get POWER9 working on core-updates before the next
release.  Hopefully the recent upgrade to GCC 10 will make it easier.

Anyone who wants to help with powerpc64le-linux is, of course, more than
welcome to coordinate and lend a hand!  I will be tagging bug reports
specific to powerpc64le-linux with the "powerpc64le-linux" usertag, for
the user "guix-devel@gnu.org".  You can see the open bugs tagged thusly
here:

https://debbugs.gnu.org/cgi-bin/pkgreport.cgi?tag=powerpc64le-linux;users=guix-devel@gnu.org

For details on usertags, start here:

https://lwn.net/Articles/150658/

-- 
Chris


signature.asc
Description: PGP signature

Re: Authenticating maintenance.git

[Chris Marusich]

Ludovic Courtès  writes:

> Hello Guix!
>
> I’ve added a ‘.guix-authorizations’ file in maintenance.git, at last!
> We can now authenticate the repository we’ve checked out:
>
>   guix git authenticate 8a7e10b447b574279a7016ae6ea15bc7bcd46253 \
> "3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5" --stats
>
> It’s also possible to authenticate all changes made to the repo since
> the first signed commit in July 2016 by running:
>
>   guix git authenticate 7f59985566b384e31da7e6f1a36744e9edfba54f \
> "3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5" \
> --historical-authorizations=historical-authorizations
>
> If you ran the first command above before, you might want to clear your
> authentication cache with:
>
>   rm -rf ~/.cache/guix/authentication/checkouts
>
> Note that ‘.guix-authorizations’ is a subset of the one on the main Guix
> repository, but we can add people as needed.  I invite committers to
> install the pre-push hook as mentioned in README:
>
>   https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/README
>
> Happy hacking!
>
> Ludo’.

I'm late to the party, but I notice that when I run this in
guix-maintenance, I get an error:

--8<---cut here---start->8---
$ guix git authenticate 8a7e10b447b574279a7016ae6ea15bc7bcd46253 "3CE4 6455 
8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5"
guix git: error: Git error: cannot locate remote-tracking branch 'keyring'
--8<---cut here---end--->8---

Am I doing something wrong?

-- 
Chris


signature.asc
Description: PGP signature

Re: GNU Guix 1.3.0rc2 available for testing!

[Chris Marusich]

Hi Maxim,

Maxim Cournoyer  writes:

> Sorry for the delayed answer.

No worries!  I've waited even longer in replying to you now, so we're
even.  :-)

> The go importer depends on a recent version of guile-lib (0.2.7), which
> added a new #:strict argument to the HTML parser.  We should probably
> skip the test depending on the already available HAVE_GUILE_LIB Automake
> conditional, like so:
>
> modified   Makefile.am
> @@ -457,7 +457,6 @@ SCM_TESTS =   \
>tests/git-authenticate.scm \
>tests/glob.scm \
>tests/gnu-maintenance.scm  \
> -  tests/go.scm   \
>tests/grafts.scm   \
>tests/graph.scm\
>tests/gremlin.scm  \
> @@ -505,6 +504,10 @@ SCM_TESTS =  \
>tests/uuid.scm \
>tests/workers.scm
>  
> +if HAVE_GUILE_LIB
> +SCM_TESTS += tests/go.scm
> +endif
> +
>  if BUILD_DAEMON_OFFLOAD
>  SCM_TESTS  += tests/offload.scm
>  else
>
> Could you give the above a try?  Feel free to commit it if it works as
> expected.

I tested this, and it works.  Thank you!  I can now build the release
(with this change) and run the tests (make check) successfully;
tests/go.scm is just omitted from the tests to run.

By chance, I noticed the following lines below what you added:

--8<---cut here---start->8---
if HAVE_GUILE_LIB
SCM_TESTS += tests/go.scm
endif

if BUILD_DAEMON_OFFLOAD
SCM_TESTS  += tests/offload.scm
else
EXTRA_DIST += tests/offload.scm
endif
--8<---cut here---end--->8---

I guess that if we omit tests/go.scm from SCM_TESTS, it not only means
that the test won't be run, but it also means the test won't get
included in the tarball distribution ("make dist").  Is that right?  It
seems undesirable to omit this test from the distribution just because
the machine on which the distribution was built might have lacked the
library necessary to run the test.  Someone who builds Guix from the
distribution might actually have that library installed and thus be able
to run the test.  To ensure that this test always gets included in the
distribution, perhaps we should also add it to EXTRA_DIST like so:

--8<---cut here---start->8---
diff -u a/Makefile.am b/Makefile.am
--- a/Makefile.am   2021-05-11 11:09:31.0 -0700
+++ b/Makefile.am   2021-06-02 12:55:06.134793001 -0700
@@ -457,7 +457,6 @@
   tests/git-authenticate.scm   \
   tests/glob.scm   \
   tests/gnu-maintenance.scm\
-  tests/go.scm \
   tests/grafts.scm \
   tests/graph.scm  \
   tests/gremlin.scm\
@@ -505,6 +504,12 @@
   tests/uuid.scm   \
   tests/workers.scm
 
+if HAVE_GUILE_LIB
+SCM_TESTS += tests/go.scm
+else
+EXTRA_DIST += tests/go.scm
+endif
+
 if BUILD_DAEMON_OFFLOAD
 SCM_TESTS  += tests/offload.scm
 else
--8<---cut here---end--->8---

What do you think?

-- 
Chris


signature.asc
Description: PGP signature

Re: GNU Guix 1.3.0rc2 available for testing!

[Chris Marusich]

Hi,

Maxim Cournoyer  writes:

> A second RC for the upcoming 1.3.0 release is now available for testing:

Thank you for preparing it!

I tested the binary installation using the guix-install.sh script for
1.3.0 (not the rc2 candidate, but the actual 1.3.0 release, which I
noticed was on the FTP server already).  I tested on powerpc64le-linux
and found no major issues; it worked as expected.

I did "guix pull" and "guix build hello".  To my surprise, I received a
substitute:

marusich@guixtestbed:~$ guix build hello
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
0.1 MB will be downloaded:
   /gnu/store/zcs3cj0mqixwng2ldf92haab2vkpsicb-hello-2.10
substituting /gnu/store/zcs3cj0mqixwng2ldf92haab2vkpsicb-hello-2.10...
downloading from 
https://ci.guix.gnu.org/nar/lzip/zcs3cj0mqixwng2ldf92haab2vkpsicb-hello-2.10 ...
 hello-2.10  52KiB239KiB/s 00:00 [##] 100.0%
/gnu/store/zcs3cj0mqixwng2ldf92haab2vkpsicb-hello-2.10

I guess something is building powerpc64le-linux substitutes?  I had
thought no substitutes would be available, but certainly it is not a
problem if substitutes are being built for powerpc64le-linux already.

I tried building from source on Debian 10 buster ppc64el.  It succeeded,
but "make check" reported one test failure.  It was in tests/go.scm:

--8<---cut here---start->8---
test-name: go-module->guix-package
location: /home/marusich/guix-1.3.0/tests/go.scm:254
source:
+ (test-equal
+   "go-module->guix-package"
+   '(package
+  (name "go-github-com-go-check-check")
+  (version "0.0.0-20201130134442-10cb98267c6c")
+  (source
+(origin
+  (method git-fetch)
+  (uri (git-reference
+ (url "https://github.com/go-check/check;)
+ (commit (go-version->git-ref version
+  (file-name (git-file-name name version))
+  (sha256
+(base32
+  "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5"
+  (build-system go-build-system)
+  (arguments
+'(#:import-path "github.com/go-check/check"))
+  (propagated-inputs
+`(("go-github-com-kr-pretty"
+   ,go-github-com-kr-pretty)))
+  (home-page "https://github.com/go-check/check;)
+  (synopsis "Instructions")
+  (description
+"Package check is a rich testing extension for Go's testing package.")
+  (license license:bsd-2))
+   (call-with-temporary-directory
+ (lambda (checkout)
+   (mock ((web client)
+  http-get
+  (mock-http-get fixtures-go-check-test))
+ (mock ((guix http-client)
+http-fetch
+(mock-http-fetch fixtures-go-check-test))
+   (mock ((guix git)
+  update-cached-checkout
+  (lambda* (url #:key ref)
+(values
+  checkout
+  (nix-base32-string->bytevector
+
"0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5")
+  #f)))
+ (go-module->guix-package
+   "github.com/go-check/check")))
expected-value: (package (name "go-github-com-go-check-check") (version 
"0.0.0-20201130134442-10cb98267c6c") (source (origin (method git-fetch) (uri 
(git-reference (url "https://github.com/go-check/check;) (commit 
(go-version->git-ref version (file-name (git-file-name name version)) 
(sha256 (base32 "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5" 
(build-system go-build-system) (arguments (quote (#:import-path 
"github.com/go-check/check"))) (propagated-inputs (quasiquote 
(("go-github-com-kr-pretty" (unquote go-github-com-kr-pretty) (home-page 
"https://github.com/go-check/check;) (synopsis "Instructions") (description 
"Package check is a rich testing extension for Go's testing package.") (license 
license:bsd-2))
actual-value: #f
actual-error:
+ (wrong-number-of-args
+   #f
+   "Wrong number of arguments to ~A"
+   (#sxml-0nf (input)>)
+   #f)
result: FAIL
--8<---cut here---end--->8---

Seems like an issue in code related to importing go packages.  I didn't
check if it was reported already.

-- 
Chris


signature.asc
Description: PGP signature

Re: The purpose of the "license" list of a Guix package

[Chris Marusich]

Chris Marusich  writes:

> In the end, I think a packager is expected to be operating in good
> faith, in accordance with the FSDG.  This means that packagers look for
> freedom issues and address them when found.  It does not mean that
> packagers are expected to provide a detailed "bill of materials" for
> every single package, although that is certainly something that some
> people think is important (and some people don't).

Another way of saying this is that, in my understanding, the license
field is "just a hint".  It should be a useful hint, of course, but it
is still just a hint, rather than an exhaustive or authoritative
description of all licensing implications for all use cases.

-- 
Chris


signature.asc
Description: PGP signature

The purpose of the "license" list of a Guix package (Was: Re: Jam: which licence is this?)

[Chris Marusich]

Hi,

Leo Famulari  writes:

> On Sun, May 02, 2021 at 12:53:07AM -0400, Mark H Weaver wrote:
>> My understanding is that the 'license' field of a package in Guix has
>> _always_ been meant to summarize the license restrictions associated
>> with the package source (the output of "guix build --source"), and
>> *not* merely the package outputs.
>
> My understanding is that the license field describes the license that
> the package is redistributed under, which is typically a single license,
> but could be a dual (or multiple) license if that is the case.
>
> The manual section "package Reference" says:
>
> The license of the package; a value from (guix licenses), or a list of
> such values.
>
> It's the license of the package, overall. Not of every single file in
> the source code.
>
>> Really?  Can you give some examples of this from our core packages?
>
> The 'hello' package is not core, but it is a typical Autotools-based
> package, and the core packages will be similar.
>
> It is, overall, GPL3 or later. However, the component source files bear
> these other licenses too:
>
> aclocal.m4: An unnamed permissive license
> AUTHORS: An unnamed permissive license
> configure: An unnamed permissive license
> configure.ac: An unnamed permissive license
> INSTALL: An unnamed permissive license
> maint.mk: An unnamed permissive license
> Makefile.am, Makefile.in: An unnamed permissive license
> README, README-dev: An unnamed permissive license
> THANKS: An unnamed permissive license
>
> build-aux/compile: GPL2+
> build-aux/config.rpath: An unnamed permissive license
> build-aux/depcomp: GPL2+
> build-aux/install-sh: Expat license
> build-aux/mdate-sh: GPL2+
> build-aux/missing: GPL2+
> build-aux/test-driver: GPL2+
>
> doc: Some files bear the GFDL
>
> m4: Maybe unnamed permissive licenses (I'm getting tired)
>
> po/Makefile.in.in: The "GPL" with no version mentioned. I assume GPL1.
> po/POTFILES.in: An unnamed permissive license
>
> tests/*: An unnamed permissive license
>
> Some of those unnamed permissive licenses are the same as each other,
> some are different.
>
> It would be unhelpful if the package definition's license field said
> "gpl3+ gpl2+ gpl1 non-copyleft expat gfdl". Nobody would be able to
> answer the question "What is the license of the 'hello' package?"
>
>> The 'license' field can only mean one of these two things, and I think
>> it's fairly clear which one it should be.  Moreover, I think that this
>> is what it has always meant in Guix.  If not, that's a problem.
>
> My survey of the "hello" package shows that the license field in Guix is
> about the overall license of the program, not an exhaustive list of the
> many licenses used for various components of the source code.
>
> I don't think it's a problem to not mention those licenses in the
> package definition. When the user acquires the source code, they still
> get the benefits of the freely licensed components. Nothing is being
> hidden.
>
> The suggestion that our package definitions' license field should
> mention every license contain in the source code has no precedent in
> Guix, at least since I joined. I don't there is a demonstrated benefit
> to making that change.

I agree with Leo.  My understanding is that the intent of the "license"
field (which can be a list) in a Guix package is to call out the "main"
(deliberately vague here) licenses related to the code, not to provide
an exhaustive or authoritative description of all the licenses affecting
every file in every possible situation.  As Leo has demonstrated, a
package's license field (like probably most summaries of license
information) is surely not exhaustive or authoritative; the licenses in
the source code files themselves are authoritative.

My understanding is that a packager is expected to audit the licenses in
the files when adding the package.  If an exhaustive audit is not
feasible (which is often the case), they should perform a reasonable
spot check and then list the "main" licenses in play in the licenses
file.  As in the GNU Hello case, there is clearly a judgment call
regarding what goes into the Guix package's licenses list, since a
simple list cannot describe everything.  In general, even if
hypothetically you did do an exhaustive audit of all files, it is not
practical to try to describe all the licensing implications in the Guix
package definition.  I don't think that's the purpose of the license
field.

One more thing.  I have always felt that the license field of a Guix
package is intended to call out the licenses that apply to the built
output of the package in particular.  In other words, I think the
license field is intended to call out the licenses that are most likely
to apply in the situation where you "link" with the built output of the
package.  That is the purpose of many packages: to be "linked" from
other source code.  Since it is often the case that software you write
will not be "linking" with the package's build scripts, but rather with

Re: bug#47615: [PATCH 1/9] gnu: bootstrap: Add support for powerpc-linux.

[Chris Marusich]

Chris Marusich  writes:

> Efraim Flashner  writes:
>
>> On 923bb70a1bff657125c3008f119a477e5cb57c2b
>>gnu:glibc-for-bootstrap: Fix patch.
>>
>> Run
>> ./pre-inst-env guix build --target=powerpc-linux-gnu bootstrap-tarballs
g>>
>> Producing
>>
>> /gnu/store/dyj1wvayyp1ihaknkxniz1xamcf4yrhl-bootstrap-tarballs-0
>>
>> With guix hash -rx 
>> /gnu/store/dyj1wvayyp1ihaknkxniz1xamcf4yrhl-bootstrap-tarballs-0
>>
>> 02xx2ydj28pwv3vflqffinpq1icj09gzi9icm8j4bwc4lca9irxn
>
> Generally speaking, this patch looks fine to me.  Just curious, what
> sort of machines does one use for 32-bit powerpc?
>
> I want to build the bootstrap binaries, see if they're reproducible (in
> particular GCC, which I suspect won't be), and verify the hashes.
>
> It might take a few days to do that, but I'll update this thread once
> I've done it.

I repeated Efraim's steps on two different x86_64-linux Guix System
machines.  In both cases, it produced exactly the same hash.  Therefore,
it would seem these bootstrap binaries are actually reproducible.  I was
surprised by this because of my experience with bug 41669.  I expected
GCC to not be reproducible, but in this case it seems reproducible.

I wonder what's different?  The powerpc64 architecture is 64-bit, and
powerpc is 32-bit, but I wonder what else might be different that could
cause the non-reproducibility to occur only in the powerpc64-linux
case.

Anyway, this is good news for the powerpc-linux port.  It is also an
interesting clue for the investigation of bug 41669, but further
discussion about that should go there, not here.

-- 
Chris


signature.asc
Description: PGP signature

Re: GNU Guix 1.3.0rc1 available for testing!

[Chris Marusich]

Maxim Cournoyer  writes:

> A first RC for the upcoming 1.3.0 release is now available for testing

I tested the binary for powerpc64le-linux on a Debian 10 ppc64el system.
It seems to be behaving as well as I expected it to.  Thank you for
preparing it!

On powerpc64le-linux, the installation script worked fine (I ran it
while logged into the root account via "su --login").  However, when I
ran "guix pull" (without substitutes, of course, since those are not yet
available for this platform), I encountered an error:

  building 
/gnu/store/6ljrcy6pd2g5c02gkfdj1zxca3ybpg18-texlive-latex-amscls-51265.drv...
  - 'build' phaseBacktrace:
13 (primitive-load 
"/gnu/store/ylynparc7pjhyxkcs9p1kwqa09an3vba-compute-guix-derivation")
  In ice-9/eval.scm:
  155:9 12 (_ _)
  159:9 11 (_ #(#(#(#(#(#(#(#(#(#(#(#(#(#(#(#(# ?) ?) 
?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?))
  In ice-9/boot-9.scm:
  152:2 10 (with-fluid* _ _ _)
  152:2  9 (with-fluid* _ _ _)
  In ./guix/store.scm:
2066:24  8 (run-with-store # _ 
#:guile-for-build _ #:system _ #:target _)
 1900:8  7 (_ _)
  In ./guix/gexp.scm:
 256:18  6 (_ _)
 1137:2  5 (_ _)
 1003:2  4 (_ _)
  849:4  3 (_ _)
  In ./guix/store.scm:
1948:12  2 (_ #)
 1362:5  1 (map/accumulate-builds # _ 
_)
1373:15  0 (_ # _ _)

  ./guix/store.scm:1373:15: ERROR:
1. :
message: "build of 
`/gnu/store/xfkrpagvnpc4xz43b6za80qrvnlsb8f0-po4a-0.61.drv' failed"
status: 100
  guix pull: error: You found a bug: the program 
'/gnu/store/ylynparc7pjhyxkcs9p1kwqa09an3vba-compute-guix-derivation'
  failed to compute the derivation for Guix (version: 
"fcd002ccfa3a2bf28a9d05ab2992464afc6e5fca"; system: "powerpc64le-linux";
  host version: "1.3.0rc1"; pull-version: 1).
  Please report it by email to .

When I tried building
/gnu/store/xfkrpagvnpc4xz43b6za80qrvnlsb8f0-po4a-0.61.drv a second time,
it succeeded:

  guix build --cores=1 \
 --max-jobs=1 \
 --keep-failed \
 /gnu/store/xfkrpagvnpc4xz43b6za80qrvnlsb8f0-po4a-0.61.drv

I was then able to successfully run "guix pull" a second time.  After
that, I successfully built GNU Hello via "guix build hello" and ran it.
So, everything seems OK, except for the non-deterministic po4a failure.

I wonder if the po4a failure affects other architectures, but maybe it
just isn't noticed because most people use substitutes.  As far as I can
tell, the po4a failure has not been reported in our bug tracker.  If I
can reproduce the issue, I'll open a bug report for it.

-- 
Chris


signature.asc
Description: PGP signature

Re: Leaving the GNU Guix community

[Chris Marusich]

Hi Léo,

I'm sorry to hear that you feel that you need to leave the community.  I
can understand why you feel that way, but I hope you'll remember that
(to my knowledge) nobody has said that they don't want you here.

I realize that in this moment, it must sting terribly to have your
commit rights temporarily revoked.  However, even before you gained
commit access, you contributed great things to Guix: patches, ideas,
computing resources, your energy and enthusiasm, and so on.  I
personally know that we would not have ported Guix successfully to
POWER9 without your help!  I guess what I'm trying to say is that even
without commit access, it's possible to be a contributor and a community
member.  And in any case, the maintainers have made it clear that the
commit access suspension does not need to be permanent:

https://lists.gnu.org/archive/html/guix-devel/2021-04/msg00489.html

They said:

"I'm sorry to say your commit privileges have been temporarily
suspended.  After one month, you are invited to get in touch with the
maintainers collective and discuss next steps."

They have explicitly left the door open.

By the way, speaking of contributions, I think your latest email here
raises some very good points.  I don't have good solutions to suggest
for any of them, I'm afraid, but I can say that I share many of your
concerns.  The "user experience" can definitely be better in many ways.
I also agree that the line between "user" and "developer" can and should
be blurred (programs like Emacs or LibreOffice or RenPy come to mind).
The more the line can be blurred successfully, the more it can empower
the user/developer, but it is tricky to do.  I mean, how many
non-programmers do you know who really use Emacs (or Vim)?  Personally,
I know none.  I know a lot of people who use Excel, though, or programs
like RenPy which are easy to "script" - i.e., program.  There are ways.

I also think it is more difficult than necessary to get started
contributing to Guix.  I wish it were easier, but I don't have any good
suggestions for that right now, either.  I think a lot of people who
have played with Guix code and participated in the Guix community also
experience these same difficulties, but I guess maybe people don't talk
about it too much because it's truly hard to come up with better
solutions than what we have.  There is also the issue of lack of time;
for those contributors (like myself, currently) who work on the project
only as volunteers in their spare time, it can sometimes be difficult to
find the time.  People tend to contribute in areas they are interested
in, which is fair, and there is always far more work to do than any
single person can finish on their own.

I wish you the very best in whatever you do, and I hope you'll return to
the Guix community when you feel that we can collaborate together again.

-- 
Chris


signature.asc
Description: PGP signature

Re: bug#47615: [PATCH 3/9] gnu: binutils: Adjust test suite on powerpc-linux.

[Chris Marusich]

Efraim Flashner  writes:

>> If it's a test failure, has the issue been reported upstream?
>
> I'll check to see if I can find something upstream. If not I'll report
> it. FWIW Debian disables the test suite for powerpc.
> https://sources.debian.org/src/binutils/2.36.1-6/debian/rules/#L537

How significant is the failure, anyway?  Are we likely to encounter
trouble down the line on powerpc if we don't resolve this?  I have no
idea, honestly, so I'm asking because I just don't know.

I looked at the rules file you linked.  Are you sure it disables the
test for powerpc?  It reads:

  with_check := yes
  ifneq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))
# override buildd admins to run the testsuite anyway ...
ifeq (,$(filter $(DEB_HOST_ARCH), m68k powerpc sh4 sparc64))
  with_check := disabled through DEB_BUILD_OPTIONS
endif
  endif
  #with_check := disabled for this upload

I'm not sure what the values for DEB_HOST_ARCH or DEB_BUILD_OPTIONS
would be here.  However, it seems to be setting with_check to disabled
if and only if (1) nocheck shows up in DEB_BUILD_OPTIONS and (2)
DEB_HOST_ARCH is not one of m68k, powerpc, sh4, or sparc64.  In other
words, it seems to enable the tests unless nocheck is in
DEB_BUILD_OPTIONS, in which case it STILL enables the tests provided
that DEB_HOST_ARCH is one of m68k, powerpc, sh4, or sparc64.

I'm not very familiar with Debian rules files, so perhaps I'm mistaken.
However, I think this means that when DEB_BUILD_OPTIONS doesn't contain
"nocheck" (presumably it doesn't usually?), the tests will be run for
every platform.

In any case, reporting this upstream seems like the right thing to do.

> I like the way you've done it (and the comment). The fewer places with
> copy/pasted code the better. I'm building it again now to see about the
> tests. With your changes I only need to add the phase in (gnu packages
> base).

I think the comment is honestly more than necessary, but if you find it
helpful, perhaps someone in the future might, too.  It describes a
possibly surprising behavior of substitute-keyword-arguments that can
happen any time you invoke it.  I just wanted to call it out
specifically for the purpose of this patch review.

-- 
Chris


signature.asc
Description: PGP signature

Re: bug#47615: [PATCH 3/9] gnu: binutils: Adjust test suite on powerpc-linux.

[Chris Marusich]

Chris Marusich  writes:

> (define binutils-boot0
>   (package
> (inherit binutils)
> (source (bootstrap-origin (package-source binutils)))
> (name "binutils-cross-boot0")
> (arguments
>  `(#:guile ,%bootstrap-guile
>#:implicit-inputs? #f
>
>#:modules ((guix build gnu-build-system)
>   (guix build utils)
>   (ice-9 ftw)); for 'scandir'
>
>,@(substitute-keyword-arguments (package-arguments binutils)
>((#:configure-flags cf)
> `(cons ,(string-append "--target=" (boot-triplet))
>,cf))
>;; The presence of '%standard-phases as the default value here is
>;; important.  It ensures that even when (package-argument
>;; binutils) does not already contain the #:phases keyword
>;; argument, the substitution will occur.  If you omit a default
>;; value and (package-arguments binutils) does not contain the
>;; #:phases keyword argument (e.g., on an x86_64-linux system),
>;; then the substitution will not occur, and no phases at all will
>;; be added.
>((#:phases phases '%standard-phases)
> `(modify-phases ,phases
>,@(if (string=? (%current-system) "powerpc-linux")
>  '((add-after 'unpack 'disable-rust-libiberty-test
>  (lambda _
>(substitute* "libiberty/testsuite/Makefile.in"
>  ((" check-rust-demangle ") ""))
>#t)))
>  '())
>(add-after 'install 'add-symlinks
>  (lambda* (#:key outputs #:allow-other-keys)
>;; The cross-gcc invokes 'as', 'ld', etc, without the
>;; triplet prefix, so add symlinks.
>(let ((out (assoc-ref outputs "out"))
>  (triplet-prefix (string-append ,(boot-triplet) "-")))
>  (define (has-triplet-prefix? name)
>(string-prefix? triplet-prefix name))
>  (define (remove-triplet-prefix name)
>(substring name (string-length triplet-prefix)))
>  (with-directory-excursion (string-append out "/bin")
>(for-each
> (lambda (name)
>   (symlink name (remove-triplet-prefix name)))
> (scandir "." has-triplet-prefix?)))
>
> (inputs (%boot0-inputs

Sorry, I meant to write this instead:

(define binutils-boot0
  (package
(inherit binutils)
(source (bootstrap-origin (package-source binutils)))
(name "binutils-cross-boot0")
(arguments
 `(#:guile ,%bootstrap-guile
   #:implicit-inputs? #f

   #:modules ((guix build gnu-build-system)
  (guix build utils)
  (ice-9 ftw)); for 'scandir'

   ,@(substitute-keyword-arguments (package-arguments binutils)
   ((#:configure-flags cf)
`(cons ,(string-append "--target=" (boot-triplet))
   ,cf))
   ;; The presence of '%standard-phases as the default value here is
   ;; important.  It ensures that even when (package-argument
   ;; binutils) does not already contain the #:phases keyword
   ;; argument, the substitution will occur.  If you omit a default
   ;; value and (package-arguments binutils) does not contain the
   ;; #:phases keyword argument (e.g., on an x86_64-linux system),
   ;; then the substitution will not occur, and no phases at all will
   ;; be added.
   ((#:phases phases '%standard-phases)
`(modify-phases ,phases
   (add-after 'install 'add-symlinks
 (lambda* (#:key outputs #:allow-other-keys)
   ;; The cross-gcc invokes 'as', 'ld', etc, without the
   ;; triplet prefix, so add symlinks.
   (let ((out (assoc-ref outputs "out"))
 (triplet-prefix (string-append ,(boot-triplet) "-")))
 (define (has-triplet-prefix? name)
   (string-prefix? triplet-prefix name))
 (define (remove-triplet-prefix name)
   (substring name (string-length triplet-prefix)))
 (with-directory-excursion (string-append out "/bin")
   (for-each
(lambda (name)
  (symlink name (remove-triplet-prefix name)))
(scandir "." has-triplet-prefix?)))

(inputs (%boot0-inputs

The point is that we can probably "inherit" the phases, so we wouldn't
have to repeat ourselves.

-- 
Chris


signature.asc
Description: PGP signature

Re: bug#47615: [PATCH 0/9] Add 32-bit powerpc support

[Chris Marusich]

Efraim Flashner  writes:

> * gnu/packages/base.scm (binutils)[arguments]: Add phase on
> powerpc-linux to adjust the test suite.
> * gnu/packages/commencement.scm (binutils-boot0)[arguments]: Move custom
> phases after inherited arguments. Add phase on powerpc-linux to adjust
> the test suite.

Nits: adjust the test suite to do what?  The message would be clearer if
it explained the purpose of the adjustment.  You could also name the
phases you added/moved, for extra clarity, if you want to.

> diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
> index dbb7c619fe..b9fc0a6e29 100644
> --- a/gnu/packages/base.scm
> +++ b/gnu/packages/base.scm
> @@ -531,7 +531,16 @@ change.  GNU make offers many powerful extensions over 
> the standard utility.")
>  
>;; Make sure 'ar' and 'ranlib' produce archives in 
> a
>;; deterministic fashion.
> -  "--enable-deterministic-archives")))
> +  "--enable-deterministic-archives")
> +  ,@(if (string=? (%current-system) "powerpc-linux")
> +  `(#:phases
> +(modify-phases %standard-phases
> +  (add-after 'unpack 'disable-rust-libiberty-test
> +(lambda _
> +  (substitute* "libiberty/testsuite/Makefile.in"
> +((" check-rust-demangle ") ""))
> +  #t
> +  '(

What's the problem?  Presumably the test fails; a comment here could
clarify that.

If it's a test failure, has the issue been reported upstream?

> (synopsis "Binary utilities: bfd gas gprof ld")
> (description
> diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
> index 7c39a84008..f707a01d30 100644
> --- a/gnu/packages/commencement.scm
> +++ b/gnu/packages/commencement.scm
> @@ -2653,7 +2653,22 @@ exec " gcc "/bin/" program
> #:modules ((guix build gnu-build-system)
>(guix build utils)
>(ice-9 ftw)); for 'scandir'
> +
> +   ;; #:phases gets modified for powerpc-linux in binutils,
> +   ;; so #:phases here needs to be after the inherited one.
> +   ,@(substitute-keyword-arguments (package-arguments binutils)
> +   ((#:configure-flags cf)
> +`(cons ,(string-append "--target=" (boot-triplet))
> +   ,cf)))
> +
> #:phases (modify-phases %standard-phases
> +  ,@(if (string=? (%current-system) "powerpc-linux")
> +  '((add-after 'unpack 'disable-rust-libiberty-test
> +  (lambda _
> +(substitute* "libiberty/testsuite/Makefile.in"
> +  ((" check-rust-demangle ") ""))
> +#t)))
> +  '())
>(add-after 'install 'add-symlinks
>  (lambda* (#:key outputs #:allow-other-keys)
>;; The cross-gcc invokes 'as', 'ld', etc, without the
> @@ -2667,12 +2682,8 @@ exec " gcc "/bin/" program
>  (with-directory-excursion (string-append out "/bin")
>(for-each (lambda (name)
>(symlink name (remove-triplet-prefix 
> name)))
> -(scandir "." has-triplet-prefix?)))
> +(scandir "." has-triplet-prefix?)
>  
> -   ,@(substitute-keyword-arguments (package-arguments binutils)
> -   ((#:configure-flags cf)
> -`(cons ,(string-append "--target=" (boot-triplet))
> -   ,cf)
>  (inputs (%boot0-inputs
>  
>  (define libstdc++-boot0

I think you can put all of this in the substitute-keyword-arguments
form, which would (1) eliminate the need for a comment, (2) eliminate
the need to worry about the order of the keyword arguments, and (3)
eliminate the need to duplicate the new phase in two package
definitions:

(define binutils-boot0
  (package
(inherit binutils)
(source (bootstrap-origin (package-source binutils)))
(name "binutils-cross-boot0")
(arguments
 `(#:guile ,%bootstrap-guile
   #:implicit-inputs? #f

   #:modules ((guix build gnu-build-system)
  (guix build utils)
  (ice-9 ftw)); for 'scandir'

   ,@(substitute-keyword-arguments (package-arguments binutils)
   ((#:configure-flags cf)
`(cons ,(string-append "--target=" (boot-triplet))
   ,cf))
   ;; The presence of '%standard-phases as the default value here is
   ;; important.  It ensures that even when (package-argument
   ;; binutils) does not already contain the #:phases keyword
   ;; argument, the substitution will occur.  If you omit a default
   ;; value and (package-arguments binutils) does not contain the
   ;; 

Re: bug#47615: [PATCH 2/9] gnu: guile-3.0: Fix building on powerpc-linux.

[Chris Marusich]

Efraim Flashner  writes:

> * gnu/packages/guile.scm (guile-3.0)[arguments]: On powerpc add two
> phases to adjust for 32-bit big-endian systems.
> ---
>  gnu/packages/guile.scm | 21 -
>  1 file changed, 20 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
> index f63322794d..dca1b1c16f 100644
> --- a/gnu/packages/guile.scm
> +++ b/gnu/packages/guile.scm
> @@ -305,7 +305,26 @@ without requiring the source code to be rewritten.")
>   (substitute-keyword-arguments (package-arguments guile-2.2)
> ((#:configure-flags flags ''())
>  `(cons "--disable-jit" ,flags)))
> - (package-arguments guile-2.2)))
> + (if (string-prefix? "powerpc-" (%current-system))
> +   (substitute-keyword-arguments (package-arguments guile-2.2)
> + ((#:phases phases)
> +  `(modify-phases ,phases
> + (add-after 'unpack 'adjust-bootstrap-flags
> +   (lambda _
> + ;; Upstream not yet notified about suggested solution.
> + ;; See existing bug reports:
> + ;; https://debbugs.gnu.org/cgi/bugreport.cgi?bug=45214
> + ;; 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=977223
> + (substitute* "bootstrap/Makefile.in"
> +   (("^GUILE_OPTIMIZATIONS.*")
> +"GUILE_OPTIMIZATIONS = -O1 -Oresolve-primitives 
> -Ocps\n"))
> + #t))
> + (add-after 'unpack 'remove-failing-tests
> +   (lambda _
> + ;; TODO: Discover why this test fails on powerpc-linux
> + (delete-file "test-suite/standalone/test-out-of-memory")
> + #t)
> +   (package-arguments guile-2.2
>  (native-search-paths
>   (list (search-path-specification
>  (variable "GUILE_LOAD_PATH")

Generally this looks reasonable.  I understand 3 weeks is a long
iteration time!  I think it's OK to proceed if it works for
bootstrapping other software on this platform.  Especially since the
change is isolated to just powerpc-linux.

Has the Guile test failure been reported upstream?

-- 
Chris


signature.asc
Description: PGP signature

Re: bug#47615: [PATCH 1/9] gnu: bootstrap: Add support for powerpc-linux.

[Chris Marusich]

Efraim Flashner  writes:

> On 923bb70a1bff657125c3008f119a477e5cb57c2b
>gnu:glibc-for-bootstrap: Fix patch.
>
> Run
> ./pre-inst-env guix build --target=powerpc-linux-gnu bootstrap-tarballs
>
> Producing
>
> /gnu/store/dyj1wvayyp1ihaknkxniz1xamcf4yrhl-bootstrap-tarballs-0
>
> With guix hash -rx 
> /gnu/store/dyj1wvayyp1ihaknkxniz1xamcf4yrhl-bootstrap-tarballs-0
>
> 02xx2ydj28pwv3vflqffinpq1icj09gzi9icm8j4bwc4lca9irxn

Generally speaking, this patch looks fine to me.  Just curious, what
sort of machines does one use for 32-bit powerpc?

I want to build the bootstrap binaries, see if they're reproducible (in
particular GCC, which I suspect won't be), and verify the hashes.

It might take a few days to do that, but I'll update this thread once
I've done it.

> @@ -139,6 +148,7 @@
>;; This is where the bootstrap executables come from.
>
> '("https://git.savannah.gnu.org/cgit/guix.git/plain/gnu/packages/bootstrap/;
>  "https://alpha.gnu.org/gnu/guix/bootstrap/;
> +"http://flashner.co.il/guix/bootstrap/;
>  "http://lilypond.org/janneke/guix/;))

Once you're reasonably sure the bootstrap binaries won't change, we
should consider uploading them to alpha.gnu.org.  Ludo did it for me for
powerpc64le-linux but I don't know who has access (I don't).

>  (define %hurd-systems
>;; The GNU/Hurd systems for which support is being developed.
> @@ -361,7 +361,7 @@ name of its URI."
>;;
>;; XXX: MIPS is unavailable in CI:
>;; .
> -  (fold delete %supported-systems '("mips64el-linux")))
> +  (fold delete %supported-systems '("mips64el-linux" "powerpc-linux")))

Any ideas for how we can get a machine for powerpc CI?  Maybe VMs, I
guess?  Can a POWER9 machine be a powerpc-linux machine...?


-- 
Chris


signature.asc
Description: PGP signature

Re: bug#47615: [PATCH 0/9] Add 32-bit powerpc support

[Chris Marusich]

Efraim Flashner  writes:

> * gnu/packages/debug.scm (american-fuzzy-lop): Add case for
> powerpc-linux.
> (qemu-for-american-fuzzy-lop): Same.
> ---
>  gnu/packages/debug.scm | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/gnu/packages/debug.scm b/gnu/packages/debug.scm
> index 2913c348f3..1326ce6e16 100644
> --- a/gnu/packages/debug.scm
> +++ b/gnu/packages/debug.scm
> @@ -179,6 +179,7 @@ tools that process C/C++ code.")
> ("aarch64-linux"  "aarch64")
> ("armhf-linux""arm")
> ("mips64el-linux" "mips64el")
> +   ("powerpc-linux"  "ppc")
> ;; Prevent errors when querying this package on 
> unsupported
> ;; platforms, e.g. when running "guix package --search="
> (_"UNSUPPORTED"
> @@ -254,6 +255,7 @@ down the road.")
> ("aarch64-linux"  "aarch64")
> ("armhf-linux""arm")
> ("mips64el-linux" "mips64el")
> +   ("powerpc-linux"  "ppc")
> ;; Prevent errors when querying this package on 
> unsupported
> ;; platforms, e.g. when running "guix package --search="
> (_"UNSUPPORTED"

LGTM, I see it's in master already like you said.

-- 
Chris


signature.asc
Description: PGP signature

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

Hi,

Chris Marusich  writes:

> This is the final draft, I think.  I intend to commit it to the "posts"
> directory in guix-artwork on Monday morning, USA time, at which point I
> believe it will automatically show up on the blog.

I have published it in commit 0129dd529347bfefee96644ac9fbabc29adbe772.
Thank you again to everyone for your help!

-- 
Chris


signature.asc
Description: PGP signature

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

Hi,

This is the final draft, I think.  I intend to commit it to the "posts"
directory in guix-artwork on Monday morning, USA time, at which point I
believe it will automatically show up on the blog.

Thank you again for your help, everyone!  If you see any last-minute
typos, please do let me know.

-- 
Chris
From e4300631958b75d996b9b57c595e74539da5f938 Mon Sep 17 00:00:00 2001
From: Chris Marusich 
Date: Tue, 6 Apr 2021 00:10:35 -0700
Subject: [PATCH] website: drafts: Add powerpc64le-linux announcement.

* website/drafts/new-system-powerpc64le-linux.md: New file.
---
 .../drafts/new-system-powerpc64le-linux.md| 405 ++
 1 file changed, 405 insertions(+)
 create mode 100644 website/drafts/new-system-powerpc64le-linux.md

diff --git a/website/drafts/new-system-powerpc64le-linux.md b/website/drafts/new-system-powerpc64le-linux.md
new file mode 100644
index 000..18f3fc4
--- /dev/null
+++ b/website/drafts/new-system-powerpc64le-linux.md
@@ -0,0 +1,405 @@
+title: New Supported Platform: powerpc64le-linux
+date: 2021-04-12 00:00
+author: Chris Marusich and Léo Le Bouter
+tags: porting, powerpc64le, bootstrapping, cross-compilation, reproducibility
+---
+
+It is a pleasure to announce that support for powerpc64le-linux
+(PowerISA v.2.07 and later) has now been
+[merged](https://issues.guix.gnu.org/47182) to the master branch of
+GNU Guix!
+
+This means that GNU Guix can be used immediately on this platform
+[from a Git
+checkout](https://guix.gnu.org/manual/en/html_node/Building-from-Git.html).
+Starting with the next release (Guix v1.2.1), you will also be able to
+[download a copy of Guix pre-built for
+powerpc64le-linux](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html#Binary-Installation).
+Regardless of how you get it, you can run the new powerpc64le-linux
+port of GNU Guix on top of any existing powerpc64le GNU/Linux
+distribution.
+
+This new platform is available as a "technology preview".  This means
+that although it is supported,
+[substitutes](https://guix.gnu.org/manual/en/html_node/Substitutes.html)
+are not yet available from the build farm, and some packages may fail
+to build.  Although powerpc64le-linux support is nascent, the Guix
+community is actively working on improving it, and this is a great
+time to [get
+involved](https://guix.gnu.org/manual/en/html_node/Contributing.html)!
+
+### Why Is This Important?
+
+This is important because it means that GNU Guix now works on the
+[Talos II, Talos II Lite, and Blackbird
+mainboards](https://www.raptorcs.com/content/base/products.html) sold
+by [Raptor Computing Systems](https://www.raptorcs.com/).  This
+modern, performant hardware uses [IBM
+POWER9](https://wiki.raptorcs.com/wiki/POWER9) processors, and it is
+designed to respect your freedom.  The Talos II and Talos II Lite have
+[recently received Respects Your Freedom (RYF)
+certification](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom)
+from the FSF, and Raptor Computing Systems is currently pursuing RYF
+certification for the more affordable Blackbird, too.  All of this
+hardware [can run without any non-free
+code](https://wiki.raptorcs.com/wiki/Platform_Comparison), even the
+bootloader and firmware.  In other words, this is a freedom-friendly
+hardware platform that aligns well with GNU Guix's commitment to
+software freedom.
+
+How is this any different from existing RYF hardware, you might ask?
+One reason is performance.  The existing RYF
+[laptops](https://ryf.fsf.org/products?category=1=All_by=created_order=DESC),
+[mainboards](https://ryf.fsf.org/products?category=5=All_by=created_order=DESC),
+and
+[workstations](https://ryf.fsf.org/products?category=30=All_by=created_order=DESC)
+can only really be used with Intel Core Duo or AMD Opteron processors.
+Those processors were released over 15 years ago.  Since then,
+processor performance has increased drastically.  People should not
+have to choose between performance and freedom, but for many years
+that is exactly what we were forced to do.  However, the POWER9
+machines sold by Raptor Computing Systems have changed this: the free
+software community now has an RYF-certified option that [can compete
+with the performance of modern Intel and AMD
+systems](https://www.phoronix.com/scan.php?page=article=power9-threadripper-core9=1).
+
+Although the performance of POWER9 processors is competitive with
+modern Intel and AMD processors, the real advantage of the Talos II,
+Talos II Lite, and Blackbird is that they were designed from the start
+to respect your freedom.  Modern processors from [both Intel and AMD
+include back
+doors](https://www.fsf.org/blogs/sysadmin/the-management-engine-an-attack-on-computer-users-freedom)
+over which you are given no control.  Even though the back doors can
+be removed [with significant effort on older hardware in some
+cases](https://www.fsf.org/news/libreboot-x200-lapt

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

Hi Tobias,

Thank you very much for taking the time to review the blog post!

Tobias Platen  writes:

> On Fri, 09 Apr 2021 00:59:44 +0200
> Léo Le Bouter  wrote:
>
>> On Thu, 2021-04-08 at 09:37 -0700, Chris Marusich wrote:
>> > They also say in that Twitter thread: "We have been putting together
>> > our
>> > systems from blob-free components only (sans NIC as is known and
>> > being
>> > actively worked), and this is an area where no low-cost blob-free
>> > silicon is available right now."
>> > 
>> 
>> I've been using the Free Software firmware replacement for the NIC
>> since a while now and it's working great: 
>> https://github.com/meklort/bcm5719-fw/
>
> I've install that firmware on my Talos II and I can confirm that it works.
> I have reviewed 0001-website-drafts-Add-powerpc64le-linux-announcement.patch 
> and it looks good.
> It would be good to mention the Libre-SoC project(https://libre-soc.org/), 
> which might be a good target for the future.

I think that project is interesting, but I don't think I'll add a
section expliclty mentioning it in the post this time.  I originally did
discuss RISC-V in passing, but it felt out of place, since the focus of
the blog post is really on the POWER9 support.  The blog post is already
quite long, so I tried to cut out what I could.

I appreciate the suggestion, though.  I hope that somebody will try
porting to Libre-SoC as well, and RISC-V, too!

-- 
Chris


signature.asc
Description: PGP signature

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

Vincent Legoll  writes:

> Why only speaking of Talos and not about the 3rd option: the blackbird ?
> Maybe just concentrate on the vendor, more than on particular models...

I specifically avoided speaking about the Blackbird, only because it's
not yet RYF-certified.  However, perhaps I'm being too strict about it.

I actually own a Blackbird, myself.  I chose to buy it instead of the
Talos II or Talos II Lite because of its physically smaller form factor
and its lower cost.  I don't know why it isn't RYF-certified yet, but
according to this Phoronix article, they are "pursuing RYF
certification" for Blackbird, too:

https://www.phoronix.com/scan.php?page=news_item=FSF-RYF-Talos-II

Raptor Computing Systems claims that the Blackbird is "completely blob
free":

https://twitter.com/RaptorCompSys/status/1048373354695208960

They also say in that Twitter thread: "We have been putting together our
systems from blob-free components only (sans NIC as is known and being
actively worked), and this is an area where no low-cost blob-free
silicon is available right now."

However, the Talos II and Blackbird both use the same NIC, so I guess
that wouldn't stop it from meeting the RYF requirements:

https://wiki.raptorcs.com/wiki/Talos_II
Networking: 2x GbE (Broadcom BCM5719)

https://wiki.raptorcs.com/wiki/Blackbird
Networking: 3x GbE (Broadcom BCM5719)

See also:

https://wiki.raptorcs.com/wiki/BCM5719

"As the BCM5719 is the only on-board device on the non-SAS Talos™ II
variants to use proprietary firmware, Raptor Computing Systems has
started a contest to see who can create a truly libre replacement
firmware[1]. Anyone with the appropriate skill set is encouraged to take
up the challenge, and contributions to this page as the device is
analyzed in detail are welcomed.

While the BCM5719 does, at least for now, execute proprietary firmware
it is prevented from corrupting the operating system and/or other
protected memory regions via the system IOMMU[2]."

Thinking about this more, I think we should mention Blackbird in our
blog post as a more affordable option.  Let's explain that it doesn't
yet have RYF certification, but the platform is very similar to the
Talos II, and Raptor Computing Systems is currently pursuing RYF
certification for it, too.

-- 
Chris


signature.asc
Description: PGP signature

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

Hi,

Here's a new version of the blog post.  It incorporates all the feedback
so far.  I've also removed the "About other freedom-friendly platforms"
because it didn't seem to add much substance.

I significantly rewrote the "Why Is This Important?" section, mainly
because I realized that I was incorrectly and unfairly implying that
POWER9 CPUs are not vulnerable to Spectre/Meltdown-style
vulnerabilities.  In fact, some POWER9 CPUs were found to be vulnerable,
but the most recent models have been fixed.  I've rewritten this section
so that it focuses more on explaining why the RYF Talos II and Talos II
Lite are "more free" than the popular Intel and AMD mainstays (even the
older, RYF-certified models, where you still have to jump over the
hurdle of removing the Intel ME or equivalent.)

For details on Spectre/Meltdown on POWER9, see:

https://wiki.raptorcs.com/wiki/Speculative_Execution_Vulnerabilities_of_2018

I added a footer describing GNU Guix, as is customary on most of our
blog posts.

I changed the title.

I also fixed various links and rephrased a few things.

Anyway, if you can cast your eye over it once more, I would appreciate
it.  I think it's just about done!

-- 
Chris
From 4d9133e51fc666f14074c1da18bb16af0d76066f Mon Sep 17 00:00:00 2001
From: Chris Marusich 
Date: Tue, 6 Apr 2021 00:10:35 -0700
Subject: [PATCH] website: drafts: Add powerpc64le-linux announcement.

* website/drafts/new-system-powerpc64le-linux.md: New file.
---
 .../drafts/new-system-powerpc64le-linux.md| 389 ++
 1 file changed, 389 insertions(+)
 create mode 100644 website/drafts/new-system-powerpc64le-linux.md

diff --git a/website/drafts/new-system-powerpc64le-linux.md b/website/drafts/new-system-powerpc64le-linux.md
new file mode 100644
index 000..d2104aa
--- /dev/null
+++ b/website/drafts/new-system-powerpc64le-linux.md
@@ -0,0 +1,389 @@
+title: New Supported Platform: powerpc64le-linux
+date: 2021-04-08 00:00
+author: Chris Marusich and Léo Le Bouter
+tags: porting, powerpc64le, bootstrapping, cross-compilation, reproducibility
+---
+
+It is a pleasure to announce that support for powerpc64le-linux
+(PowerISA v.2.07 and later) has now been
+[merged](https://issues.guix.gnu.org/47182) to the master branch of
+GNU Guix!
+
+This means that GNU Guix can be used immediately on this platform from
+a [from a Git
+checkout](https://guix.gnu.org/manual/en/html_node/Building-from-Git.html).
+Starting with the next release (Guix v1.2.1), you will also be able to
+[download a copy of Guix pre-built for
+powerpc64le-linux](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html#Binary-Installation).
+Regardless of how you get it, you can run the new powerpc64le-linux
+port of GNU Guix on top of any existing powerpc64le GNU/Linux
+distribution.
+
+This new platform is available as a "technology preview".  This means
+that although it is supported,
+[substitutes](https://guix.gnu.org/manual/en/html_node/Substitutes.html)
+are not yet available from the build farm, and some packages may fail
+to build.  Although powerpc64le-linux support is nascent, the Guix
+community is actively working on improving it, and this is a great
+time to [get
+involved](https://guix.gnu.org/manual/en/html_node/Contributing.html)!
+
+### Why Is This Important?
+
+This is important because it means that GNU Guix now works on the
+[Talos II and Talos II Lite
+mainboards](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom),
+which use [IBM POWER9
+processors](https://wiki.raptorcs.com/wiki/POWER9).  This is a modern,
+performant hardware platform that has recently received [Respects Your
+Freedom (RYF) certification](https://ryf.fsf.org/) from the FSF.  It
+can run without any non-free code, all the way down to its bootloader
+and firmware.  In other words, it's a freedom-friendly platform that
+aligns well with GNU Guix's commitment to software freedom.
+
+How is this any different from existing RYF hardware, you might ask?
+One reason is performance.  The existing RYF
+[laptops](https://ryf.fsf.org/products?category=1=All_by=created_order=DESC),
+[mainboards](https://ryf.fsf.org/products?category=5=All_by=created_order=DESC),
+and
+[workstations](https://ryf.fsf.org/products?category=30=All_by=created_order=DESC)
+can only really be used with Intel Core Duo or AMD Opteron processors.
+Those processors were released over 15 years ago.  Since then,
+processor performance has increased drastically.  People should not
+have to choose between performance and freedom, but for many years
+that is exactly what we were forced to do.  However, the Talos II and
+Talos II Lite have changed this: the free software community now has
+an RYF-certified option that [can compete with the performance of
+modern Intel and AMD
+systems](https://www.phoronix.com/scan.php?page=article=power9-threadripper-core9=1).
+
+Although the perform

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

Hi Léo,

Léo Le Bouter  writes:

> It's been mostly you here Chris, thank you so much for writing it, as
> others said, it is really beautifully written! Unfortunately I havent
> felt enough peace of mind to write like you did :-(

You've been busy!  It's totally understandable.  The encouragement from
you and others has been very useful and motivating for me, so thank you.

> I would've liked to write about the early days where I met some
> problems with the core-updates branch having to rebase several times
> and learning GNU Guix at the same time since my first ever project
> related to GNU Guix was porting before even trying to use it elsewhere.
> Having to learn the GNU commit message guidelines, then learning git-
> send-email and GNU Emacs (since that's where all dev tools are), all
> that to contribute to GNU Guix and get this port in. Aaaahh very long
> story!

I agree.  Those are interesting topics.  I tried to include some
discussion about it, but the post became too lengthy.  I just want it to
be about the new support, mainly, and why it's exciting.

I think that the following related topics would make good candidates for
future blog posts:

- An analysis of trust in Guix, with an eye towards bootstrapping.  If
  you use substitutes, what are you implicitly trusting?  If you build
  without substitutes, what are you implicitly trusting?  If you build
  Guix from source without using Guix, like you have to do when you
  first port Guix to a new platform, what are you trusting?  A
  comparison of similar paths of trust when using other software.  Make
  a script to find out if there are any forgotten "bootstrap roots"
  beyond the bootstrap binaries, like there apparently used to be for
  some self-hosted compilers (see:
  https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00814.html).
  Stuff like that.  I think it is not obvious.

- An analysis of the hurdles / friction involved in contributing.
  Preferably with suggestions for ways to remove the hurdles and reduce
  friction.  It is easy to complain or bikeshed, of course, but the
  point is not to do that.  The point is to discuss issues to try and
  make things better.

Thank you again for your help!  This is just the beginning - let's keep
hacking away at it and improving POWER9 support together!  Hopefully
others will see the benefits of the platform and join us along the way.

-- 
Chris


signature.asc
Description: PGP signature

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

Hi Joshua,

Joshua Branson  writes:

> Awesome!  Great work!  I read the below draft blog post like a Harry
> Potter novel!  It is superbly written.  And it makes a lot of sense!

Thank you for the kind words, and your feedback!

>> +### Why Is This Important?
>> +
>> +This is important because it means that GNU Guix now works on the [RYF
>> +Talos II and Talos II Lite
>> +mainboards](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom)
>> +and it's IBM POWER9 processor.  This is a modern, performant hardware
>
> I believe you should use "its".  it's is short for "it is".

Good catch!

>> +platform that respects your freedom.  It can run without any non-free
>> +code, all the way down to its bootloader and firmware.  It's a
>> +freedom-friendly platform that aligns well with GNU Guix's commitment
>> +to software freedom.
>> +
>> +How is this any different from existing RYF hardware, you might ask?
>> +The existing RYF
>> +[laptops](https://ryf.fsf.org/products?category=1=All_by=created_order=DESC),
>> +[mainboards](https://ryf.fsf.org/products?category=5=All_by=created_order=DESC),
>> +and
>> +[workstations](https://ryf.fsf.org/products?category=30=All_by=created_order=DESC)
>> +can only really be used with Intel Core Duo or AMD Opteron processors.
>> +Those processors were released over 15 years ago.  Since then,
>> +processor performance has increased drastically.  People should not
>> +have to choose between performance and freedom, but the fact is that
>> +for many years, that is exactly what we were forced to do.  However,
>> +the Talos II and Talos II Lite have changed this: the free software
>> +community now has an RYF-certified option that can compete with the
>> +performance of modern Intel and AMD systems.
>> +
>> +Although the performance of POWER9 processors is competitive with
>> +modern Intel and AMD processors, its real advantage is that it<
>
> Why is there "it<"  ?  Is that some markup I'm not familiar with?

Nope, it's a typo.  Good eyes!

>> +respects your freedom.  Modern processors from [both Intel and AMD
>> +include back
>> +doors](https://www.fsf.org/blogs/sysadmin/the-management-engine-an-attack-on-computer-users-freedom)
>> +over which you are given no control.  Additionally, hardware design
>> +defects in the processors of both vendors have been discovered, giving
>> +rise to critical security vulnerabilities like
>> +[Spectre](https://en.wikipedia.org/wiki/Spectre_(security_vulnerability))
>> +and
>> +[Meltdown](https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability)).
>> +In many cases, these vulnerabilities can only be fixed by installing
>> +[non-free CPU microcode](https://wiki.debian.org/Microcode) - unless,
>> +of course, [the vendor decides not to provide any fix at
>> +all](https://arstechnica.com/gadgets/2018/04/intel-drops-plans-to-develop-spectre-microcode-for-ancient-chips/)!
>> +
>> +Compared to that, the RYF Talos II and Talos II Lite are a breath of
>> +fresh air that the free software community really deserves.  Raptor
>> +Computing Systems' commitment to software freedom and owner control is
>> +an inspiring reminder that it **is** possible to ship a great product
>> +that respects the freedom of your customers.  And going forward, the
>> +future looks bright for the open, royalty-free Power ISA, [which is
>> +now a Linux Foundation
>> +project](https://www.linuxfoundation.org/press-release/2019/08/the-linux-foundation-announces-new-open-hardware-technologies-and-collaboration/)
>> +(see also: [the same announcement from The OpenPOWER
>> +Foundation](https://openpowerfoundation.org/the-next-step-in-the-openpower-foundation-journey/).
>> +
>> +
>> +In Guix, all software for a given system (e.g., powerpc64le-linux) is
>> +built starting from its bootstrap binaries.  It is intended that the
>> +bootstrap binaries are the only pieces of software in the entire
>> +package collection that Guix cannot build from source.  In practice,
>> +[additional bootstrap roots are
>> +possible](https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00814.html),
>> +but introducing them in Guix is highly discouraged, and our community
>> +[actively](https://guix.gnu.org/en/blog/2019/guix-reduces-bootstrap-seed-by-50/)
>> +[works](https://guix.gnu.org/en/blog/2020/guix-further-reduces-bootstrap-seed-to-25/)
>> +to [reduce](https://guix.gnu.org/en/blog/2018/bootstrapping-rust/) our
>> +overall bootstrap footprint.
>> +
>> +So first you need to build the the bootstrap binaries for your
>
> "the the" --> "the"

Fixed!

>> +platform.  In theory, you can do this in many ways.  For example, you
>> +might try to manually compile them on an existing system.  However,
>> +Guix has [package
>> +definitions](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/make-bootstrap.scm?id=5d8c2c00d60196c46a32b68c618ccbe2b3aa48f4)
>> +that you can use to build them - using Guix, of course!
>>
>> +In both the big-endian 

Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

Hi,

Léo and I have drafted the following blog post.  Could you take a few
minutes to read it and give us your thoughts?

It's a work in progress.  The primary goal is to announce the new
powerpc64le-linux support and explain why it matters (POWER9 is an
exciting, freedom-friendly platform!).  The secondary goal is to explain
some of the details about what we did, and invite people to get
involved.

Your feedback would be highly appreciated!

-- 
Chris
From 8900d918106f6a70b20df461c5f086b5275773cc Mon Sep 17 00:00:00 2001
From: Chris Marusich 
Date: Tue, 6 Apr 2021 00:10:35 -0700
Subject: [PATCH] website: drafts: Add powerpc64le-linux announcement.

* website/drafts/new-system-powerpc64le-linux.md: New file.
---
 .../drafts/new-system-powerpc64le-linux.md| 326 ++
 1 file changed, 326 insertions(+)
 create mode 100644 website/drafts/new-system-powerpc64le-linux.md

diff --git a/website/drafts/new-system-powerpc64le-linux.md b/website/drafts/new-system-powerpc64le-linux.md
new file mode 100644
index 000..e3de5ba
--- /dev/null
+++ b/website/drafts/new-system-powerpc64le-linux.md
@@ -0,0 +1,326 @@
+title: powerpc64le-linux support in GNU Guix
+date: 2021-03-26 00:00
+author: Chris Marusich and Léo Le Bouter
+tags: porting, powerpc64le
+---
+
+It is a pleasure to announce that support for powerpc64le-linux
+(PowerISA v.2.07 and later) has now been
+[merged](https://issues.guix.gnu.org/47182) to the master branch of
+GNU Guix!
+
+This means that GNU Guix can be used immediately on this platform from
+a [from a Git
+checkout](https://guix.gnu.org/manual/en/html_node/Building-from-Git.html).
+Starting with the next release (Guix v1.2.1), you will also be able to
+[download a copy of Guix pre-built for
+powerpc64le-linux](https://guix.gnu.org/manual/en/guix.html#Binary-Installation).
+Regardless of how you get it, you can run the new powerpc64le-linux
+port of GNU Guix on top of any existing powerpc64le GNU/Linux
+distribution.
+
+This new platform is available as a "technology preview".  This means
+that although it is supported, substitutes are not yet available from
+the build farm, and some packages may fail to build.  Although
+powerpc64le-linux support is nascent, the Guix community is actively
+working on improving it, and this is a great time to [get
+involved](https://guix.gnu.org/manual/en/html_node/Contributing.html)!
+
+### Why Is This Important?
+
+This is important because it means that GNU Guix now works on the [RYF
+Talos II and Talos II Lite
+mainboards](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom)
+and it's IBM POWER9 processor.  This is a modern, performant hardware
+platform that respects your freedom.  It can run without any non-free
+code, all the way down to its bootloader and firmware.  It's a
+freedom-friendly platform that aligns well with GNU Guix's commitment
+to software freedom.
+
+How is this any different from existing RYF hardware, you might ask?
+The existing RYF
+[laptops](https://ryf.fsf.org/products?category=1=All_by=created_order=DESC),
+[mainboards](https://ryf.fsf.org/products?category=5=All_by=created_order=DESC),
+and
+[workstations](https://ryf.fsf.org/products?category=30=All_by=created_order=DESC)
+can only really be used with Intel Core Duo or AMD Opteron processors.
+Those processors were released over 15 years ago.  Since then,
+processor performance has increased drastically.  People should not
+have to choose between performance and freedom, but the fact is that
+for many years, that is exactly what we were forced to do.  However,
+the Talos II and Talos II Lite have changed this: the free software
+community now has an RYF-certified option that can compete with the
+performance of modern Intel and AMD systems.
+
+Although the performance of POWER9 processors is competitive with
+modern Intel and AMD processors, its real advantage is that it<
+respects your freedom.  Modern processors from [both Intel and AMD
+include back
+doors](https://www.fsf.org/blogs/sysadmin/the-management-engine-an-attack-on-computer-users-freedom)
+over which you are given no control.  Additionally, hardware design
+defects in the processors of both vendors have been discovered, giving
+rise to critical security vulnerabilities like
+[Spectre](https://en.wikipedia.org/wiki/Spectre_(security_vulnerability))
+and
+[Meltdown](https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability)).
+In many cases, these vulnerabilities can only be fixed by installing
+[non-free CPU microcode](https://wiki.debian.org/Microcode) - unless,
+of course, [the vendor decides not to provide any fix at
+all](https://arstechnica.com/gadgets/2018/04/intel-drops-plans-to-develop-spectre-microcode-for-ancient-chips/)!
+
+Compared to that, the RYF Talos II and Talos II Lite are a breath of
+fresh air that the free software community really deserves.  Raptor
+Computing Systems' commitment to software free

Re: Security related tooling project

[Chris Marusich]

Christopher Baines  writes:

> Chris Marusich  writes:
>
>> Christopher Baines  writes:
>>
>>> In terms of looking at security from a project perspective, I'm thinking
>>> about these kinds of needs/questions:
>>>
>>>  - What security issues affect this revision of Guix? (latest or otherwise)
>>>
>>>  - How do Guix contributors find out about new security issues that
>>>affect Guix revisions they're interested in?
>>>
>>> From the user perspective, I want to look at things like:
>>>
>>>  - How do I find out what (if any) security issues affect the software
>>>I'm currently running (through Guix)?
>>>
>>>  - How can I get notified when a new security issue affects the software
>>>I'm currently running (through Guix)?
>>>
>>> Please let me know if you have any comments or questions!
>>
>> I think this is a great plan! The last two points in particular are
>> particularly useful, I think.
>>
>> Everyone needs security.  I think Guix is in a unique position where it
>> is so easy to modify packages that (in theory, at least) anyone who
>> cares can figure out how to submit a change to upgrade and fix security
>> vulnerabilities.
>>
>> People and companies are more likely to go out of their way to fix
>> packages they care about.  Therefore, making it easy to identify
>> vulnerabilities in specifically the packages they care about, and making
>> it easier to get involved in the community to fix them, are important
>> goals.
>
> Cool :) While it's not directly security related, I really want the
> subscriptions functionality I'm planning to work on to be done so that
> people can subscribe to things related to the packages they use, like
> new versions becoming available, or the build breaking for example, as
> that might help people stay involved.

Yes, that would be cool.  I can imagine various ways that a user could
get information like this.  For instance, just as how the news entries
tell you what's new when you "guix pull," perhaps we could add something
similar (optional?) for when you install packages, like: "Hey, I see
you're using packages Foo and Bar.  New versions are available!  They
are affected by these CVEs!  Run "guix refresh" from a checkout to try
upgrading them!" and so on.

Another option could be to add some sort of functionality to Cuirass
(does it already exist?) or the Guix Data Service which allows one to
create a custom RSS feed or atom feed or similar.  Imagine crafting a
URL that says "This is my search query - spit out an RSS feed showing me
what's going on recently with these packages".  I know some wiki-style
software features something like this, where you can encode a search
query into a URL, and it will spit out a dynamically created RSS feed.

Another idea is: just as "guix lint" can report CVEs, perhaps the code
could be adapted to enable a command that lets you lint an
already-installed profile to inform you about what CVEs or updates are
available for that specific collection of installed software.  This
could be used as a "security scanner", where you can "scan" some
installed profiles to see what's vulnerable on a system.  Simply keeping
the package definitions up to date is half the battle; actually
upgrading them on systems you care about is the other half...

Just some ideas.  Perhaps one resonates with you; if not, that's fine
too!  Maybe the UI is the easier part, and the mechanism of reliably
determining what has changed, what security updates are available, etc.,
is harder.

-- 
Chris


signature.asc
Description: PGP signature

Re: Security related tooling project

[Chris Marusich]

Christopher Baines  writes:

> In terms of looking at security from a project perspective, I'm thinking
> about these kinds of needs/questions:
>
>  - What security issues affect this revision of Guix? (latest or otherwise)
>
>  - How do Guix contributors find out about new security issues that
>affect Guix revisions they're interested in?
>
> From the user perspective, I want to look at things like:
>
>  - How do I find out what (if any) security issues affect the software
>I'm currently running (through Guix)?
>
>  - How can I get notified when a new security issue affects the software
>I'm currently running (through Guix)?
>
> Please let me know if you have any comments or questions!

I think this is a great plan! The last two points in particular are
particularly useful, I think.

Everyone needs security.  I think Guix is in a unique position where it
is so easy to modify packages that (in theory, at least) anyone who
cares can figure out how to submit a change to upgrade and fix security
vulnerabilities.

People and companies are more likely to go out of their way to fix
packages they care about.  Therefore, making it easy to identify
vulnerabilities in specifically the packages they care about, and making
it easier to get involved in the community to fix them, are important
goals.

-- 
Chris


signature.asc
Description: PGP signature

Re: I have merged wip-ppc64le-for-master to master

[Chris Marusich]

Ludovic Courtès  writes:

> Chris Marusich  skribis:
>
>> FYI, I have merged wip-ppc64le-for-master to master and closed bug
>> 47182.  I have also deleted the wip-ppc64le-for-master branch.
>
> Yay, congrats!  \o/
>
> I now have access to the OSUOSL POWER9 machine.  I’ve set up offloading
> and it seems to work well.
>
> Thanks to all the hackers involved + OSUOSL!

That's fantastic news!  Please let us know if you have any trouble
running the "make release" target.  I think the recipe is good to go,
since SUPPORTED_SYSTEMS now contains powerpc64le-linux.

Is there something else I can help with for the release?  Leo and I are
working on a blog post; we should probably share it soon for review.

By the way, I have just now updated the documentation in commit
e52ec6c64a17a99ae4bb6ff02309067499915b06 and the news.scm file in
0374617920e3d278e68c71826fec1f590921e31b.

I'm really excited about this release!  I'm looking forward to
continuing to work on POWER9 support with everyone going forward.

-- 
Chris


signature.asc
Description: PGP signature

I have merged wip-ppc64le-for-master to master

[Chris Marusich]

Hi,

FYI, I have merged wip-ppc64le-for-master to master and closed bug
47182.  I have also deleted the wip-ppc64le-for-master branch.

Later this week, I will update the manual to mention that
powerpc64le-linux is supported, and I will update the "release" target
of the Makefile so that we can make a release!

-- 
Chris


signature.asc
Description: PGP signature

Re: Let's include powerpc64le-linux in the next release, Re: Let's include powerpc64le-linux in the next release

[Chris Marusich]

Tobias Geerinckx-Rice  writes:

> Ludovic Courtès 写道：
>> Like Efraim wrote, the person who makes the release (I’m afraid
>> it’ll be
>> me) needs to be able to offload to a “trustworthy” machine for that
>> platform.
>
> Define trustworthy?  The project has access to an 8-core POWER9 VM
> from OSUOSL, currently running Debian, to which efraim, lle-bout, 
> vincele, and me have root access.

That would probably work.  My understanding is that because we cannot
install Guix from an existing release binary, somebody will need to
manually build Guix on that machine first, for offloading to work.

I can help with this.

Ludovic Courtès  writes:

> If we can do that, we should adjust the ‘release’ target in
> ‘Makefile.am’ to do that.
>
> However, since we won’t be providing substitutes, we should label it as
> “technology preview” in the manual (info "(guix) GNU Distribution").

OK!  I was planning to add some commits to update the docs, anyway.  I
can do those two tasks.

Additionally, I plan to finally merge wip-ppc64le-for-master to master
later today.  I'll send a note when that's done.

-- 
Chris


signature.asc
Description: PGP signature

Re: Release 1.2.1: status

[Chris Marusich]

Hi simon,

zimoun  writes:

> First, thanks to Chris and folks, powerpc64le-linux will be [1] in the
> next release, great! Isn’t it? :-)

I'm very excited about this!  Efraim was able to rework our patches so
they could be applied to master without rebuilding the entire world.
The patches are currently under review here:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47182

More review is welcome, but unless some unforeseen issues are
discovered, I agree that we can go ahead with adding support for
powerpc64le-linux.

Where is the release work happening?  Where can I merge or apply these
changes to ensure they get included in the next release?

-- 
Chris


signature.asc
Description: PGP signature

Re: I've rebased wip-ppc64le onto core-updates, Re: Release on April 18th?

[Chris Marusich]

Hi Ludo,

Ludovic Courtès  writes:

> Maybe you could “formally” ask for a review of the branch when you think
> it’s ready (perhaps even git-send-emailing the patches to guix-patches,
> for convenience), and then we can merge in ‘core-updates’ (if/when that
> matches other branch scheduling choices).

That's a good idea.  I'll do that for wip-ppc64le-for-master shortly.

I'll do it for wip-ppc64le later, after incorporating the changes from
wip-ppc64le-for-master, and after resolving the issues with the GCC 8
upgrade.

Ludovic Courtès  writes:

> It works on my laptop.  :-) If you’re confident, please push—thanks for
> the fix!

Done in commit 341dfe7eda4972af0a027357015ea595314438b0!

> (Next time please report the issue to bug-guix where it’s more likely to
> be seen and dealt with.  :-))

Good call.  It's easy to miss an email in a big thread like this.  Thank
you for noticing it and replying.  In the future, I'll send bugs/patches
to bug-guix/guix-patches to increase their visibility.

-- 
Chris


signature.asc
Description: PGP signature

Let's include powerpc64le-linux in the next release (was: Re: Release on April 18th?)

[Chris Marusich]

Hi,

I have good news!

Chris Marusich  writes:

> Subject: [PATCH] syscalls: mount: Fix a matching bug.

If there are no concerns about this patch, I will commit it to master
(with the "mount" typo fixed so it reads "mounts") in the next few days.

> Efraim Flashner  writes:
>
>> I'm now running make guix-binary.powerpc64le-linux.tar.xz and so far
>> it's made it past the initial building stages, we're on to building the
>> grafts now.
>
> Thank you.  This fixed the patch-related problem for me, too.  I'm
> currently running "make guix-binary.powerpc64le-linux.tar.xz", also, on
> my Debian ppc64le machine.  We'll see how far it gets - fingers crossed!

I'm happy to report that I was able to do all of the following things
successfully on the wip-ppc64le-for-master branch after (1) applying the
syscalls patch mentioned above, (2) running "make update-guix-package",
and (3) locally committing the updated guix package:

- On a bare metal Debian ppc64le machine, I ran "make
  guix-binary.powerpc64le-linux.tar.xz" successfully.

- I installed the resulting guix binary in a fresh Debian ppc64le VM
  successfully.

- In the VM, using the newly installed guix binary, I successfully ran
  "guix pull" to update Guix to the commit I made in (3) above.

- In the VM, using the newly pulled guix, I built GNU Hello and verified
  that it ran successfully.

This demonstrates that wip-ppc64le-for-master is working.  Therefore, we
should merge it to master and officially include powerpc64le-linux
support in the next Guix release!  Thank you, Efraim, for taking the
initiative to adapt some of the commits from wip-ppc64le so they could
be applied to master without rebuilding the world on other systems.

I've verified that the wip-ppc64le-for-master branch does not rebuild
the world.  I did this by verifying that the derivations for the "hello"
and "gcc-toolchain" packages were the same at the branch point as they
are at the tip of the branch (e.g.: ./pre-inst-env guix build -d hello).

As I see it, the next tasks for powerpc64le-linux in this release are:

- Do a final rebase of wip-ppc64le-for-master, then merge it to master.

- When the release happens, make a guix-binary.powerpc64le-linux.tar.xz
  available wherever binary releases are normally published.

- Start building powerpc64le-linux substitutes in the build farm,
  ideally on POWER9 hardware.

How shall we build the binary tarball for the release?  Of course,
anybody with a copy of the (source) release tarball can build their own
guix binary by invoking "make guix-binary.powerpc64le-linux.tar.xz"
themselves.  However, for convenience, it would be nice to provide a
pre-built binary if possible.  Shall I build this myself when the time
comes, or would people prefer to do it a different way?

-- 
Chris


signature.asc
Description: PGP signature

Re: Release on April 18th?

[Chris Marusich]

Chris Marusich  writes:

> Subject: [PATCH] syscalls: mount: Fix a matching bug.

In the patch message, "mount" should be "mounts".  Sorry for the typo!

-- 
Chris


signature.asc
Description: PGP signature

Re: Release on April 18th?

[Chris Marusich]

Hi,

Vincent Legoll  writes:

> I rebuilt guix on core-updates with gcc-8 succesfully
> I'll now try the same above wip-ppc64le.

Awesome!  Thank you for doing this.  I'm sure there will be some bumps,
and the sooner we can fix them, the easier it will be to integrate
later.

I'm still working on getting you access to ppc64le hardware or a VM.

Andreas Enge  writes:

> Am Fri, Mar 12, 2021 at 12:33:18AM -0800 schrieb Chris Marusich:
>> The proc man page has this to say about column 7:
>>   (7) optional fields: zero or more fields of the form "tag[:value]"
>
> And it goes on like this:
> (8)  separator: the end of the optional fields is marked
>by a single hyphen.
>
> So it looks like it is enough to search for a hyphen surrounded by spaces.
> The hyphen is apparently there also when there is no optional field (7),
> as can be seen from your examples and also my own system, where both occur
> in parallel:
> 34 26 253:0 /gnu/store /gnu/store ro,noatime - ext4 /dev/mapper/cryptroot rw
> 51 26 253:0 /var/lib/docker /var/lib/docker rw,relatime shared:1 - ext4 
> /dev/mapper/cryptroot rw
>
> Alternatively, one can also count from the back to get the fields labelled
> (9) to (11) (which may be (8) to (10) in case there are no optional fields).
> (I was momentarily confused by "(12) super option*s*"; but these are
> apparently separated by commas and not whitespace.)

That's true.  How about the following patch?  It fixes the issue for me
on my systems, and I manually tried out the new match pattern with some
lines from Leo's output, and it seems to behave correctly.  If nobody
has concerns, I'd like to apply it to master directly, since the tests
are already failing there on some systems, like mine.  Here's the patch:

From 5417f68ee5892f6a587895105854cadf29eb7297 Mon Sep 17 00:00:00 2001
From: Chris Marusich 
Date: Thu, 11 Mar 2021 23:19:30 -0800
Subject: [PATCH] syscalls: mount: Fix a matching bug.

On some systems, the columns in /proc/self/mountinfo look like this:

23 28 0:21 / /proc rw,nosuid,nodev,noexec,relatime shared:11 - proc proc rw

Before this change, the mount procedure was written with the assumption that
the type and source could always be found in columns 8 and 9, respectively.
However, the proc(5) man page explains that there can be zero or more optional
fields starting at column 7 (e.g., "shared:11" above), so this assumption is
false in some situations.

* guix/build/syscalls.scm (mount): Update the match pattern to use ellipsis to
match zero or more optional fields followed by a single hyphen.  Remove the
trailing ellipsis, since multiple ellipses are not allowed in the same level.
The proc(5) man page indicates that there are no additional columns, so it is
probably OK to match an exact number of columns at the end like this.
---
 guix/build/syscalls.scm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/guix/build/syscalls.scm b/guix/build/syscalls.scm
index 552343a481d..726c8e86d06 100644
--- a/guix/build/syscalls.scm
+++ b/guix/build/syscalls.scm
@@ -621,8 +621,9 @@ current process."
   (if (eof-object? line)
   (reverse result)
   (match (string-tokenize line)
+;; See the proc(5) man page for a description of the columns.
 ((id parent-id major:minor root mount-point
- options _ type source _ ...)
+ options _ ... "-" type source _)
  (let ((devno (string->device-number major:minor)))
(loop (cons (%mount (octal-decode source)
(octal-decode mount-point)
-- 
2.26.2


Efraim Flashner  writes:

>> Something about the way in which we are searching for the patch is off,
>> but I don't have time just now to investigate.  Efraim, if you can
>> figure it out, that'd be nice, but I'll look into it more tomorrow.
>> It's probably something simple and related to commit 2712703.
>
> Leo told me about it yesterday and I pushed a second commit that fixed
> it. We needed to make sure the patch file was included as an input,
> that's why it got #f instead of a string. In any case, commit
> 710cfc330a7ed06934a193583b159fbdf07bf2fe takes care of it; it's the
> combination of 2712703 and the squash commit.
>
> I'm now running make guix-binary.powerpc64le-linux.tar.xz and so far
> it's made it past the initial building stages, we're on to building the
> grafts now.

Thank you.  This fixed the patch-related problem for me, too.  I'm
currently running "make guix-binary.powerpc64le-linux.tar.xz", also, on
my Debian ppc64le machine.  We'll see how far it gets - fingers crossed!

-- 
Chris


signature.asc
Description: PGP signature

Re: Release on April 18th?

[Chris Marusich]

Hi Efraim and Ludo,

Efraim Flashner  writes:

> My plan was absolutely to merge master into core-updates after and then
> integrate all the changes into their affected packages. I'd also make
> sure to bump gcc to 8 (assuming we don't go straight to 9).

Sounds good.  If we can get powerpc64le-linux working on master, I'd be
very happy.  We can simultaneously try that while still working on
wip-ppc64le (based on core-updates).

I tried out the wip-ppc64le-for-master branch.  I can build it manually
on my Debian ppc64le system, but "make check" fails.  There are two test
failures.

The first test failure is tests/syscalls.scm.  It seems that Ludo's
recently added mount procedure (7e9d9f2 syscalls: Add 'mounts' and the
 record type.) does not work on my system.  In fact, it does not
work on my Debian ppc64le system, nor does it work on my x86_64 Fedora
system.  In both cases, the code makes the incorrect assumption that the
type and source are located in columns 8 and 9, like so:

--8<---cut here---start->8---
(define (mounts)
  "Return the list of mounts ( records) visible in the namespace of the
current process."
  (define (string->device-number str)
(match (string-split str #\:)
  (((= string->number major) (= string->number minor))
   (+ (* major 256) minor

  (call-with-input-file "/proc/self/mountinfo"
(lambda (port)
  (let loop ((result '()))
(let ((line (read-line port)))
  (if (eof-object? line)
  (reverse result)
  (match (string-tokenize line)
((id parent-id major:minor root mount-point
 options _ _ type source _ ...)
 (let ((devno (string->device-number major:minor)))
   (loop (cons (%mount (octal-decode source)
   (octal-decode mount-point)
   devno type options)
   result)))
--8<---cut here---end--->8---

However, on my systems, the correct columns are 9 and 10.  Here's the
first few lines of output from Fedora:

--8<---cut here---start->8---
$ cat /proc/self/mountinfo 
22 97 0:21 / /sys rw,nosuid,nodev,noexec,relatime shared:2 - sysfs sysfs rw
23 97 0:5 / /proc rw,nosuid,nodev,noexec,relatime shared:24 - proc proc rw
24 97 0:6 / /dev rw,nosuid shared:20 - devtmpfs devtmpfs 
rw,size=3923828k,nr_inodes=980957,mode=755
...
--8<---cut here---end--->8---

And here it is from Debian:

--8<---cut here---start->8---
22 28 0:20 / /sys rw,nosuid,nodev,noexec,relatime shared:7 - sysfs sysfs rw
23 28 0:21 / /proc rw,nosuid,nodev,noexec,relatime shared:11 - proc proc rw
24 28 0:5 / /dev rw,nosuid,noexec,relatime shared:2 - devtmpfs udev 
rw,size=15625664k,nr_inodes=244151,mode=755
...
--8<---cut here---end--->8---

On these systems, the 7th column is an optional field, like "shared:7".
The proc man page has this to say about column 7:

  (7) optional fields: zero or more fields of the form "tag[:value]"

So presumably there can be even more than just one optional field.  In
any case, Leo Le Bouter kindly checked his own x86_64 system, where he
observed different output:

--8<---cut here---start->8---
$ cat /proc/self/mountinfo 
23 27 0:21 / /proc rw,relatime - proc none rw
24 27 0:5 / /dev rw,relatime - devtmpfs none 
rw,size=7972408k,nr_inodes=1993102,mode=755
25 27 0:22 / /sys rw,relatime - sysfs none rw
--8<---cut here---end--->8---

As you can see, in this case there is no optional column, so the mount
procedure works fine on Leo's system.  But it fails on mine.

Ludo, do you have an opinion on how to fix this?  It seems like we can't
assume the number of columns will always be the same, so I guess we'll
have to somehow ignore the optional columns more intelligently, if we
want to keep using string-tokenize to do this.

The second test failure is tests/pack.scm.  There are two contributing
factors to this test failure.

The first contributing factor was commit 8f52ea2 on
wip-ppc64le-for-master.  It fixes an issue that is not present on
master, and for that reason it actually causes a problem if it's
included.  I have removed it from the branch in Savannah; please update
your local copy.

The second contributing factor is this bug:

https://issues.guix.gnu.org/47018

However, we can work around it by simply not running guix-daemon when
running the test.  When guix builds guix, it'll be done in the build
environment, so these problematic tests will probably be skipped.

Finally, I tried running make guix-binary.powerpc64le-linux.tar.xz just
to see how far it would get, anyway.  I was quickly greeted with this
failure when building glibc-intermediate:

--8<---cut 

Re: Release on April 18th?

[Chris Marusich]

Vincent Legoll  writes:

> I'm all for that, what can I do to help ?

Thank you for offering to help!

> I don't have a Talos, though...
>
> So only cross- or emulated- stuff...
>
> Willing to help, but needs directions.

I can probably also set you up with a temporary multi-core VM for
testing, too.  I'll see if I can set something like that up for you in
the next few days.

In addition to the ways you can help as Efraim mentioned elsewhere, on
my Debian ppc64le system, where I have manually built the wip-ppc64le
branch, I have observed a valgrind build failure when attempting to
build guix (e.g., when running "make
guix-binary.powerpc64le-linux.tar.xz").  I haven't had time to
investigate it more deeply, so that's one thing you could look into.

However, I have also noticed that after rebasing wip-ppc64le today onto
core-updates, there is a new test failure when "make check" runs.  The
tests/syscalls.scm test fails; it did not fail before the rebase, so
something on core-updates must have changed that broke it...  You could
investigate this, too, if you had a machine.

Basically, if you have a machine, you can fetch wip-ppc64le and try to
build it (./bootstrap && ./configure --localstatedir=/var && make -j &&
make check), and report/investigate the errors.  Feel free to send bugs
to bug-g...@gnu.org mentioning the commit and the reproduction steps, so
we can talk in a new bug report about any details.  That would be the
quite helpful, I think!

And yes, as Efraim mentioned, even without hardware, you could try
building your favorite packages on x86_64-linux (or whatever system you
are currently using) on the wip-ppc64le branch and report when there are
problems.  We don't want to break anything for the other system types,
but you never know what will happen.  This sort of testing might uncover
problems on core-updates, too, since wip-ppc64le is currently based off
of core-updates.

Anyway, I'll email you privately about VM access later.

-- 
Chris


signature.asc
Description: PGP signature

Re: Release on April 18th?

[Chris Marusich]

Hi Efraim,

Efraim Flashner  writes:

> I wanted an Easy Win™ and I've pushed a branch wip-ppc64le-for-master,
> which cherry-picked (I think) all of the powerpc64le commits and
> adjusted them to work against master WITHOUT affecting other
> architectures.

It looks like you modified "gnu: glibc: Fix ldd path on powerpc" and
"gnu: gcc-4.7: On powerpc64le, fix /lib64 references" to avoid changing
the package definitions on other architectures.  That might work out;
I'll try testing it on my hardware and report back.  Believe me, I'd be
happy to have powerpc64le-linux working on any branch.

However, the moment master and core-updates are integrated with one
another, glibc will be upgraded upgraded from 2.31 to 2.32, and it will
be necessary to upgrade GCC, also.  I don't think there's any way around
that.  If possible, I'd really prefer not to hide that big of a change
behind an "if powerpc64le-linux" statement, since I don't like the idea
of having to deal with unique failures on powerpc64le-linux that might
stem from using a different GCC version than the rest of Guix.  It is
good to be consistent with other architectures, when we can be.  Then we
can share the same problems and help fix them together.

-- 
Chris


signature.asc
Description: PGP signature

Re: I've rebased wip-ppc64le onto core-updates

[Chris Marusich]

Hi,

I've rebased wip-ppc64le again just now onto the latest core-updates.
There were no conflicts.

Because I committed 5d2863dfe4613d5091e61800fcd5a48922c8ce4e and
001fc29b43fd0beb365d536774fae96624309413 directly to core-updates before
rebasing, I removed some similar commits from wip-ppc64le that are no
longer needed.  The rest is the same as before.

Ludovic Courtès  writes:

> We’ll have to think about merging.  At things stand, it seems that the
> bits that could go to ‘master’ (those that don’t trigger a world
> rebuild) are already there, right?
>
> For the rest, we could aim for rebasing on core-updates and merging
> there.

Essentially, yes.  The current plan is to periodically rebase
wip-ppc64le onto core-updates while working out the last few
powerpc64le-linux issues, then merge wip-ppc64le into core-updates, and
then merge core-updates into master.

I've taken care to only include in wip-ppc64le those commits that are
relevant to the porting effort.  Although there are some minor commits
on wip-ppc64le that could be cherry-picked onto master, I'm not sure
it's necessary or particularly helpful to go out of our way to do that
right now.  I would prefer to just finish the wip-ppc64le branch, merge
it into core-updates, and then merge core-updates into master.

Efraim Flashner  writes:

> As noted in another thread this would involve updating gcc to gcc-8.
> There are some other patches which could go straight into master (I'm
> looking at libelf specifically, probably others), but glibc and
> gcc/libstdc++ I'm not sure how to do those ones.
>
> Actually, looking at the branch there's also findutils-boot0, but that
> can be changed to only be for target-powerpc for now and changed in
> core-updates. I have a similar one I had to do for binutils on
> powerpc-linux that I managed to make not affect other architectures.

The libelf fix (b13ba3e gnu: libelf: Fix compilation for
powerpc64le-linux) could go to master, but it's a no-op for all systems
other than powerpc64le-linux.  Since nobody is using the
powerpc64le-linux port yet, there is no need for that change to go to
master at this time.

The findutils-boot0 fix (f4cbd18 gnu: commencement: Fix findutils-boot0
on some systems) fixes findutils-boot0 not only for powerpc64le-linux,
but for any system that does not use glibc-mesboot (i.e., any system
other than x86_64-linux or i686-linux).  We could limit the change to
just fix it for powerpc64le-linux, but I think the other systems will
need the fix, too.

In short, I think it will be simplest to just merge wip-ppc64le into
core-updates and then merge core-updates into master.

-- 
Chris


signature.asc
Description: PGP signature

Re: Release on April 18th?

[Chris Marusich]

Hi,

zimoun  writes:

> Is it doable to have core-updates merged in the next weeks?  Or not at
> all.

Do we plan to upgrade GCC?  This is required for the powerpc64le-linux
port; see below for details.

The wip-ppc64le branch, which ports Guix to an architecture that can be
run on freedom-friendly POWER9 systems like the Talos II and Blackbird
from Raptor Computing Systems, is "almost" done, and I would really like
to try to get it into the next release if possible.

However, there is still some work to do, and I wouldn't necessarily want
to block the release just for sake of the powerpc64le-linux port.

In fact, the wip-ppc64le branch was "fully functional" for a little
while.  By "fully functional," I mean that "make check" succeeded, "make
guix-binary.powerpc64le-linux.tar.xz" succeeded, the resulting binary
could be installed on a fresh Debian ppc64le system, the binary could
successfully build and run GNU Hello, and the binary could successfully
run "guix pull", and the newly installed "guix pull" could do the same.

However, that was using glibc 2.31.  On core-updates, glibc has been
updated to 2.32, which unfortunately breaks wip-ppc64le.  To fix it, we
must upgrade GCC from 7 to 8 (or greater), and that is what we have done
on the wip-ppc64le branch (upgrade GCC to 8).  Currently, we are working
through build failures that occurred as a result, but I'm optimistic that
we can resolve those in the coming weeks.

The real question, I think, is when do we plan to upgrade GCC on
core-updates?  It's still on GCC 7.5.0.  The powerpc64le-linux port will
require GCC 8 or greater, since core-updates is now using glibc 2.32.

-- 
Chris


signature.asc
Description: PGP signature

Re: core-updates: Emacs is only supported on x86_64-linux?

[Chris Marusich]

Chris Marusich  writes:

> How about a patch like the following - would it be acceptable to you?

Actually, I've realized that that patch wasn't quite right.  I've
attached a corrected version to this email.

Although the %current-system parameter will look like "x86_64-linux"
because it's a Guix system name, the %current-target-system parameter
will look like "x86_64-linux-gnu" (or whatever the user happens to
supply, e.g. via the --target option of "guix build") because it's a GNU
triplet.  The attached patch accomplishes the same goal as before, but
it uses string-prefix? to check whether we're building for an x86_64
machine, rather than matching on an exact string.

-- 
Chris
From 0eb5583f243a399293ae52a3e78ccf7d3a153a47 Mon Sep 17 00:00:00 2001
From: Chris Marusich 
Date: Mon, 8 Mar 2021 23:13:39 -0800
Subject: [PATCH] gnu: Restore emacs' supported systems.

Recently, librsvg was updated to depend upon rust.  That change inadvertently
restricted the "supported" systems of emacs (among other package) to
x86_64-linux only, since that is the only system declared in the rust
package's list of supported systems.  This unintentionally made emacs
unavailable on all other systems.

This change fixes that by removing the rust dependency from some packages.
The packages remain unchanged from the perspective of an x86_64-linux system,
but from the perspective of other systems, the packages are now supported
again (albeit without certain optional SVG features).

* gnu/packages/gtk.scm (gtk+, gtk+-2)[propagated-inputs]: If compiling for
x86_64, use gdk-pixbuf+svg as the "gdk-pixbuf" input, as usual.  Otherwise,
use gdk-pixbuf, which avoids adding librsvg (thus rust) to the closure of
inputs.  Note that both gtk+ and gtk+-2 are in the transitive closure of
inputs of emacs, so these two changes are necessary.
* gnu/packages/emacs (emacs)[inputs]: If compiling for x86_64, add librsvg to
the inputs, as usual.  Otherwise, do not add it, which avoids adding rust to
the closure of inputs.
---
 gnu/packages/emacs.scm |  8 +++-
 gnu/packages/gtk.scm   | 14 --
 2 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 98061c93ae0..fe5b3b25b3d 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -71,6 +71,7 @@
   #:use-module (gnu packages xml)
   #:use-module (gnu packages xorg)
   #:use-module (guix utils)
+  #:use-module (ice-9 match)
   #:use-module (srfi srfi-1))
 
 (define-public emacs
@@ -236,7 +237,12 @@
("libpng" ,libpng)
("zlib" ,zlib)
 
-   ("librsvg" ,librsvg)
+   ;; librsvg is an optional dependency that pulls in rust.  Rust is not
+   ;; supported well on every architecture yet.
+   ,@(if (string-prefix? "x86_64" (or (%current-target-system)
+  (%current-system)))
+ `(("librsvg" ,librsvg))
+ '())
("libxpm" ,libxpm)
("libxml2" ,libxml2)
("libice" ,libice)
diff --git a/gnu/packages/gtk.scm b/gnu/packages/gtk.scm
index f458366fb6a..d396425d9a6 100644
--- a/gnu/packages/gtk.scm
+++ b/gnu/packages/gtk.scm
@@ -776,7 +776,12 @@ is part of the GNOME accessibility project.")
(outputs '("out" "bin" "doc"))
(propagated-inputs
 `(("atk" ,atk)
-  ("gdk-pixbuf" ,gdk-pixbuf+svg)
+  ;; SVG support is optional and requires librsvg, which pulls in rust.
+  ;; Rust is not supported well on every architecture yet.
+  ("gdk-pixbuf" ,(if (string-prefix? "x86_64" (or (%current-target-system)
+  (%current-system)))
+ gdk-pixbuf+svg
+ gdk-pixbuf))
   ("pango" ,pango)))
(inputs
 `(("cups" ,cups)
@@ -843,7 +848,12 @@ application suites.")
(propagated-inputs
 `(("at-spi2-atk" ,at-spi2-atk)
   ("atk" ,atk)
-  ("gdk-pixbuf" ,gdk-pixbuf+svg)
+  ;; SVG support is optional and requires librsvg, which pulls in rust.
+  ;; Rust is not supported well on every architecture yet.
+  ("gdk-pixbuf" ,(if (string-prefix? "x86_64" (or (%current-target-system)
+  (%current-system)))
+ gdk-pixbuf+svg
+ gdk-pixbuf))
   ("libepoxy" ,libepoxy)
   ("libxcursor" ,libxcursor)
   ("libxi" ,libxi)
-- 
2.26.2



signature.asc
Description: PGP signature

Re: core-updates: Emacs is only supported on x86_64-linux?

[Chris Marusich]

Hi,

Mark H Weaver  writes:

> Ugh.  I'd prefer to avoid removing 'gtk+' from the inputs to 'emacs' on
> any system, because the distinguishing characteristic of that package
> (compared with the other Emacs variants) is that it's the Gtk+ variant
> of Emacs.

How about a patch like the following - would it be acceptable to you?

diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 98061c93ae..4d2caf205a 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -71,6 +71,7 @@
   #:use-module (gnu packages xml)
   #:use-module (gnu packages xorg)
   #:use-module (guix utils)
+  #:use-module (ice-9 match)
   #:use-module (srfi srfi-1))
 
 (define-public emacs
@@ -236,7 +237,12 @@
("libpng" ,libpng)
("zlib" ,zlib)
 
-   ("librsvg" ,librsvg)
+   ;; librsvg is an optional dependency that pulls in rust.  Rust is not
+   ;; supported well on every architecture yet.
+   ,@(match (or (%current-target-system)
+(%current-system))
+("x86_64-linux" `(("librsvg" ,librsvg)))
+   (_ '()))
("libxpm" ,libxpm)
("libxml2" ,libxml2)
("libice" ,libice)
diff --git a/gnu/packages/gtk.scm b/gnu/packages/gtk.scm
index f458366fb6..0468cbf830 100644
--- a/gnu/packages/gtk.scm
+++ b/gnu/packages/gtk.scm
@@ -776,7 +776,12 @@ is part of the GNOME accessibility project.")
(outputs '("out" "bin" "doc"))
(propagated-inputs
 `(("atk" ,atk)
-  ("gdk-pixbuf" ,gdk-pixbuf+svg)
+  ;; SVG support is optional and requires librsvg, which pulls in rust.
+  ;; Rust is not supported well on every architecture yet.
+  ("gdk-pixbuf" ,(match (or (%current-target-system)
+(%current-system))
+("x86_64-linux" gdk-pixbuf+svg)
+(_ gdk-pixbuf)))
   ("pango" ,pango)))
(inputs
 `(("cups" ,cups)
@@ -843,7 +848,12 @@ application suites.")
(propagated-inputs
 `(("at-spi2-atk" ,at-spi2-atk)
   ("atk" ,atk)
-  ("gdk-pixbuf" ,gdk-pixbuf+svg)
+  ;; SVG support is optional and requires librsvg, which pulls in rust.
+  ;; Rust is not supported well on every architecture yet.
+  ("gdk-pixbuf" ,(match (or (%current-target-system)
+(%current-system))
+("x86_64-linux" gdk-pixbuf+svg)
+(_ gdk-pixbuf)))
   ("libepoxy" ,libepoxy)
   ("libxcursor" ,libxcursor)
   ("libxi" ,libxi)

I've tested this patch (on its own, applied to wip-ppc64le locally,
without the other two patches mentioned earlier in this thread), and it
successfully restores the "supported systems" for emacs (thus fixing the
tests/package.sh test failure), without changing rust's list of
supported systems, which remains hard-coded to x86_64-linux.

Eventually I think I will definitely need a change like the one Chris
proposed in order to actually troubleshoot build failures involving rust
on powerpc64le-linux, but I suppose when the time comes, I can do it in
a private branch and save the build farm some wasted cycles.  It's fine
with me if we don't make a change like that right now, since it isn't
blocking my porting work.

Mark H Weaver  writes:

> Aside: I wish that Guix included a convenient tool to answer the
> question "Why does package X depend on package Y?", i.e. "What paths of
> dependencies lead from package X to package Y?", without having to view
> the entire dependency graph (which is often too complex to grasp
> visually).

Ricardo Wurmus  writes:

> (“guix graph” is of no help here, because it doesn’t show build system
> packages, so I looked through the derivations.)

The "--paths" option with "--type=bag" shows you this (results
below were, of course, taken before applying the patch above):

--8<---cut here---start->8---
marusich@suzaku:~/repos/guix$ ./pre-inst-env guix graph --type=bag --path gtk+ 
rust
gtk+@3.24.24
gdk-pixbuf+svg@2.42.2
librsvg@2.50.3
rust@1.49.0
--8<---cut here---end--->8---

Christopher Baines  writes:

> I've gone ahead and pushed the patch I proposed to master, I think it's
> a step forward.
>
> As you say, adapting the change for core-updates might be good as
> well. I want to check though if rust builds for i686-linux on
> core-updates, as the path is different to master, so it may well work.
>
> So yeah, once I've found out whether rust works on i686-linux on
> core-updates, I might make a change there too.

I don't have a strong personal opinion about this, since I'm building
everything from source anyway, and I want a patch like this eventually
on core-updates, too.  However, in light of Mark's comments, is it a
good idea to apply that patch right now?

-- 
Chris


signature.asc
Description: PGP signature

Re: core-updates: Emacs is only supported on x86_64-linux?

[Chris Marusich]

Hi all,

FYI, I'm sending this message to 46...@debbugs.gnu.org via Bcc, also.
I'm not sure if that works, but we'll see...

Mark H Weaver  writes:

> It's not intended.  Emacs should certainly be supported on every system
> that Guix supports.

Thank you for confirming.  That is what I expected.

> For now, I suggest that Emacs should have input 'librsvg' only on
> 'x86_64-linux' systems.  Something like this (untested), for
> core-updates:
>
> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
> index 98061c93ae..de6101cf17 100644
> --- a/gnu/packages/emacs.scm
> +++ b/gnu/packages/emacs.scm
> @@ -71,6 +71,7 @@
>#:use-module (gnu packages xml)
>#:use-module (gnu packages xorg)
>#:use-module (guix utils)
> +  #:use-module (ice-9 match)
>#:use-module (srfi srfi-1))
>  
>  (define-public emacs
> @@ -236,7 +237,10 @@
> ("libpng" ,libpng)
> ("zlib" ,zlib)
>  
> -   ("librsvg" ,librsvg)
> +   ,@(match (or (%current-target-system)
> +(%current-system))
> +   ("x86_64-linux" `(("librsvg" ,librsvg)))
> +   (_ `()))
> ("libxpm" ,libxpm)
> ("libxml2" ,libxml2)
> ("libice" ,libice)
>
> Ditto for all other packages with 'librsvg' as an optional dependency.

I've confirmed that works for emacs, except that you actually have to
also do it for gtk+, too, since rust gets pulled in via gtk+ also.  So,
something like this:

From 649c89e5862e1ed887f5fd863ef7bb32f97bbe74 Mon Sep 17 00:00:00 2001
From: Chris Marusich 
Date: Sun, 7 Mar 2021 17:42:37 -0800
Subject: [PATCH] gnu: emacs: Use librsvg and gtk+ on x86_64-linux only.

* gnu/packages/emacs.scm (emacs)[inputs]: Only add librsvg when the
%current-target-system or %current-system is "x86_64-linux".  This avoids
pulling rust into the transitive closure of inputs on systems where Rust
support is currently lacking.
---
 gnu/packages/emacs.scm | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 98061c93ae0..f0797ae2347 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -71,6 +71,7 @@
   #:use-module (gnu packages xml)
   #:use-module (gnu packages xorg)
   #:use-module (guix utils)
+  #:use-module (ice-9 match)
   #:use-module (srfi srfi-1))
 
 (define-public emacs
@@ -219,7 +220,6 @@
 
;; TODO: Add the optional dependencies.
("libx11" ,libx11)
-   ("gtk+" ,gtk+)
("cairo" ,cairo)
("pango" ,pango)
("harfbuzz" ,harfbuzz)
@@ -236,7 +236,6 @@
("libpng" ,libpng)
("zlib" ,zlib)
 
-   ("librsvg" ,librsvg)
("libxpm" ,libxpm)
("libxml2" ,libxml2)
("libice" ,libice)
@@ -246,7 +245,15 @@
 
;; multilingualization support
("libotf" ,libotf)
-   ("m17n-lib" ,m17n-lib)))
+   ("m17n-lib" ,m17n-lib)
+
+   ;; These are optional dependencies that pull in rust.  Rust is not
+   ;; supported well on every architecture yet.
+   ,@(match (or (%current-target-system)
+(%current-system))
+   ("x86_64-linux" `(("gtk+" ,gtk+)
+ ("librsvg" ,librsvg)))
+   (_ '()
 (native-inputs
  `(("guix-emacs.el" ,(search-auxiliary-file "emacs/guix-emacs.el"))
("pkg-config" ,pkg-config)
-- 
2.26.2


What do you think about that?  If there are no problems, I'll go ahead
and commit it to core-updates.

Note that because that patch re-orders the inputs, it causes emacs to be
rebuilt, which should be fine because it's core-updates.  It seemed
better to group these two inputs in a single match clause, rather than
scattering them in two different but basically identical match clauses
just to keep their original ordering intact.

Christopher Baines  writes:

> Chris Marusich  writes:
>
>> I've noticed that the emacs package only supports x86_64-linux, at least
>> on core-updates.  Is that intended?
>
> The relevant commit [1] does mention the intent "Remove "i686-linux"
> from supported systems.", but that does differ from my interpretation of
> the change, which is only support x86_64-linux.
>
> 1: 
> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=0ed631866cc0b7cece2b0a0b50e39b37ae91bb67
>
> I've sent a patch that should add back "support" for other systems:
>
>   https://issues.guix.gnu.org/46987
>
> That's for master at least, I haven't looked in to what the situation is
> on core-updates. It's worth there checking if the build stil

core-updates: Emacs is only supported on x86_64-linux?

[Chris Marusich]

Hi,

I've noticed that the emacs package only supports x86_64-linux, at least
on core-updates.  Is that intended?

I noticed because it caused "make check" to fail on the wip-ppc64le
branch (which is based on core-updates).  I fixed one failing test on
the wip-ppc64le branch by using coreutils instead of emacs in the test:

https://git.savannah.gnu.org/cgit/guix.git/commit/?h=wip-ppc64le=1900a6227e99427cf3b28a86dbbee2c55f375f8c

I suspect that - on core-updates, at least - Guix does not currently
pass its "make check" test suite on any system other than x86_64-linux.

I would like to cherry-pick the above fix onto core-updates.  However,
before doing that, I wanted to check on this list to see if anyone knew
anything about the current situation.  Is it intended that the emacs
package only supports x86_64-linux?

As for the cause, it looks like one contributing factor might be the
rust package.  It was recently added to the transitive closure of inputs
of emacs.  The rust package explicitly declares x86_64-linux as its only
supported system.  This restriction percolates up to emacs, and indeed
to any other package that contains rust in its transitive closure of
inputs.  However, as of 1ced8379c7641788fa607b19b7a66d18f045362b, emacs
did not contain rust in its transitive closure of inputs, so the change
must have happened in some commit after that.

It would be nice if someday the rust package could support more systems.
However, my primary goal here is just to get Guix to build and pass its
tests on powerpc64le-linux.  Getting things like rust and emacs to work
after that will be another challenge.

-- 
Chris


signature.asc
Description: PGP signature

I've rebased wip-ppc64le onto core-updates

[Chris Marusich]

Hi Léo and others,

I just wanted to let you know that I've rebased the wip-ppc64le branch
onto core-updates.  The wip-ppc64le branch head used to be
147b74817e6cf97f37090ecfd52e2588f4c39f7e, but now it's
df5d633db7acf6389ca9d444b8f5784a0da5ac5d.

I wanted to inform you so you don't get caught off-guard the next time
you update your local copy.

I've squished some commits together where it made sense to do so.  I've
omitted some commits which were cherry-picked before, but which are
already in the core-updates branch.  And I have explicitly NOT merged
master into wip-ppc64le at this time.

I have also taken the liberty of cherry-picking Tobias' commit
65fb3a3eb9a759e3dba18e78113e80d7e5b353f4 from master, which adds Léo to
.guix-authorizations.  This should allow him to add commits on
wip-ppc64le without worrying about causing problems for "guix pull"
later.

-- 
Chris


signature.asc
Description: PGP signature

Re: 02/03: tests: gremlin: Skip file-needed/recursive if DT_NEEDED is empty.

[Chris Marusich]

Hi Ludo,

Ludovic Courtès  writes:

> guix-comm...@gnu.org skribis:
>
>> +++ b/tests/gremlin.scm
>> @@ -61,7 +61,12 @@
>>   (elf-dynamic-info-needed dyninfo))
>>  
>>  (unless (and %guile-executable (not (getenv "LD_LIBRARY_PATH"))
>> - (file-needed %guile-executable)) ;statically linked?
>> + (file-needed %guile-executable) ;statically linked?
>> + ;; When Guix has been built on a foreign distro, using a
>> + ;; toolchain and libraries from that foreign distro, it is not
>> + ;; unusual for the runpath to be empty.
>> + (and=> (file-runpath %guile-executable)
>> +(compose not null-list?)))
>
> Nitpick: you can write ‘pair?’ instead of ‘(compose …)’.

Thank you for the suggestion!  I've done this in commit
b22dcb24207022d9a716893836578e96f4b580a1.

> I guess the problem is that ‘file-needed/recursive’ only looks at
> RUNPATH, ignoring the notion of “standard directories” like /usr/lib.

Yes, that's exactly my thought as well.  Is it intended that
"gremlin.scm" should be able to search those places?

My understanding is that the answer is no, we don't expect it to be able
to search those places, since in theory (despite the Filesystem
Hierarchy Standard) the directories to search could be anywhere,
depending on how libraries are managed by the foreign distro, and this
code is intended primarily to be used on the build side, where we
wouldn't encounter such cases anyway.

My understanding is that skipping this test is the right solution,
rather than attempting to teach gremlin to always be able to find
libraries when the runpath is empty.  What do you think?

-- 
Chris


signature.asc
Description: PGP signature

Re: Branch management, wip-ppc64le, and POWER9

[Chris Marusich]

Hi,

Christopher Baines  writes:

> I think rebasing introduces issues with commit signatures, since you'd
> then be signing others commits. If multiple people are committing to the
> branch, I'd treat it like staging/core-updates, and merge rather than
> rebasing.

Leo Famulari  writes:

> The signatures will be lost, but for "wip-" branches the expectation has
> always been that history may be rewritten. For me, rebasing is a more
> comfortable workflow for this kind of exploratory branch. I recommend
> against using merges here.

The original signatures will be lost, yes.  Depending on how you've
configured Git, your own signature may or may not be applied to the
newly recreated commits.  I just tested this in a throwaway repository,
and when I rebase commits, I re-sign my commits. However, that is
probably because I've set commit.gpgSign to true in my git config.
Either way, you're right that it's important to watch out for.

Regarding rebasing vs. merging, both can work.  I think it depends on
who is making the changes.  For now, I would prefer to avoid rebasing on
wip-ppc64le.  If somebody wants to commit, please feel free, but please
make sure the commits are of the same quality that you would normally
expect for changes going to master, core-updates, or staging, since
eventually we will be merging it into those branches.

If we need to rebase wip-ppc64le, that's fine, but let's talk about it
first so nobody (especially me!) is caught off-guard.

For other wip branches, I think whoever is working on them should agree
on how to coordinate.  I'm just focusing on wip-ppc64le right now, since
I want POWER9 support.

Christopher Baines  writes:

> core-updates is pretty broken at the moment (building the channel
> derivation fails), so I wouldn't recommend that.

This is good to know!  I'll try to find a stable point from which to
merge, when the time comes (which might be soon!).

> Merging master as soon as conflicts arise sounds good, I think it's
> easier to handle the conflicts in smaller batches, so more frequent
> merges are useful.

I agree.  Frequent merges (or rebases) are important for avoiding
conflicts.  Merges from stable points, if possible, are better than
merges from random commits in a branch, though.

> Once both branches are in a good state, maybe then will be the time to
> merge core-updates in to wip-ppc64le to test that combination of
> changes, and then if that goes well, merge wip-ppc64le in to
> core-updates and stop using the wip-ppc64le branch.

On IRC, dftxbs3e also mentioned that it would be unwise to merge from
core-updates at this time for other reasons, beyond simply "guix pull"
possibly being broken.  So we'll be holding off on merging core-updates
into wip-ppc64le just yet.  See here for details (search for "merge"):

https://logs.guix.gnu.org/guix/2021-02-06.log

When wip-ppc64le is in better shape, we can decide what to do.  In the
meantime, dftxbs3e and I agree that it would be best to cherry-pick
essential fixes that are already on core-updates (e.g., I cherry-picked
a48a3f0640d76cb5e5945557c9aae6dabce39d93 onto wip-ppc64le), rather than
merging core-updates prematurely.

>> - For Efraim specifically, there is some overlap between the work
>>   required on wip-ppc64le and the work required on wip-ppc.  I have
>>   already cherry-picked one of your commits
>>   (2da8fcfdee7cfde8110a68806f3c4d497f217fe5) onto wip-ppc64le (as commit
>>   52f0b3db6ef3d3c5067c704c2190f72d2994a999), since it was needed.  How
>>   would you like to coordinate this work?
>>
>> - Are there any other tips/advice/rules?
>
> I'd really like to develop the Guix Data Service instance here [1] and
> associated Guix Build Coordinator instance in to something that can be
> used to work with non-master branches.
>
> 1: https://data.guix-patches.cbaines.net/
>
> wip-ppc64le is already being processed [2], but ppc64le derivations
> aren't being computed. I think adding a change like [3] to the branch
> would make sense, and prompt the Guix Data Service to compute those
> derivations.
>
> 2: https://data.guix-patches.cbaines.net/repository/2/branch/wip-ppc64le
> 3: 
> https://git.guix-patches.cbaines.net/guix-patches/commit/?h=wip-lle-bout-le1=53dcca860787ae6cf3949d9c03c0108c47d47c9e
>
> Once there are computed derivations, then the connected Guix Build
> Coordinator instance can start building them. This isn't meant for
> providing substitutes to users, but instead for quality assurance. Does
> that all make sense?

Yes, that makes sense, and I totally agree.  We definitely want to keep
the other architectures in mind, and hopefully this will be a great tool
to help understand any impact there.  I expect our changes to cause
full-world rebuilds for all architectures, since it requires changes in
packages deep in the dependency graph, but none of the changes on
wip-ppc64le should break anything for the other architectures.

I plan to make a change similar to
53dcca860787ae6cf3949d9c03c0108c47d47c9e, in 

Re: Emacs and URLs in Git commit messages

[Chris Marusich]

Hi,

Thank you for the replies!

Maxime Devos  writes:

> I don't known any emacs command for that, but you inspired me to write
> such a command myself: [1].
>
> Maxime.
> [1]: 
> https://notabug.org/mdevos/things/commit/b0400ba06b6f031e88f1f89b47079c3c6d7dcac4

zimoun  writes:

> I am not representative since I commit few fixes.  Well, I have a tiny
> helper that pushes to the kill ring:
>
> 
>
> Especially reading bug#45314 in Debbugs mode, I type “M-x
> my/guix-issues” then the URL ’http://issues.guix.gnu.org/issue/45314’ is
> stashed.  If not in debbugs mode, as here in message mode, the tiny
> helper asks the bug number then pushes it to the kill ring.  This tiny
> helper is far from perfect, any improvements is welcome. :-) Especially
> if it is already provided by an Emacs command.
>
> About the brackets, I type them.

Ludovic Courtès  writes:

> I have this helper for debbugs.el:
>
> (defun ludo-copy-debbugs-url ()
>   "Add to the kill ring the URL of the Debbugs issue at point."
>   (interactive)
>   (let ((url1 (concat "https://bugs.gnu.org/;
> (number-to-string (debbugs-gnu-current-id
>   (url2 (concat "https://issues.guix.gnu.org/;
> (number-to-string (debbugs-gnu-current-id)
> (kill-new url1)
> (kill-new url2)
> (message "Copied %s and %s" url1 url2)))
>
> (define-key debbugs-gnu-mode-map (kbd "C-w") 'ludo-copy-debbugs-url)
>
> That way I can C-w on a bug in *Guix Bugs* and I get the two URLs in the
> clipboard (I normally use “bugs.gnu.org” as the canonical bug URL.)
>
> Ludo’.

OK, I see.  So I'm not missing out on some built-in Emacs (or
debbugs-emacs) magic; most people just put together something convenient
on their own.  That makes sense!  Thank you for the examples; it's
helpful to see how others are doing it.  I think I'll try something
similar.

Bengt Richter  writes:

> I am not sure I understand your context or goal in searching ;/

I'm always interested in finding more effective ways to get things done.
Sometimes, asking people how they do something is the best way to
discover new methods, even if the answer seems simple or obvious.

-- 
Chris


signature.asc
Description: PGP signature

Branch management, wip-ppc64le, and POWER9

[Chris Marusich]

Hi,

Since I'm now making commits on wip-ppc64le to add support for POWER9
machines like the Talos II and Blackbird from Raptor Computing Systems,
I'd like to understand more about how I am expected to manage that
branch.  Before I touched it, it sat with just one lonely commit,
essentially unused.  So it seems to me like I'm not stepping on anyone's
toes, and that I should just make commits like I normally would on
master, staging, or core-updates, being careful to follow the usual
guidelines.

Specifically, I'm curious to know:

- Is it usually expected that wip branches can be rebased?  I don't plan
  to rebase wip-ppc64le, since I'd like to be able to coordinate with
  others using this branch, but I'm not sure what others expect.

- Should I just periodically merge from master, or perhaps core-updates,
  to keep wip-ppc64le up to date and resolve conflicts early?  I've
  already merged master into this branch once, to get it back up to
  date.  I haven't merged core-updates into it yet.  I think
  periodically merging branches into wip-ppc64le is the right thing to
  do, since eventually the end goal will be to merge wip-ppc64le into
  core-updates to get POWER9 support into a release.  However, I suppose
  if I merge from different branches, I could wind up dealing with
  unnecessary conflicts.

- For Efraim specifically, there is some overlap between the work
  required on wip-ppc64le and the work required on wip-ppc.  I have
  already cherry-picked one of your commits
  (2da8fcfdee7cfde8110a68806f3c4d497f217fe5) onto wip-ppc64le (as commit
  52f0b3db6ef3d3c5067c704c2190f72d2994a999), since it was needed.  How
  would you like to coordinate this work?

- Are there any other tips/advice/rules?

Thank you,

-- 
Chris


signature.asc
Description: PGP signature

Emacs and URLs in Git commit messages

[Chris Marusich]

Hi,

Many Guix commits look like this:

  commit f9978346e73359ac1d8b88c9ed874edc7225582b
  Author: Ludovic Courtès 
  Date:   Fri Dec 18 18:10:04 2020 +0100

  avahi: Remove poll timeout when possible.

  Fixes .

  * guix/avahi.scm (avahi-browse-service-thread): Change timeout default 
value
  to false when no "stop-loop?" procedure is passed. Adapt 
"iterate-simple-poll"
  call accordingly.

  Signed-off-by: Mathieu Othacehe 

Regarding the URL, do people just type it all out, including the opening
and closing brackets (<>)?  Or is there an Emacs command that does it
for you?  I've briefly looked on the Internet, but this is the kind of
thing that seems difficult to search for.

-- 
Chris


signature.asc
Description: PGP signature

PowerPC: reference to static-bash-for-glibc in binutils-final

[Chris Marusich]

Hi Efraim,

The other day, I asked on IRC why it's OK for binutils-final to refer to
static-bash-for-glibc on powerpc architectures, like in commit
2da8fcfdee7cfde8110a68806f3c4d497f217fe5, but it isn't OK on other
architectures.  You said, "there's an extra file that's a bash script
specific to powerpc machines. I suppose we could just un-patch-shebang
it back to /bin/sh".  Thank you for taking the time to respond!

When I build binutils-final on powerpc64le-linux, I get this message:

output (`/gnu/store/n2ivj40h30wa55qwp9dazzfywqb6s6vz-binutils-2.34') is not 
allowed to refer to path 
`/gnu/store/vnshq5g4ghhbr6c3s69q9p8fp6hr0gpx-bootstrap-bi
naries-0'

Looks like the file in question is embedspu, and it does refer to sh
like you said:

marusich@suzaku:~$ grep -r 
/gnu/store/vnshq5g4ghhbr6c3s69q9p8fp6hr0gpx-bootstrap-binaries-0 
/gnu/store/n2ivj40h30wa55qwp9dazzfywqb6s6vz-binutils-2.34
/gnu/store/n2ivj40h30wa55qwp9dazzfywqb6s6vz-binutils-2.34/bin/embedspu:#!/gnu/store/vnshq5g4ghhbr6c3s69q9p8fp6hr0gpx-bootstrap-binaries-0/bin/sh

Well, what's that script, anyway?  The first lines say:

#!/gnu/store/vnshq5g4ghhbr6c3s69q9p8fp6hr0gpx-bootstrap-binaries-0/bin/sh
# Embed an SPU ELF executable into a PowerPC object file.

OK, so yeah, it's probably necessary on various PowerPC architectures,
that seems clear now.  But would changing this to /bin/sh actually work?
If we changed the reference to /bin/sh, wouldn't it cause problems in
build environments, since /bin/sh isn't available there?

Anyway, I'm fine with making a change like
2da8fcfdee7cfde8110a68806f3c4d497f217fe5 for powerpc64le-linux, since it
clearly seems like a necessary reference on that architecture.  That's
probably what I'll do.

-- 
Chris


signature.asc
Description: PGP signature

Re: guix build -d with a target causes many builds

[Chris Marusich]

Hi Chris,

Christopher Baines  writes:

> Grafts.
>
> → guix build --no-grafts -d --target=powerpc64-linux-gnu -e '(@@ (gnu 
> packages make-bootstrap) %gcc-static)'
> /gnu/store/i5wn3xl6p0zw1vglscgk0bs9dwc6hdh6-gcc-static-5.5.0.drv
>
> They're on by default, so when you use -d without --no-grafts, I believe
> you're actually getting the derivation that does the grafting, rather
> than the derivation for the package build.

Grafts!  I should have known.  That makes more sense.  Thank you!

-- 
Chris


signature.asc
Description: PGP signature

Re: guix build -d with a target causes many builds

[Chris Marusich]

Chris Marusich  writes:

> Hi,
>
> I've noticed that Guix builds many things when I ask it to instantiate a
> derivation in the following way:
>
> [0] [env] marusich@garuda-lan:~/guix/repos/guix-worktrees/wip-ppc64
> $ guix build -d --target=powerpc64-linux-gnu -e '(@@ (gnu packages 
> make-bootstrap) %gcc-static)'
> The following derivations will be built:
>/gnu/store/i5wn3xl6p0zw1vglscgk0bs9dwc6hdh6-gcc-static-5.5.0.drv
>
> /gnu/store/3h2sk37iim53fh7g9r3sd1q0xzhqwa51-gcc-cross-powerpc64-linux-gnu-7.5.0.drv
>
> /gnu/store/84k0j5jm316cwf7h66vrw1vmvkd4kbck-glibc-cross-powerpc64-linux-gnu-2.31.drv
>
> /gnu/store/d36n7qy9xbgwpaw3nw8k9dj51hzmdnr4-gcc-cross-sans-libc-powerpc64-linux-gnu-7.5.0.drv
>
> /gnu/store/mqar9bnapfcfkna3rvy28awhlpd3q65q-binutils-cross-powerpc64-linux-gnu-2.34.drv
>
> /gnu/store/pzp93dw3rr6sp2ybi3dzs6kd7gvigfsk-ld-wrapper-powerpc64-linux-gnu-0.drv
>
> /gnu/store/n7dhpsq41q4kdbqgniljbwrlawvmmlp6-linux-libre-headers-cross-powerpc64-linux-gnu-5.4.20.drv
>
> /gnu/store/9p5anrji5wgkf66k09jhbsr3fqwwi7cn-gcc-cross-powerpc64-linux-gnu-7.5.0.drv
>
> /gnu/store/r4ac80znwlrnh4jmj2sbczc4mn66mqdg-glibc-cross-powerpc64-linux-gnu-2.31.drv
>/gnu/store/ap8ri9ddka13vyrsl72pzqslagi4v7vj-gmp-6.2.0.drv
>
> /gnu/store/arxf2alzwf9rmz5hz8h11j4j12drxm3i-glibc-cross-powerpc64-linux-gnu-2.31.drv
>
> /gnu/store/d127w5flv12s4bfmpf4nwrvg3sibvfya-linux-libre-headers-cross-powerpc64-linux-gnu-5.4.20.drv
>/gnu/store/j3d5kr7qlr6g3lq0dwc8z8jh6w814z9v-isl-0.18.drv
>/gnu/store/j90wwahzd5ldw7ai11zf5lnp3kbbrmkh-mpfr-4.0.2.drv
>/gnu/store/mz9fdir4avdda5cw1snyf8vhpq70c9na-libelf-0.8.13.drv
>/gnu/store/q9x04y75mq2nfp2a6gwa0pvrgv60aah9-mpc-1.1.0.drv
>/gnu/store/xk4yv7xj15qnl3zv2m8nnzrw0bdgjsx3-zlib-1.2.11.drv
> 171.3 MB will be downloaded:
>/gnu/store/ir3092v7657h6g4g2vlsw3zrli3rndb3-zlib-1.2.11.tar.gz
>/gnu/store/amc0nizxsdcj212nk9a3ivr946hzhl6c-mpc-1.1.0
>/gnu/store/j4npmpn7dxmfknyfnhj4q4jmdwmk3klg-mpc-1.1.0.tar.gz
>/gnu/store/0z3z3lhig0xyy817nv70p2hp1n1wqawa-libelf-0.8.13.tar.gz
>/gnu/store/bkyiyc4hrjcd4ljx6jqf7z05hm4qxcwd-mpfr-4.0.2.tar.xz
>/gnu/store/2jj3il6p5xrc4gkncj9303an81x2csc9-perl-5.30.2
>/gnu/store/n1yvkd7jk50qg1vv9cca6ywynkqvaqgq-ncurses-6.2
>/gnu/store/j709qpwy790bcra6w8kvyz1v5zcsw8df-texinfo-6.7
>/gnu/store/jk5k0sgqpj0sj4ymgq7m8g8617i0xji2-m4-1.4.18
>/gnu/store/57i37x74wz7ar703smykildzvhpdds1g-gmp-6.2.0
>/gnu/store/f2r1w8y7l3lpwh4i47nq2s1vqlqxq0jb-glibc-2.31
>/gnu/store/rgi1k6kx4v9m8449w00i6jfxvpgaz73g-glibc-2.31-static
>/gnu/store/df1gdl0vwwbzv04snfha0g88rj02pni9-gcc-5.5.0
>/gnu/store/waz3iz17vlbpfc2fm9yiym6bgbsajghf-mpfr-4.0.2
>/gnu/store/hnsi8iaimgss3v81h7h1r8ck55c0968h-popt-1.18
>/gnu/store/vpy0bcjw0yzaj7j7qx8rfc88c7r357k3-rsync-3.1.3
>/gnu/store/0zcl1i3rbjc356138hx86b7yaz29g6fj-linux-libre-5.4.20-gnu.tar.xz
>/gnu/store/l788x07ska5vffayz0gayv4hsx5flxal-module-import-compiled
>/gnu/store/lqz1pygx3x5dd6ad2l3n8ixm1vh6czj4-python-minimal-3.8.2
>/gnu/store/ba6s3g925nggb57b1gpj2jkhqsq24s4q-libstdc++-7.5.0
>/gnu/store/xaclbfx6rvnbsq5qmry0251r7y82rgnv-libstdc++-headers-7.5.0
>/gnu/store/j8b9i4czpzb298zwa15wpyr42471qfbm-module-import-compiled
>
> The Guix documentation ((guix) Additional Build Options) says the "-d"
> option should just give me the derivation paths, not the output paths:
>
> ‘--derivations’
> ‘-d’
>  Return the derivation paths, not the output paths, of the given
>  packages.
>
> So, I'm confused about why all these builds need to happen.  Didn't I
> only ask Guix to instantiate the derivation - not realize it?  In other
> words, I expected Guix to calculate the transitive closure of the
> requested derivation's inputs (mainly other derivations, I think?) and
> write them to the store, without actually executing any significant
> builds.
>
> I suppose that Guix is building these things in order to do just that,
> but I don't quite understand why this happens.  Can someone explain it?

This can also happen even if I don't include --target:

[0] marusich@garuda-lan:~/guix/repos/guix-worktrees/wip-ppc64
$ guix build -d -e '(@@ (gnu packages make-bootstrap) %gcc-static)'
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivations will be built:
   /gnu/store/jx7frgnihxqn75sgxwb9md420dbrh6cg-gcc-static-5.5.0.drv
   /gnu/store/h3y11yg8g44ss2b02rlq7is4mxqd8qfs-isl-0.18.drv
   /gnu/store/wx5mmblp5p5q85g55ggpcpc2kh12dsbv-zlib-1.2.11.drv
   /gnu/store/xk149wji2hlx0ls404agj3is77kazxx6-libelf-0.8.13.drv
10.9 MB will be downloaded:
   /gnu/store/f2r1w8y7l3lpwh4i47nq2s1vqlqxq0jb-glibc-2.31
   /gnu/store/rgi1k6kx4v9m8449w00i6jfxvpgaz73g-glibc-2.31-static
   /gnu/store/b1940pfi34

guix build -d with a target causes many builds

[Chris Marusich]

Hi,

I've noticed that Guix builds many things when I ask it to instantiate a
derivation in the following way:

[0] [env] marusich@garuda-lan:~/guix/repos/guix-worktrees/wip-ppc64
$ guix build -d --target=powerpc64-linux-gnu -e '(@@ (gnu packages 
make-bootstrap) %gcc-static)'
The following derivations will be built:
   /gnu/store/i5wn3xl6p0zw1vglscgk0bs9dwc6hdh6-gcc-static-5.5.0.drv
   
/gnu/store/3h2sk37iim53fh7g9r3sd1q0xzhqwa51-gcc-cross-powerpc64-linux-gnu-7.5.0.drv
   
/gnu/store/84k0j5jm316cwf7h66vrw1vmvkd4kbck-glibc-cross-powerpc64-linux-gnu-2.31.drv
   
/gnu/store/d36n7qy9xbgwpaw3nw8k9dj51hzmdnr4-gcc-cross-sans-libc-powerpc64-linux-gnu-7.5.0.drv
   
/gnu/store/mqar9bnapfcfkna3rvy28awhlpd3q65q-binutils-cross-powerpc64-linux-gnu-2.34.drv
   
/gnu/store/pzp93dw3rr6sp2ybi3dzs6kd7gvigfsk-ld-wrapper-powerpc64-linux-gnu-0.drv
   
/gnu/store/n7dhpsq41q4kdbqgniljbwrlawvmmlp6-linux-libre-headers-cross-powerpc64-linux-gnu-5.4.20.drv
   
/gnu/store/9p5anrji5wgkf66k09jhbsr3fqwwi7cn-gcc-cross-powerpc64-linux-gnu-7.5.0.drv
   
/gnu/store/r4ac80znwlrnh4jmj2sbczc4mn66mqdg-glibc-cross-powerpc64-linux-gnu-2.31.drv
   /gnu/store/ap8ri9ddka13vyrsl72pzqslagi4v7vj-gmp-6.2.0.drv
   
/gnu/store/arxf2alzwf9rmz5hz8h11j4j12drxm3i-glibc-cross-powerpc64-linux-gnu-2.31.drv
   
/gnu/store/d127w5flv12s4bfmpf4nwrvg3sibvfya-linux-libre-headers-cross-powerpc64-linux-gnu-5.4.20.drv
   /gnu/store/j3d5kr7qlr6g3lq0dwc8z8jh6w814z9v-isl-0.18.drv
   /gnu/store/j90wwahzd5ldw7ai11zf5lnp3kbbrmkh-mpfr-4.0.2.drv
   /gnu/store/mz9fdir4avdda5cw1snyf8vhpq70c9na-libelf-0.8.13.drv
   /gnu/store/q9x04y75mq2nfp2a6gwa0pvrgv60aah9-mpc-1.1.0.drv
   /gnu/store/xk4yv7xj15qnl3zv2m8nnzrw0bdgjsx3-zlib-1.2.11.drv
171.3 MB will be downloaded:
   /gnu/store/ir3092v7657h6g4g2vlsw3zrli3rndb3-zlib-1.2.11.tar.gz
   /gnu/store/amc0nizxsdcj212nk9a3ivr946hzhl6c-mpc-1.1.0
   /gnu/store/j4npmpn7dxmfknyfnhj4q4jmdwmk3klg-mpc-1.1.0.tar.gz
   /gnu/store/0z3z3lhig0xyy817nv70p2hp1n1wqawa-libelf-0.8.13.tar.gz
   /gnu/store/bkyiyc4hrjcd4ljx6jqf7z05hm4qxcwd-mpfr-4.0.2.tar.xz
   /gnu/store/2jj3il6p5xrc4gkncj9303an81x2csc9-perl-5.30.2
   /gnu/store/n1yvkd7jk50qg1vv9cca6ywynkqvaqgq-ncurses-6.2
   /gnu/store/j709qpwy790bcra6w8kvyz1v5zcsw8df-texinfo-6.7
   /gnu/store/jk5k0sgqpj0sj4ymgq7m8g8617i0xji2-m4-1.4.18
   /gnu/store/57i37x74wz7ar703smykildzvhpdds1g-gmp-6.2.0
   /gnu/store/f2r1w8y7l3lpwh4i47nq2s1vqlqxq0jb-glibc-2.31
   /gnu/store/rgi1k6kx4v9m8449w00i6jfxvpgaz73g-glibc-2.31-static
   /gnu/store/df1gdl0vwwbzv04snfha0g88rj02pni9-gcc-5.5.0
   /gnu/store/waz3iz17vlbpfc2fm9yiym6bgbsajghf-mpfr-4.0.2
   /gnu/store/hnsi8iaimgss3v81h7h1r8ck55c0968h-popt-1.18
   /gnu/store/vpy0bcjw0yzaj7j7qx8rfc88c7r357k3-rsync-3.1.3
   /gnu/store/0zcl1i3rbjc356138hx86b7yaz29g6fj-linux-libre-5.4.20-gnu.tar.xz
   /gnu/store/l788x07ska5vffayz0gayv4hsx5flxal-module-import-compiled
   /gnu/store/lqz1pygx3x5dd6ad2l3n8ixm1vh6czj4-python-minimal-3.8.2
   /gnu/store/ba6s3g925nggb57b1gpj2jkhqsq24s4q-libstdc++-7.5.0
   /gnu/store/xaclbfx6rvnbsq5qmry0251r7y82rgnv-libstdc++-headers-7.5.0
   /gnu/store/j8b9i4czpzb298zwa15wpyr42471qfbm-module-import-compiled

The Guix documentation ((guix) Additional Build Options) says the "-d"
option should just give me the derivation paths, not the output paths:

‘--derivations’
‘-d’
 Return the derivation paths, not the output paths, of the given
 packages.

So, I'm confused about why all these builds need to happen.  Didn't I
only ask Guix to instantiate the derivation - not realize it?  In other
words, I expected Guix to calculate the transitive closure of the
requested derivation's inputs (mainly other derivations, I think?) and
write them to the store, without actually executing any significant
builds.

I suppose that Guix is building these things in order to do just that,
but I don't quite understand why this happens.  Can someone explain it?

-- 
Chris


signature.asc
Description: PGP signature

Re: [bug#45252] [PATCH] gnu: libffi: Add unreleased patch to fix float128 on powerpc64le., [bug#45252] [PATCH] gnu: libffi: Add unreleased patch to fix float128 on powerpc64le., [bug#45252] [PATCH] gn

[Chris Marusich]

Hi,

Efraim Flashner  writes:

> "regular powerpc", ie macppc/ppc32/powerpc-linux-gnu, does have some
> bootstrap binaries built but isn't near ready for merging. Go ahead and
> make any changes necessary.

Mark H Weaver  writes:

> Hi Efraim,
>
> Efraim Flashner  writes:
>> "regular powerpc", ie macppc/ppc32/powerpc-linux-gnu, does have some
>> bootstrap binaries built but isn't near ready for merging. Go ahead and
>> make any changes necessary.
>
> I appreciate that, but if rebuilding the world on regular powerpc would
> significantly add to the burden of even a single developer, then it's
> probably not worth it.  I suggested fixing the powerpc64le case now only
> because it was just added a few days ago, and more generally to raise
> awareness about how best to run the 'patch' program in Guix.
>
> If it's truly no extra burden, then you could change "--batch" to
> "--force" on line 69 of libffi.c (in the "powerpc-*" case).

OK.  I've made this change on master in commit
662e7e28d576ada91fc9dec7d27c100666114f03.

Mark H Weaver  writes:

> Hi Chris,
>
> Chris Marusich  writes:
>
>> Mark H Weaver  writes:
>>
>>> Earlier, I wrote:
>>>> When invoking 'patch' in Guix, you should *always* use "--force" instead
>>>> of "--batch".
>>>
>>> (See <https://bugs.gnu.org/45252#19> for my earlier message).
>>
>> Thank you for letting me know about this.  I didn't know about the
>> difference between "--batch" and "--force".  I agree we should use
>> "--force" instead of "--batch".  How do you recommend that I proceed?
>
> Simply changing "--batch" to "--force" on line 79 (in the powerpc64le-*
> case, i.e. the one that was just added) seems like the right thing.
> That will force a rebuild of almost everything on the powerpc64le-*
> architecture, but should not cause any rebuilds on other architectures.

OK, I've made this change on master in commit
fdb90e9ee8a578c88ef3a33067e8a532e43ae7b8.

>>> Since writing the message above, I've found another problem in the same
>>> commit (7eaa2f24ea77cddbb4bbc2d6a6905673a36f8f99): it searches for the
>>> 'patch' program in 'inputs'.  This is a mistake, because when
>>> cross-compiling, 'inputs' will contain software compiled to run on the
>>> target system instead of the build system.
>>
>> Is it searching for the "patch" program, or is it searching for the
>> patch file?  It looks to me like the code is searching for the patch
>> file in inputs, not the "patch" program.
>
> LOL, you're right, I got confused.  Please disregard my second email in
> this thread, and apologies for that noise.

No worries!  Thanks again for your help.

-- 
Chris


signature.asc
Description: PGP signature

Re: [PATCH] gnu: libffi: Add unreleased patch to fix float128 on powerpc64le.

[Chris Marusich]

Hi Mark,

Mark H Weaver  writes:

> Earlier, I wrote:
>> When invoking 'patch' in Guix, you should *always* use "--force" instead
>> of "--batch".
>
> (See  for my earlier message).

Thank you for letting me know about this.  I didn't know about the
difference between "--batch" and "--force".  I agree we should use
"--force" instead of "--batch".  How do you recommend that I proceed?

I can definitely make another commit on the master branch to change the
option from "--batch" to "--force".  However, I'm reluctant to change
the option in the existing code on the master branch (introduced in
commit 02f5ee01c96589fc13f1e21b85b0b48100aec4e8), since I'm not sure how
many packages would be rebuilt as a result.  The powerpc64le
architecture is not bootstrapped at all yet, so it's not a problem for
that architecture, but I don't know the status for all the other
architectures beginning with "powerpc".

Before I make a new commit to change the patch option on the master
branch, I'd appreciate your advice on how to proceed.  Do you think it
would be better to make a commit on the master branch to fix just the
option I introduced in commit 7eaa2f24ea77cddbb4bbc2d6a6905673a36f8f99?
Or, do you think it would be better to make a commit on the core-updates
branch to change all the "--batch" options to "--force" options in the
libffi package definition?  If we did it on core-updates, we could just
replace the manual invocation of the "patch" tool with a change that
looks more like 4fff5ab24126a152b50c036b9bf8dc6f2740f094, in particular
this part:

diff --git a/gnu/packages/libffi.scm b/gnu/packages/libffi.scm
index 0e6a31d78c..0db8fa3e82 100644
--- a/gnu/packages/libffi.scm
+++ b/gnu/packages/libffi.scm
@@ -51,7 +51,8 @@
   (sha256
(base32
 "0mi0cpf8aa40ljjmzxb7im6dbj45bb0kllcd09xgmp834y9agyvj"))
-  (patches (search-patches "libffi-3.3-powerpc-fixes.patch"
+  (patches (search-patches "libffi-3.3-powerpc-fixes.patch"
+   "libffi-float128-powerpc64le.patch"
 (build-system gnu-build-system)
 (arguments
  `(;; Prevent the build system from passing -march and -mtune to the

I thought we wanted to apply patch 45252 on the master branch.  That's
why I made commit 7eaa2f24ea77cddbb4bbc2d6a6905673a36f8f99.  Afterwords,
I actually reverted commit 4fff5ab24126a152b50c036b9bf8dc6f2740f094 on
the core-updates branch in commit
b50341dba9811c048bed852c0279b828c7ddba66.  I reverted it because I
thought it would be undesirable to solve the same problem in two
different ways on two separate branches, and I thought that reverting it
would reduce the risk of merge conflicts later.

However, now that I think about it, I'm not sure the reversion was
necessary.  I'm actually not sure what the normal procedure is for
merging to/from core-updates and master (I've done many merges in my own
projects, but I've never done a merge in the Guix project), so I'm not
sure how a "TODO" task like the one mentioned in commit
7eaa2f24ea77cddbb4bbc2d6a6905673a36f8f99 ("Inline patches on next
rebuild cycle") would normally be resolved.  I would welcome any advice
you have about that.

By the way, a wip-ppc64le branch also exists, but I don't know what its
status is or whether I'm allowed to touch it.  I just assumed things
would be simpler if we applied patches to master branch when possible.

> Since writing the message above, I've found another problem in the same
> commit (7eaa2f24ea77cddbb4bbc2d6a6905673a36f8f99): it searches for the
> 'patch' program in 'inputs'.  This is a mistake, because when
> cross-compiling, 'inputs' will contain software compiled to run on the
> target system instead of the build system.

Is it searching for the "patch" program, or is it searching for the
patch file?  It looks to me like the code is searching for the patch
file in inputs, not the "patch" program.  The relevant code is here:

   ,@(if (string-prefix? "powerpc64le-" (or (%current-target-system)
(%current-system)))
 '(#:phases (modify-phases %standard-phases
  (add-after 'unpack 'apply-patch2
(lambda* (#:key inputs #:allow-other-keys)
  (let ((patch (assoc-ref inputs
  "powerpc64le-patch")))
(invoke "patch" "--batch" "-p1"
"-i" patch))

The code invokes the "patch" program in the usual way.  My understanding
is that whatever version of the "patch" program that Guix has placed in
the PATH environment variable will be used.  Therefore, Guix will use
the correct "patch" program, regardless of whether or not the package is
being cross-compiled.  Am I misunderstanding something?

Again, thank you for taking the time to bring these topics up.  I'm

Re: Makefile logic to create Guix documentation

[Chris Marusich]

Hi Julien,

Julien Lepiller  writes:

> Not sure for the rest, but .dot.eps: is similar to:
>
> %.eps: %.dot
>
> You often find .c.o in Makefiles for instance.

Ohhh, I see!  With this info, I was able to determine that the relevant
information is documented, but I just missed it (see: (make) Suffix
Rules).  Thank you very much for pointing it out to me!

I'm still not sure about the rest, though...  Maybe Ludo knows.

-- 
Chris


signature.asc
Description: PGP signature

Helping with powerpc64-linux

[Chris Marusich]

Hi Léo,

As you know, I'm still investigating why the powerpc64-linux bootstrap
binaries (really just GCC) are not reproducible [1].  Eventually, if we
can't figure it out, we might want to just bite the bullet and use the
non-reproducible bootstrap binaries to get started, but I'm hopeful we
can figure it out and have at least two different people build the same
binaries before doing that.

The investigation is slow, partially because building things without
substitutes is very slow (iteration time is in hours or days).  In the
meantime, I understand you've been continuing to work on building more
powerpc64-linux packages on top of your own bootstrap binaries [2].  I
also understand you've had some trouble setting up Cuirass on your
POWER9 PC.  I'd like to help (and invite others to help, if they want),
but I'm not sure what the current status is.

Is Cuirass still not working?  I checked your POWER9 Gentoo VM, and I
saw that Cuirass is not currently running.

Which branches in your Git repository are still relevant?  My
understanding is that only the following branches in your repo might
currently contain changes we would want to upstream into Guix:

  - master: the first attempts you made.
  - wip-lle-bout-be1: your more recent attempts.

Do you think we should we try adding powerpc64le-linux?  Since the
powerpc64-linux bootstrap binaries are not reproducible, I see no reason
to expect that the powerpc64le-linux binaries would be reproducible,
either, but you never know.  Maybe it's worth a try.

Is there something else I can help you with?  I will have time while
waiting for long builds to finish, but please forgive me if my progress
is slow.  Life is busy.  Slowly but surely, we can make progress, and
hopefully we can keep collaborating to make it happen!

Footnotes: 
[1]  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669

[2]  https://gitlab.com/lle-bout/guix

-- 
Chris


signature.asc
Description: PGP signature

Makefile logic to create Guix documentation

[Chris Marusich]

Hi Ludo,

I was reading through the Makefile that creates the documentation, and I
came across some parts I couldn't understand, even though I spent a few
hours trying to figure it out.  I thought you might know the answers to
my questions off the top of your head, since I think you wrote it.

First, consider this snippet from doc/local.mk:

--8<---cut here---start->8---
# We cannot add new dependencies to `%D%/guix.pdf' & co. (info "(automake)
# Extending").  Using the `-local' rules is imperfect, because they may be
# triggered after the main rule.  Oh, well.
pdf-local: $(DOT_FILES=%.dot=$(top_srcdir)/%.pdf)
info-local: $(DOT_FILES=%.dot=$(top_srcdir)/%.png)
ps-local: $(DOT_FILES=%.dot=$(top_srcdir)/%.eps)\
  $(top_srcdir)/%D%/images/coreutils-size-map.eps
dvi-local: ps-local
--8<---cut here---end--->8---

What is this syntax called?  I checked the Make, Automake, and Autoconf
manuals, but I couldn't find anything.  I'm talking about this syntax:

  info-local: $(DOT_FILES=%.dot=$(top_srcdir)/%.png)

It looks like when you added this, you intended to add a *.png
prerequisite to the info-local target for every equivalent *.dot file
that exists.  My guess is you want to ensure that the PNG files are
generated before the info page gets created, since the PNG files are
required in order to build the info page.  However, is it possible you
meant to write it like the following instead?  (The first "=" has been
replaced with a ":".)

  info-local: $(DOT_FILES:%.dot=$(top_srcdir)/%.png)
 
When using ":", I recognize this syntax as a "substitution reference"
(see: (make) Substitution Refs).  However, I do not know what it is
supposed to be when the ":" is replaced with a "=".  Is it a typo?

I experimented by adding a snippet like the following to the generated
Makefile...

--8<---cut here---start->8---
marucustom: $(DOT_FILES=%.dot=$(top_srcdir)/%.eps)
@echo X expansion is: _$(DOT_FILES=%.dot=$(top_srcdir)/%.eps)_
@echo X prerequisite: $<
@echo X target: $@
--8<---cut here---end--->8---

...and it seems to cause make to substitute an empty string:

--8<---cut here---start->8---
$ make marucustom
X expansion is: __
X prerequisite:
X target: marucustom
--8<---cut here---end--->8---

Second, I noticed some rules like the following:

--8<---cut here---start->8---
.dot.eps:
$(AM_V_DOT)$(DOT) -Teps $(DOT_OPTIONS) < "$<" > "$(srcdir)/$@.tmp"; \
mv "$(srcdir)/$@.tmp" "$(srcdir)/$@"
--8<---cut here---end--->8---

What I do understand is that AM_V_DOT and DOT are set in Makefile.am and
configure.ac, and that they are used to invoke the "dot" program.
However, I couldn't quite understand the rest of this rule.

Why are there no prerequisites?  It looks like the rule doesn't declare
any prerequisites, so I'm confused about why the recipe includes
references to the name of the first prerequisite ("$<").

What causes this rule to be run?  I tried adding echo commands in the
recipe like so...

--8<---cut here---start->8---
.dot.eps:
@echo Y .dot.eps is running
@echo Y target: %@
@echo Y prerequisite: _$<_
$(AM_V_DOT)$(DOT) -Teps $(DOT_OPTIONS) < "$<" > "$(srcdir)/$@.tmp"; \
mv "$(srcdir)/$@.tmp" "$(srcdir)/$@"
--8<---cut here---end--->8---

...but I didn't see new messages when I ran "make" (from a clean
checkout).  Maybe I didn't invoke make correctly.

If you or anyone else could point out if I'm missing something, that'd
be really helpful.  Thank you in advance!

-- 
Chris


signature.asc
Description: PGP signature

Re: Request to verify powerpc64-linux bootstrap binaries

[Chris Marusich]

Hi Vincent, Jack, and Maxim,

Thank you for the quick replies!  OK, so gcc differs for each of us:

* Chris: 
8aca7f332a1ba8e3c2225c161a7545b0a04ddd690d164dc97afee9c9ea067b0c49bc155e9f06d285c22e24cdd16d91e59730af5f1dd9efcda13a26bede5948a2
* Vincent: 
87f7583cf483ac3ba0ab978862873e68757bc4ddd10f739a90a9e4598f79e7fa45ec369c6efcff8d72fba87ea99f1e7a01a39450c7bf20790bc1d89d4b69a15b
* Jack: 
15f93200ef1cdde5a5721b1d4cfb4c9c5e22b4b945c77f56f60c388da99cf557d13474d14205a529cf9af9f29fb69591e17392387c0444e0efb7c85edcf30ff0
* Maxim: 
841f1839c041512f893d5fa62fbc402dda1589222ee5365849d2e3f0a55df7abd0ca856c302b4d7ef80d07abfa6b04468cc494efdccbc097df2eceb181eebd15

I have opened up a bug report for this issue, so we can continue the
discussion there:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669

I will try on my end to obtain a differing gcc to see if I can analyze
the difference.  In the meantime, could you share your gcc with me so I
can see what the contents looks like?  This is mine:

https://media.marusich.info/gcc-stripped-5.5.0-powerpc64-linux-gnu.tar.xz

Hopefully we can pinpoint the source of the non-determinism and fix it!

-- 
Chris


signature.asc
Description: PGP signature

Request to verify powerpc64-linux bootstrap binaries

[Chris Marusich]

Hi everyone!

Thanks to Léo's help, as of commit
8159ce1970d91567468cf1bacac313099a009d2a, the master branch now contains
all the changes necessary to cross-compile powerpc64-linux bootstrap
binaries.  I've done this without substitutes by running the following
commands on an x86_64-liinux machine.

First, to ensure you're using commit
8159ce1970d91567468cf1bacac313099a009d2a, put something like this in
your ~/.config/guix/channels.scm file:

(list (channel
(name 'guix)
(url "https://git.savannah.gnu.org/git/guix.git;)
(commit
  "8159ce1970d91567468cf1bacac313099a009d2a")))

Then, run these commands:

# Confirm you're using the right guix.
guix describe
# Clear as many GC roots as possible, and do a GC run.
guix gc --delete-generations
# Build the bootstrap tarballs without substitutes.
guix build --no-substitutes --target=powerpc64-linux-gnu bootstrap-tarballs

After a few hours, you should see the following message:

successfully built 
/gnu/store/icnj0m294b94pc3rhpmkz6zc41w8vyqj-bootstrap-tarballs-0.drv
/gnu/store/dy4fhy2vqyxqfad59mfdqdd0qpcrwmyx-bootstrap-tarballs-0

On my end, the SHA-512 hashes of the binaries are:

--8<---cut here---start->8---
426e5f1d0d7023a90e73286ccda1fa55a359301e998a19dfe00f5b4f5d387e69d7a247f47056f41e609393893b0238a908698fbd28d73b183b32a5dadcfe9fbb
  binutils-static-stripped-2.34-powerpc64-linux-gnu.tar.xz
8aca7f332a1ba8e3c2225c161a7545b0a04ddd690d164dc97afee9c9ea067b0c49bc155e9f06d285c22e24cdd16d91e59730af5f1dd9efcda13a26bede5948a2
  gcc-stripped-5.5.0-powerpc64-linux-gnu.tar.xz
a717a420e765accf12cfc1e18ebed76e9359ee58e8781601ca9066ced59196f88a528ddc554c0f57c77e2c01908cafe596f3c8d1df135beb4cae4073b9a999d2
  glibc-stripped-2.31-powerpc64-linux-gnu.tar.xz
e2e70c7fcc477fced12eb76704212f9bda0e1ec2cf40137ff6a32a85ca75fec10ec20076b73698438e48c3ce45d24542aa309bb99274f4c3d4f9d49ec9d1dd7b
  guile-static-stripped-2.0.14-powerpc64-linux-gnu.tar.xz
04d9203467ecb48e9f1fca5130199c292212d4d119153778d398899aeef517fc8bce5d25f3505063f38e433fa09e3c723a6da5dee4943dbc9d3728279356879b
  static-binaries-0-powerpc64-linux-gnu.tar.xz
--8<---cut here---end--->8---

Hopefully, you'll get identical results!  You don't have to run "guix
gc" if you don't want to, but doing so will increase the likelihood of
catching nondeterminism issues propagated from dependencies (which seem
unlikely, but you never know).  It took 3 or 4 for me hours on a modern
16-core machine.

Once we verify the binaries, we can actually start using them to build
stuff!  Léo has already gotten an optimistic start on that work, and
many things are building successfully.  Exciting!!

-- 
Chris


signature.asc
Description: PGP signature

Re: G-Expressions and Scope Preservation

[Chris Marusich]

Hi Ludo,

Ludovic Courtès  writes:

> Hello!
>
> Chris Marusich  skribis:
>
>> In your paper "Code Staging in GNU Guix" [1], you use the following
>> example to illustrate how G-Expressions are hygienic ("they preserve
>> lexical scope across stages"):
>>
>> (let ((gen-body (lambda (x)
>>   #~(let ((x 40))
>>   (+ x #$x)
>>   #~(let ((x 2))
>>   #$(gen-body #~x)))
>>
>> You explain that it expands to something like this:
>>
>> (let ((x-1bd8-0 2))
>>   (let ((x-4f05-0 40))
>> (+ x-4f05-0 x-1bd8-0)))
>>
>> However, when I write this gexp to disk, it doesn't look like that:
>
> Ah ha!  That bit is still in the ‘wip-gexp-hygiene’ branch.
>
> The reason I haven’t merged it, other than I didn’t take the time, is
> that the output depends on Guile’s ‘hash’ function, which is not
> necessarily stable.  Actually, it changed between 2.0 and 2.2, but I
> think it’s the same in 2.2 and 3.0.  This needs to be checked, because
> if it differs, then people will get different results depending on the
> Guile version they use, and that’d be a serious issue.
>
> I thought I had mentioned this before, but apparently not:
>
>   https://lists.gnu.org/archive/html/guix-devel/2017-07/msg00181.html
>   https://lists.gnu.org/archive/html/guix-devel/2017-09/msg00093.html
>
> We should also do some more testing to make sure nothing breaks.
>
> Ludo’.

Ah!  I see.  That explains it.  Thank you for pointing this out.  I'm
still finalizing my presentation, but if I find time after that, I might
play around with that branch.

-- 
Chris


signature.asc
Description: PGP signature

Re: G-Expressions and Scope Preservation

[Chris Marusich]

zimoun  writes:

> the Unison language

Thank you for sharing the link!  I only read the overview, but I can see
why this makes you think of G-Expressions.  It's always interesting to
read about how content addressability can make some problems easier.

> I cannot wait for your presentation at FOSDEM. :-)

I look forward to seeing you there!

-- 
Chris


signature.asc
Description: PGP signature

G-Expressions and Scope Preservation

[Chris Marusich]

Hi Ludo,

In your paper "Code Staging in GNU Guix" [1], you use the following
example to illustrate how G-Expressions are hygienic ("they preserve
lexical scope across stages"):

(let ((gen-body (lambda (x)
  #~(let ((x 40))
  (+ x #$x)
  #~(let ((x 2))
  #$(gen-body #~x)))

You explain that it expands to something like this:

(let ((x-1bd8-0 2))
  (let ((x-4f05-0 40))
(+ x-4f05-0 x-1bd8-0)))

However, when I write this gexp to disk, it doesn't look like that:

scheme@(guile-user)> ,use (guix)
scheme@(guile-user)> (define ex
   (let ((gen-body (lambda (x)
 #~(let ((x 40))
 (+ x #$x)
 #~(let ((x 2))
 #$(gen-body #~x
scheme@(guile-user)> ex
$2 = #:out>)) 7f29a4ed9240>:out>) 7f29a4ed91e0>
scheme@(guile-user)> ,run-in-store (lower-object (scheme-file "example" ex))
$3 = # 
/gnu/store/mhlvarq8hc9c40by7sfq7yqvxvjdq7rp-example 7f299a13abe0>
scheme@(guile-user)> ,run-in-store (built-derivations (list $3))
building path(s) `/gnu/store/mhlvarq8hc9c40by7sfq7yqvxvjdq7rp-example'
$4 = #t

The file /gnu/store/mhlvarq8hc9c40by7sfq7yqvxvjdq7rp-example contains
the following expression:

(let ((x 2)) (let ((x 40)) (+ x x)))

This looks different than what I expected.  I expected something more
like what you had written in the paper.  Am I missing something?

I thought this would be an easy way to see the scope preservation in
action, but I get the feeling I've misunderstood something.  I tried
some other ways to use the gexp, and the results were similar:

scheme@(guile-user)> (define write-result
   #~(with-output-to-file #$output
   (lambda ()
 (write #$ex
scheme@(guile-user)> (define write-ex-literally
   #~(with-output-to-file #$output
   (lambda ()
 (write '#$ex
scheme@(guile-user)> ,run-in-store (gexp->derivation "write-result" 
write-result)
$7 = # /gnu/store/jaf44b767y5n2m0zd6q9qswhzv2hsy96-write-result 7fda1342e960>
scheme@(guile-user)> ,run-in-store (gexp->derivation "write-ex-literally" 
write-ex-literally)
$8 = # 
/gnu/store/rhhm38j6yxfs0q7jrvdrv02r7zb9ai8j-write-ex-literally 7fda1342e780>
scheme@(guile-user)> 

The file /gnu/store/jaf44b767y5n2m0zd6q9qswhzv2hsy96-write-result
contained "80", and the file
/gnu/store/rhhm38j6yxfs0q7jrvdrv02r7zb9ai8j-write-ex-literally contained
the following S-Expression:

(let ((x 2)) (let ((x 40)) (+ x x)))

Can you help me to understand why I'm seeing this result instead of a
result that looks more like the one you presented in your paper?

Thank you,

Footnotes: 
[1]  https://hal.inria.fr/hal-01580582

-- 
Chris


signature.asc
Description: PGP signature

Re: Documenting Yubikey setup

[Chris Marusich]

Hi Martin,

Martin Becze  writes:

> For a user to access a Yubikey an udev rule needs to be added. This can
> be done by using the udev rules found in libu2f-host package. To use the
> rules the following should be added to your config.scm
>
> (use-modules (gnu packages security-token))
>
> ...
>
> (define %modified-desktop-services
>   (modify-services %minimal-desktop-services
> (udev-service-type
>  config =>
>  (udev-configuration (inherit config)
>  (rules (cons libu2f-host
>   (udev-configuration-rules
> config)))
>
>
>
> Reference 
> https://guix.gnu.org/manual/en/html_node/Base-Services.html
>
> ---
>
> Of course there is more to say about yubikey but this is all I needed
> for now. 

Did you write something for the cookbook?  I've also set up a YubiKey
using Guix - I actually packaged some of the things like libu2f-host -
and would be happy to review / add to whatever you've written.

-- 
Chris


signature.asc
Description: PGP signature

Re: Overhauling the cargo-build-system

[Chris Marusich]

Hi,

Ludovic Courtès  writes:

> What I would have liked is to somehow replace the #:cargo-inputs
> argument (which is build-system-specific and thus “opaque”) with regular
> ‘native-inputs’ or ‘inputs’ field.

That would be nice.  However, it doesn't seem possible to express
Cargo's "dependencies" and "dev-dependencies" concepts using Guix's
current package DSL.

Consider the proc-macro2 and quote crates.  We added these two crates in
commit 2444abd9c124cc55f8f19a0462e06a2094f25a9d, in the same patch
series where we added #:cargo-inputs and #:cargo-development-inputs:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35318

Here is the Cargo.toml file for proc-macro2:

  https://github.com/alexcrichton/proc-macro2/blob/master/Cargo.toml

  [dev-dependencies]
  quote = { version = "1.0", default_features = false }

And here is the Cargo.toml file for quote:

  https://github.com/dtolnay/quote/blob/master/Cargo.toml

  [dependencies]
  proc-macro2 = { version = "1.0", default-features = false }

Here is a diagram of their dependency relationship:

  +---+
  | quote | <+
  +---+  |
||
| dependencies   | dev-dependencies
v|
  +---+  |
  |  proc-macro2  | -+
  +---+

To Cargo, this cycle is not a problem, since "dev-dependencies" are
treated differently from "dependencies":

  https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html

  "Dev-dependencies are not used when compiling a package for building,
  but are used for compiling tests, examples, and benchmarks.

  These dependencies are not propagated to other packages which depend
  on this package."

The reason proc-macro2 declares a "dev-dependency" on quote is because
proc-macro2 uses quote in its doc tests:

  
https://github.com/alexcrichton/proc-macro2/blob/e82e8571460a0a0e00f52f011a74a5e0359acf3e/src/lib.rs#L785

This relationship between proc-macro2 and quote cannot be readily
expressed using the current package DSL in Guix.  If you try to model
"dependencies" and "dev-dependencies" as "inputs" (or "native-inputs",
or some combination of the two), Guix will fail due to the cycle.

Presumably, proc-macro2 just needs the source of quote (and the source
of proc-macro2's other dependency, unicode-xid).  When Cargo builds
proc-macro2, it will take care of building quote and making it available
during proc-macro2's tests.  Guix "just" needs to provide proc-macro2
with the quote source.  You might think this poses a bootstrapping
problem for Cargo, but I guess it doesn't.  As long as Cargo has the
source for proc-macro2, quote, and unicode-xid, I guess it can build
proc-macro2 and quote in any order.

Unless we missed something in our discussion of patch 35318, there is no
easy way to express the relationship between proc-macro2 and quote
without changing (or mis-using) the existing package DSL.  In the same
way that the package DSL introduced "native-inputs" and "inputs" as
concepts to facilitate cross-compilation, one way to solve this problem
might be to introduce a new concept to the package DSL that would make
it possible for Guix to express this kind of relationship correctly.

However, in the discussion of patch 35318, everyone (myself included)
seemed opposed to changing the package DSL if we didn't have to.  For
example, in response to an earlier version of the patch series in which
we tried to map "dependencies" and "dev-dependencies" onto the "inputs"
and "native-inputs" concepts (which was probably an abuse of the package
DSL, since "native-inputs" is a cross-compilation concept), you said: "I
don't understand yet why you change the role of 'inputs' compared to how
it is in the rest of Guix."  Ultimately, we decided not to modify the
package DSL or the meaning of "inputs".  Instead, we decided to encode
the necessary information about dependencies in the cargo-build-system's
arguments.  That is how we arrived at #:cargo-inputs and
#:cargo-development-inputs.

By introducing #:cargo-inputs and #:cargo-development-inputs as package
arguments to the cargo-build-system, we were able to solve the cyclic
dependency problem in one specific way.  Perhaps there are better ways.
I agree it would be nice if it were integrated into the package DSL.  I
think that changing the package DSL to suit our needs might work, but
I'm not sure how to change it without making it too Cargo-specific.

> I know it’s not that easy with Rust and Cargo, I just never manage to
> fully grasp why :-), but at least that should be our horizon IMO.

Well, you're not alone!  I'm not (yet!) an expert in Rust, but I find
these problems difficult to understand, too.  Cyclic dependencies are
just one issue.  There are other problems, too.

One problem that Efraim has mentioned is that every crate wants all of
its sources to be available at build time.  It's as if each crate wants
all of its source crates (and all of their source crates, transitively)
to be propagated 

Re: New POWER9 machines for the Guix build farm?

[Chris Marusich]

Tobias Geerinckx-Rice  writes:

> Fellow Guix,
>
> The Guix sysadmins are considering buying shiny hardware for the
> ci.guix.gnu.org build farm, and it would be awesome if that included
> our first POWER9 machine(s)!
>
> However, few (if any) Guixers have any hands-on experience with this
> architecture, let alone buying and installing whole systems. Some
> remember a bad experience with a prominent vendor, and it would appear
> that they're not alone[0].
>
> There's also some concern that getting these machines up and running
> will take significant effort.
>
> So please, share your expertise and experience in this area! Ideally,
> we need someone to volunteer to (help) set up any new POWER9 boxes and
> later take care of them when needed.  It would certainly help justify
> the multi-thousand-euro bill.
>
> Kind regards,
>
> T G-R
>
> PS: For the shorter term, I've applied for an 8-core POWER9 LE
> instance (with 16 GiB of RAM) for Guix at OSUOSL[1].  Assuming that
> it's accepted, it should be available within a week.
>
> [0]:
> https://drewdevault.com/2019/09/23/RaptorCS-Blackbird-a-horror-story.html,
> but read https://drewdevault.com/2019/10/10/RaptorCS-redemption.html
> as well :-)
> [1]: https://osuosl.org/services/powerdev/

I think it's a great idea.  To test the waters, someone could try using
one of these free VMs and see how Guix System does on them:

https://openpowerfoundation.org/minicloud-free-openpower-cloud/

First, bootstrap binaries have to be built for the hardware, anyway,
right?  Maybe someone can do that work on one of those free VMs?

-- 
Chris


signature.asc
Description: PGP signature

Re: guix pull failed after 8 hours

[Chris Marusich]

Konrad Hinsen  writes:

> Hi Ludo and Chris,
>
>> Could it be that those old systems were talking to hydra.gnu.org and
>> thus not getting any substitutes?
>>
>>   https://lists.gnu.org/archive/html/info-guix/2019-06/msg1.html
>
> That's what happened to me on my oldest Guix installation (hosted on
> Ubuntu). I had done regular pulls on my user account, but not on the
> root account, meaning that I was running an ancient daemon that tried to
> connect to hydra.

Yes, it's possible this was the cause.  Thank you to everyone who
replied!  In the end, I was able to run "guix pull".

-- 
Chris


signature.asc
Description: PGP signature

Re: guix pull failed after 8 hours

[Chris Marusich]

Hi Giovanni and everyone,

Giovanni Biscuolo  writes:

> Chris Marusich  writes:
>
> [...]
>
>> Has anyone run "guix pull" successfully recently?  I have been trying
>> for days without success.
>
> I was also able to run guix pull on a foreign system a few hours ago and
> on a Guix System a few minutes ago
>
> do you still have problems?

After a few more tries (each of which took multiple hours to fail), it
finally succeeded.  I guess I just got very unlucky.

I've been updating some systems with a fairly old version of Guix, so
perhaps that contributed to the very long "guix pull" times.  It felt
like Guix wanted to compile the whole world just to run "guix pull".

Anyway, it's succeeded, so please disregard my original message.  Thank
you for all your responses!


-- 
Chris


signature.asc
Description: PGP signature

guix pull failed after 8 hours

[Chris Marusich]

Hi,

After 8 hours of waiting, "guix pull" failed with this message:

srfi/srfi-1.scm:592:17: In procedure map1:
Throw to key `srfi-34' with args `(#)'.
guix pull: error: You found a bug: the program 
'/gnu/store/cgwq319bn468wr2vrnfih4l7k9wdiqi8-compute-guix-derivation'
failed to compute the derivation for Guix (version: 
"1051facc8108110e582f2c0a9d8a5fd38b1486b6"; system: "x86_64-linux";
host version: "9ffbc2456babcc8e61e5291bef1f04037145bd95"; pull-version: 1).
Please report it by email to .

Has anyone run "guix pull" successfully recently?  I have been trying
for days without success.

-- 
Chris


signature.asc
Description: PGP signature

Re: Help needed packaging rust-cbindgen, a dependency of IceCat 68

[Chris Marusich]

Mark H Weaver  writes:

> I have good news and bad news.  The good news is that thanks to the
> heroic efforts of Amin Bandali , a recently appointed
> co-maintainer of GNU IceCat, there now exists a preliminary version of
> IceCat 68 that builds successfully and works on Trisquel.

I'm subscribed to bug-gnuzilla, but I haven't seen messages about this.
Where does communication and coordination for IceCat happen these days?
I'd like to help, and it's tough to work alone.

> The bad news is that IceCat 68 has a new dependency: rust-cbindgen,
> which itself depends on *245* other Rust libraries that are not yet
> packaged for Guix.

I see Efraim is already helping with this, which is great!  Ivan Petkov
and I worked a bit on the rust build system a while back, and although
it works to a certain extent, I think neither of us would claim it's
perfect.  There is a lot of room for improvement, and I'm glad Efraim
has been experimenting with it!

-- 
Chris


signature.asc
Description: PGP signature

Re: Multiseat in Guix

[Chris Marusich]

Brendan Tildesley  writes:

> I don't know how to do this but I've always been fascinated by it, so I
> just wanted to bump this and say that I think it's a really important
> feature to get working (eventually), because for example schools can set
> up a computer lab of 30 workstations using only 15 actual computers,
> saving money and waste.

I'm in the same situation.  I know multiseat is a nice feature, but I
don't know much about how it's implemented.

If we have any multiseat gurus lurking on the list, and you've been
wondering how you can help Guix out...maybe this is your time to shine!

-- 
Chris


signature.asc
Description: PGP signature

Multiseat in Guix

[Chris Marusich]

Hi,

Guix does not seem to have multiseat support.  What would it take to add
it?  Is anyone on the list familiar with how multiseat is achieved in
other distros, such as Fedora?

Here is an example of a problem that happens because we don't have good
multiseat support:

When I launch virt-manager via "sudo -E virt-manager", I can connect a
USB device from the host to a running VM by clicking on the "Virtual
Machine > Redirect USB Device" menu entry.  However, if I launch
virt-manager normally (as the unprivileged user "marusich") and try
this, it fails due to insufficient permissions:

  spice-client-error-quark: Could not redirect [the device] at [the
  device's address]: Could not open usb device: Access denied
  (insufficient permissions) [-3] (0)

I can work around the issue without root privileges by giving myself
write permission on the device in question.  For example:

  sudo setfacl -m u:marusich:rw /dev/bus/usb/001/007

Alternatively, I could have just changed the file mode or ownership.

Here are the file mode, ownership, and ACLs after I did this:

  [0] marusich@garuda.local:~
  $ ls -l /dev/bus/usb/001/007
  crw-rw-r--+ 1 root root 189, 6 Oct 19 13:31 /dev/bus/usb/001/007
  [0] marusich@garuda.local:~
  $ getfacl /dev/bus/usb/001/007
  getfacl: Removing leading '/' from absolute path names
  # file: dev/bus/usb/001/007
  # owner: root
  # group: root
  user::rw-
  user:marusich:rw-
  group::rw-
  mask::rw-
  other::r--

My user is in these groups:

  $ id
  uid=1000(marusich) gid=998(users) 
groups=998(users),976(libvirt),977(tor),984(kvm),990(netdev),992(video),999(wheel),30001(plugdev)

I would like to be able to attach USB devices to VMs without running
virt-manager as root, and without manually granting access to device
files.  How can we achieve that in Guix?

Well, to do that we would need an automatic mechanism which grants
appropriate permissions on the relevant device nodes.  There are many
ways to accomplish that.  For example, Fedora automatically detects when
a device is connected to a user's seat (I'm not sure if that's the right
terminology) and grants them access (via ACLs, I believe).  Concretely,
Fedora accomplishes this by configuring systemd, udev rules, and perhaps
other parts of the system in specific ways.  This allows two different
users Alice and Bob to have access to their own hardware on their own
seats (e.g., in a shared computer lab situation), without allowing Alice
to access Bob's hardware on Bob's seat, or vice versa.  That's really
nice.  I'm not very familiar with all the mechanisms, but I think anyone
would want the result, which is called "multiseat":

  https://www.freedesktop.org/wiki/Software/systemd/multiseat/

For now, the immediate, course-grained, automatic solution for my
virt-manager problem is: I can add udev rules that will unconditionally
set the group of USB device nodes to a special group, maybe named "usb".
If I then add my user to the "usb" group, I will have access to all USB
devices without any extra effort.

However, this solution is too course-grained.  Alice and Bob would both
need to be in the "usb" group to access their own seat's devices, but
Alice will be able to access Bob's devices, and vice versa, which is not
good.  The multiseat solution seems nicer, but it seems complicated to
implement.  Since it seems to rely on systemd in some fashion, it may be
even more difficult to implement in Guix, as we only use extracted parts
of systemd (e.g., elogind).

What would it take to add multiseat support in Guix?

-- 
Chris


signature.asc
Description: PGP signature

Re: fftwf tests running for 25+ hours - is this normal?

[Chris Marusich]

Hi Bengt, Matteo, and others,

I've stopped the build.  The process ran for 73 hours and didn't
complete.  I've also noticed that there are no recent builds in Cuirass,
so I wonder: has anyone built fftwf successfully recently?  I was trying
to build fftwf based off of what is currently in the core-updates
branch.

The latest successful build for fftwf 3.3.8 on x86_64-linux was on
September 1st (of this year, I think):

https://ci.guix.gnu.org/build/1640133/details

It supposedly took 13145 seconds to complete.  However, I'm not sure how
accurate that is, since this other entry for a build on armhf-linux says
it took 1570098173 seconds (is probably some kind of Cuirass bug, since
that puts the supposed start time around the epoch):

https://ci.guix.gnu.org/build/1690957/details

For now, I'm going to try building an older version of Guix or fftwf and
see how that goes.

Bengt Richter  writes:

> lscpu|grep -i '^CPU m'
> [...]
> grep '^cpu MHz' /proc/cpuinfo

Thanks for this tip!  I can see the CPU speed was reduced to 800 MHz at
one point, but it seems it's back to 2400 MHz now:

--8<---cut here---start->8---
$ lscpu  | grep -i '^cpu m'
CPU MHz: 2400.035
CPU max MHz: 2400.
CPU min MHz: 800.
--8<---cut here---end--->8---

Same info from /proc/cpuinfo:

--8<---cut here---start->8---
$ cat /proc/cpuinfo | grep '^cpu MHz'
cpu MHz : 2400.035
cpu MHz : 2400.035
--8<---cut here---end--->8---

-- 
Chris


signature.asc
Description: PGP signature

Re: fftwf tests running for 25+ hours - is this normal?

[Chris Marusich]

Hi Bengt and Matteo,

Bengt Richter  writes:

> Have you checked sensors for overheating that might induce CPU clock 
> throttling?

Actually, yes, I happened to be watching dmesg output at the time, and I
did notice these messages (no similar messages have been printed since
then, which was many hours ago):

--8<---cut here---start->8---
[180270.045081] mce: CPU1: Core temperature above threshold, cpu clock 
throttled (total events = 124733)
[180270.045567] mce: CPU1: Core temperature/speed normal
[180570.044352] mce: CPU1: Core temperature above threshold, cpu clock 
throttled (total events = 134902)
[180570.044838] mce: CPU1: Core temperature/speed normal
[180875.663432] mce: CPU1: Core temperature above threshold, cpu clock 
throttled (total events = 143897)
[180875.663918] mce: CPU1: Core temperature/speed normal
[181175.748616] mce: CPU1: Core temperature above threshold, cpu clock 
throttled (total events = 153503)
[181175.749103] mce: CPU1: Core temperature/speed normal
[181476.496915] mce: CPU1: Core temperature above threshold, cpu clock 
throttled (total events = 171377)
[181476.497401] mce: CPU1: Core temperature/speed normal
[181776.914264] mce: CPU1: Core temperature above threshold, cpu clock 
throttled (total events = 185828)
[181776.914751] mce: CPU1: Core temperature/speed normal
[182076.914391] mce: CPU1: Core temperature/speed normal
[182377.322112] mce: CPU1: Core temperature above threshold, cpu clock 
throttled (total events = 231578)
[182377.322598] mce: CPU1: Core temperature/speed normal
--8<---cut here---end--->8---

Is this throttling permanent, or is the throttling released after the
temperature returns to normal?

Matteo Frigo  writes:

> thanks for reporting this issue.  (By the way, did the test ever
> finish?)

It's still running.  It's been 38 hours now.

> Most likely, the problem is in FFTW and not in your machine.  Would you
> mind reporting the exact configure options (I care specifically about
> --enable-sse2, --enable-avx, etc.) and the exact command line that is
> taking 25h?

These two options were included in the configure invocation.

> The issue is probably related to the funny integer 34320.  The FFT
> algorithm depends on the factorization of the size, 34320 in this case,
> which is 34320=2*2*2*2*3*5*11*13.  FFTW tries to find the optimal way to
> decompose 34320 into small problems that it can solve.  For example, it
> may reduce a DFT of size 34320 into 34320/13 FFTs of size 13 followed by
> 13 FFTs of size 34320/13.  Or it may reduce a DFT of size 34320 into
> 34320/11 FFTs of size 11 followed by 11 FFTs of size 34320/11.  Because
> there are many distinct prime factors, the number of factorizations is
> really large and FFTW is pretty much trying them all.  The problem is
> even harder once you consider the fact that some sizes are odd, and thus
> they don't naturally match the vector length of SSE/AVX instructions.
>
> FFTW has some heuristics to limit the search space in cases like this,
> but these heuristics were implemented in a simpler world (before AVX)
> and maybe they are not adequate any longer, so I want to look into this.

Thank you for taking a look!  Here is how the test is being run.

Output of "pstree -lap 20098":

--8<---cut here---start->8---
make,20098 check -j 2
  `-bash,20099 -c fail=; \\\012if (target_option=k; case ${target_option-} in 
?) ;; *) echo "am__make_running_with_option: internal error: invalid" "target 
option '${target_option-}' specified" >&2; exit 1;; esac; has_opt=no; 
sane_makeflags=$MAKEFLAGS; if { if test -z '0'; then false; elif test -n 
'x86_64-unknown-linux-gnu'; then true; elif test -n '4.2.1' && test -n 
'/tmp/guix-build-fftwf-3.3.8.drv-0/fftw-3.3.8'; then true; else false; fi; }; 
then sane_makeflags=$MFLAGS; else case $MAKEFLAGS in *[\\ \\\011]*) 
bs=; sane_makeflags=`printf '%s\\n' "$MAKEFLAGS" | sed "s/$bs$bs[$bs 
$bs\011]*//g"`;; esac; fi; skip_next=no; strip_trailopt () { flg=`printf 
'%s\\n' "$flg" | sed "s/$1.*$//"`; }; for flg in $sane_makeflags; do test 
$skip_next = yes && { skip_next=no; continue; }; case $flg in *=*|--*) 
continue;; -*I) strip_trailopt 'I'; skip_next=yes;; -*I?*) strip_trailopt 'I';; 
-*O) strip_trailopt 'O'; skip_next=yes;; -*O?*) strip_trailopt 'O';; -*l) 
strip_trailopt 'l'; skip_next=yes;; -*l?*) strip_trailopt 'l';; -[dEDm]) 
skip_next=yes;; -[JT]) skip_next=yes;; esac; case $flg in *$target_option*) 
has_opt=yes; break;; esac; done; test $has_opt = yes); then \\\012  
failcom='fail=yes'; \\\012else \\\012  failcom='exit 1'; \\\012fi; 
\\\012dot_seen=no; \\\012target=`echo check-recursive | sed s/-recursive//`; 
\\\012case "check-recursive" in \\\012  distclean-* | maintainer-clean-*) 
list='support genfft kernel simd-support dft rdft reodft api libbench2 . 
threads tests mpi doc tools m4' ;; \\\012  *) list='support  kernel 
simd-support dft rdft reodft api 

fftwf tests running for 25+ hours - is this normal?

[Chris Marusich]

Hi,

I've been trying to reconfigure my system on my x200 laptop.  It's been
a day or two, and I've found that my computer has spent the last 25
hours running fftwf tests.  I know my laptop is rather weak (2 cores, 8
GB of memory, an SSD) compared to most desktops today, but still, 25
hours seems pretty long.

Guix is running

  make check -j 2

which ultimately calls

  perl -w ./check.pl -r -c=30 -v --nthreads=2 
/tmp/guix-build-fftwf-3.3.8.drv-0/fftw-3.3.8/tests/bench

which calls

  /tmp/guix-build-fftwf-3.3.8.drv-0/fftw-3.3.8/tests/.libs/bench -o nthreads=2 
--verbose=1 --verify //obcd34320 [... a lot of arguments ...]

Despite the fact that "-j 2" was provided to make, only 1 CPU is pegged
at 100% utilization.  I suppose I'll just wait and see what happens, but
25 hours feels very long.  It seems on ci.guix.gnu.org, the build times
are generally less than 1 hour:

  https://ci.guix.gnu.org/search?query=fftwf

Is this normal, or should I kill the build and try again?  What kind of
system is ci.guix.gnu.org building on (how many cores, how much memory)?

-- 
Chris


signature.asc
Description: PGP signature

Re: i686-linux GCC package on x86_64

[Chris Marusich]

Ricardo Wurmus  writes:

> Chris Marusich  writes:
>
>> That said, I am curious about something.  If I want to make a
>> cross-compiler available for the purpose of hacking around on some code
>> and cross-compiling it, is there an equivalent to "guix package -i
>> gcc-toolchain" which will give me a cross-compilation toolchain?  My
>> feeling was that Guix can create cross-compilation toolchains on demand,
>> but there is no UI exposed for making it available via a guix command,
>> since people are encouraged to just ask for the cross-built product.
>
> Both avr-toolchain and arm-none-eabi-toolchain-6 (among others) are
> examples of cross toolchains that can be installed on one type of system
> to build binaries for another architecture.

Thank you for the tip!  I never knew we had such packages!

-- 
Chris


signature.asc
Description: PGP signature

Re: i686-linux GCC package on x86_64

[Chris Marusich]

Danny Milosavljevic  writes:

> On Fri, 04 Oct 2019 17:46:43 +0200
> Jelle Licht  wrote:
>
>> Mathieu Othacehe  writes:
>> 
>> >
>> > --8<---cut here---start->8---
>> > (native-inputs
>> >  `(,@(if (not (string-prefix? "i686" (%current-system)))
>> >`(("cross-gcc" ,(cross-gcc "i686-unknown-linux-gnu"))
>> >  ("cross-binutils" ,(cross-binutils "i686-unknown-linux-gnu")))
>> >'(
>> > --8<---cut here---end--->8---
>> >
>> > that uses the current gcc if you're already building on an i686 system,
>> > or define and use a cross-gcc targeting i686 systems otherwise.  
>> 
>> This snippet might make a lot of sense to seasoned schemers/guixfolk,
>
> Basically just ignore the birdshit characters to understand what it does :)
>
> The first unquote is to evaluate (%current-system) at the toplevel which is
> what interprets the package definitions in the first place.
>
> "`(,@" is a no-op.  Not sure why it's written like that.
>
>>`(("cross-gcc" ,(cross-gcc "i686-unknown-linux-gnu"))
>>  ("cross-binutils" ,(cross-binutils "i686-unknown-linux-gnu")))
>
> is like we always write inputs, but it's calling the "cross-gcc" function in 
> the
> toplevel in order to get the package to use.

Perhaps it's because of the "if" expression.

>> with the multiple levels of (un)quoting and what not. It does not seem
>> like what somebody with little experience in either would think of by
>> themselves.
>> 
>> Would it make sense to have a section/stub in the cookbook about cross
>> compilation?
>
> I don't think that cross-gcc is stable API that much--and it's not
> common that you need it anyway.  A normal user just cross compiles
> by using
>
>guix build --target=i686-linux pkg
>
> or better yet, uses qemu to natively compile it for a foreign system
>
>guix build --system=i686-linux pkg
>
> .
>
> The above is only necessary when for some reason your package has
> parts which only compile on one system and other parts which only
> compile on another system--that's very rare.
>
> Nowadays, packages are supposed to be cross-platform, so using cross-gcc
> directly is unnecessary, too.
>
> Right now cross-gcc is not documented, so I guess that counts as
> "not a public interface".

Yeah.  I'm inclined to agree with Danny.  With Guix, it is best to
cross-compile in the way Danny is suggesting.  If there are projects
that require other tricks to cross-compile, perhaps they would be
interesting to discuss.  I feel like there may be a lot of "quick and
dirty" projects with non-standard build logic that can't easily be
packaged into Guix for cross-compilation.

That said, I am curious about something.  If I want to make a
cross-compiler available for the purpose of hacking around on some code
and cross-compiling it, is there an equivalent to "guix package -i
gcc-toolchain" which will give me a cross-compilation toolchain?  My
feeling was that Guix can create cross-compilation toolchains on demand,
but there is no UI exposed for making it available via a guix command,
since people are encouraged to just ask for the cross-built product.

-- 
Chris


signature.asc
Description: PGP signature

Re: Maintainer needed for Linux-libre and IceCat packages

[Chris Marusich]

Hi Mark,

Mark H Weaver  writes:

> I need to take a break from Guix.  I'm not sure if I'll be back or not.
> Someone else should take over maintenance of the Linux-libre and IceCat
> packages, starting with the recent kernel updates for 4.4.192, 4.9.192,
> 4.14.143, 4.19.72, and 5.2.14.

Thank you for all your work on these important parts of the Guix package
collection.  I and many others have directly benefited from your
contributions and helpful insights.  I will miss you, and I'm sure
others will, too.

Regarding IceCat specifically, is there any work-in-progress towards the
latest Firefox ESR release (68.1.0) that we should know about?

No matter where life takes you, I wish you the best of luck!

-- 
Chris


signature.asc
Description: PGP signature

Re: 01/01: services: Add ‘/usr/bin/env’ special file.

[Chris Marusich]

Hi everyone,

Ricardo Wurmus  writes:

> Using a custom script with a /usr/bin/env shebang is pretty common.  You
> don’t need to be a power user for that, and certainly not a *Guix* power
> user.
>
> [...]
>
> Personally, I think it’s a good idea to default to having /usr/bin/env
> shebangs just work on Guix Systems.
>
> [...]
>
> With the same reasoning we could argue against having coreutils on PATH.
> This is not an exaggeration: R depended on coreutils to be on PATH at
> runtime and we only found out when we ran it in a container without
> coreutils.

I agree.  At first, I thought I could do without /usr/bin/env, but I
always end up installing it because scripts that others share with me
frequently use it, and it's just too painful to have to patch them all
or package up those scripts in Guix package definitions.  I encounter
scrips like this frequently enough that I've decided I want
/usr/bin/env.

Ludovic Courtès  writes:

> Like I wrote in  and in the
> message it refers to, although I was initially mildly reluctant to
> having /usr/bin/env by default, I’ve come to think that lack of
> /usr/bin/env is a gratuitous annoyance—not to me of course, but to
> newcomers as we’ve seen repeatedly, be they seasoned GNU/Linux users or
> not.
>
> With that in mind, adding /usr/bin/env by default is probably a good move.
>
> Now, we can add a snippet in the manual with the ‘modify-services’ trick
> to remove /usr/bin/env.  :-)
>
> [...]
>
> Well anyway, if we take a step back, we’re talking about a really tiny
> issue in the grand scheme of things, and it’s certainly not worth losing
> our hair over it.  :-)

I feel the same way.

Regardless of how we came to discuss it, I'm glad we've discussed the
issue in this email thread, and I'm glad to hear that I'm not alone in
changing my mind about /usr/bin/env.  It's good to have it by default.

-- 
Chris


signature.asc
Description: PGP signature

Re: bug#36855: guix system switch-generation doesn't

[Chris Marusich]

Hi Robert,

Robert Vollmert  writes:

> On 8. Aug 2019, at 18:40, Chris Marusich  wrote:
>> zerodaysford...@sdf.lonestar.org (Jakob L. Kreuze) writes:
>> 
>>> 'switch-to-system-generation' doesn't call out to
>>> 'upgrade-shepherd-services'. I'm not sure if this was an intentional
>>> decision or not
>> 
>> It is intentional, but only because there is currently no way to call
>> upgrade-shepherd-services when switching system generations.
>
> How does shepherd work on a non-guix system? Can’t be it be configured
> like other daemons to read its configuration from a file, e.g. from
>
>/run/current-system/etc/shepherd.conf
>
> and be told via signal to reload its configuration from disk?

Maybe!  In the email thread I linked, Ludo talked about storing a
description of the Shepherd services in the system generation for future
reference.  Maybe we could store it in a place like this, and maybe
Shepherd already has mechanisms for reloading configurations like this.
I don't intend to work on this because I need to focus on other things
right now, but I would be happy if someone took up this work!

> (I feel a bit cheated right now. This behaviour makes Guix System entirely
> unsuitable for server use. It shouldn’t be advertised as supporting
> transactional upgrades and rollbacks if those require a reboot.)

I agree that Guix should update as many Shepherd services as it can when
switching generations.  However, I don't think it's inaccurate to say
that Guix supports transactional upgrades and rollbacks.  When you
invoke "guix system switch-generation", the system profile symlink is
flipped atomically, so you get an atomic update from one version of the
system to another.  Software running in the system never sees an
inconsistent view of the system.  Contrast this with nearly any other
mutable GNU/Linux system, in which files are more or less sprayed into
the existing file system with no guarantee of consistency or atomicity.

-- 
Chris


signature.asc
Description: PGP signature

Re: "python setup.py test" is deprecated

[Chris Marusich]

Marius Bakke  writes:

> Pythons setuptools are deprecating the "python setup.py test" command:
>
> https://github.com/pypa/setuptools/issues/1684
>
> As you may know, "python setup.py test" is what python-build-system does
> during the 'check' phase.
>
> I'm not sure what we should do about it, and there does not seem to be
> an established "community consensus" yet.  I suppose many packages will
> be migrating to Pytest and/or Tox?  Perhaps we could check package
> inputs and try to guess the correct test command?
>
> Something to keep in mind for future python-build-system improvements...
>
> PS: Apparently "python setup.py install" is going away too:
> https://github.com/pypa/setuptools/issues/1684#issuecomment-508302910

To my knowledge, the "targets" of the setup.py script are somewhat not
standardized, even though they seem like they ought to be.  At the very
least, you are right about the "test" target.  Even pytest dropped
support for it since last year:

https://docs.pytest.org/en/latest/changelog.html?highlight=%22setup.py%20test%22#id511

This is just one example of how a project can implement its own
idiosyncratic build logic.  Somebody uses tox, someone else uses a
Makefile with a "check" target (that runs pytest), somebody else just
runs pytest directly...  It's business as usual as far as I can tell.
Guix can accommodate all those idiosyncrasies via modifications to the
phases, and if the Python community starts using one method much more
often, we can just make that the default in the build system.

-- 
Chris


signature.asc
Description: PGP signature

Re: bug#36855: guix system switch-generation doesn't

[Chris Marusich]

Hi Jakob,

zerodaysford...@sdf.lonestar.org (Jakob L. Kreuze) writes:

> 'switch-to-system-generation' doesn't call out to
> 'upgrade-shepherd-services'. I'm not sure if this was an intentional
> decision or not

It is intentional, but only because there is currently no way to call
upgrade-shepherd-services when switching system generations.

Consider the procedure upgrade-shepherd-services: you must pass it an
 record.  When you are flipping from one generation to
another, how do you get the  record that was used for
the generation you're switching to?  Guix doesn't currently store the
operating system configuration file or its  record
anywhere, so we can't call upgrade-shepherd-services.

This was discussed in 2016 and we agreed we need to persist some
information to enable us to handle Shepherd services correctly.  This is
what Ludo suggested at the time:

https://lists.gnu.org/archive/html/guix-devel/2016-06/msg00173.html

"Maybe we could store in the system output (result of ‘guix system
build’) an sexp representation of (part of) our 
records:

  (shepherd-service
(provisions (x y z))
(requirements (a b c))
(start-script "/gnu/store/…-start-foo.scm")
(stop-script "/gnu/store/…-stop-foo.scm")
…)

Then ‘upgrade-shepherd-services’ could start from this simplified
representation instead of using the full-blown 
objects, and thus could work both when instantiating a new generation
and when rolling back."

Until that happens, you'll always have to reboot to complete the
switch.

FYI, last I checked (about 3 years ago), in NixOS they took a slightly
different approach: instead of storing state describing the previous
system generation and relying on the current system's logic to correctly
parse it and use it to revert the system to a prior configuration, they
just dump everything into a self-contained script that knows how to
update the entire system to one specific configuration.  That approach
is nice in some ways because switching generations is dead simple - you
just run the switching script belonging to the generation you want to
switch to - but it also has downsides.

For example, if the target generation is old enough compared to the
current system, then the target generation's old switching script might
not understand how to deal with the current system correctly.  Instead,
if you only store the bare minimum of state required to take the right
actions, and you implement the meat of the logic in the current Guix
installation, you are more likely to be able to switch generations even
to very old ones where the world was very different, since the current
Guix can be taught how to deal gracefully with the old world.  But it
seems more complicated.  It's all about trade-offs.

That said, I've never gone back to implement this.  If you want to give
it a try, you're more than welcome to do so!

-- 
Chris


signature.asc
Description: PGP signature