Re: Should we include nss-certs out of the box?

[Maxim Cournoyer]

Hello!

Clément Lassieur  writes:

> On Wed, Apr 03 2024, Maxim Cournoyer wrote:
>
>> It's been Guix policy to let people choose whether to install or not TLS
>> root certificates and which one to their machine.  While I applaud the
>> idea to have the users make a conscious decision about it, in practice I
>> suppose very few of us choose to *not* install any as that basically
>> breaks using web browsers, especially ones like IceCat which (by
>> default) ensures HTTPS is used on every page.
>
> I'd be surprised Icecat breaks from this as it uses its own cert
> database and allows HTTP when HTTPS doesn't work.

I didn't know Icecat had its own cert database.

About allowing HTTP, it can access pages using it, but not without going
through a "Continue despite security risks" dialog, and perhaps turning
off the HTTPS everywhere add-on for the page, which is installed by
default.

-- 
Thanks,
Maxim

Re: Sustainability fund application ongoing

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès  writes:

> Hi Maxim,
>
> Maxim Cournoyer  skribis:
>
>> It was pointed to us that the "Free and Open Source Software
>> Sustainability Fund" [0] is currently receiving applications.  The fund
>> aim to sponsors free software projects for their
>> maintenance/organisational activities.
>
> Nice!  You forgot the link but I guess it’s
> <https://www.opentech.fund/funds/free-and-open-source-software-sustainability-fund/>.
> Deadline is May 17th.

Indeed, thanks for pointing it out.

>> I'm tempted to apply myself, but I thought I'd share it here: the more
>> submissions they get the more likely Guix is to receive support.
>
> Agreed, it’s great to have people apply there (even better if it works,
> of course).
>
> Maybe we should provide a contact point for people who want to
> coordinate when applying for Guix-related funding, to avoid overlapping
> or competing applications?

I guess we can use guix-devel for that purpose; the traffic would not justify a 
new
mailing list I guess (and we already have too many to keep track of :-)).

-- 
Thanks,
Maxim

Re: No default OpenJDK version?

[Maxim Cournoyer]

Hi,

Markku Korkeala  writes:

> On Tue, Apr 16, 2024 at 10:37:30PM +0200, Julien Lepiller wrote:
>> Currently, most java packages use the implicit jdk from the build
>> system (ant- or maven-build-system), which is… icedtea@8. We still
>> have quite a lot of old packages that don't build with openjdk9, so
>> I'm not sure when we can update the default jdk…
>
> Hi,
>
> is there effort to update the default jdk at some point? I could help with
> it. I'm not familiar with the guix java build systems, but have long
> experience as a Java developer. I also maintain few java packages in Fedora
> and saw the transition to to jdk11 [1], jdk17 [2] and now to jdk21 [3]. The
> pages have documented common issues and workarounds, which might help.

I'm not aware of such effort in Guix currently, so feel free to
spearhead it!  It'd be beneficial, although I supposed difficult to
achieve due to bootstrap reasons, perhaps.

-- 
Thanks,
Maxim

Re: Python's native-inputs

[Maxim Cournoyer]

Hi Nicolas,

Nicolas Graves  writes:

[...]

>  TL;DR :
>  - patch series in big progress, not done yet because I don't really
>  know where to stop and massive rebuilds.
>  - WDYT about tweaking the build-system for pytest?
>  - not done : tweaking the pypi import to ignore those packages. I've
>  done it in the juliahub import code, but I haven't delved into that
>  though.
>  - not done : adding those packages to guix lint - trivial. We should
>  probably add a variable in pypi import that is exported and added into
>  guix lint.
>
>  Acting on the three (guix import, guix lint, guix build) can make it
>  quite easy on maintainers in the end IMO.

You're bringing good arguments.  If you can pull it off in the way
outlined above, I agree it's be valuable to have.

-- 
Thanks,
Maxim

Re: Status of ‘core-updates’

[Maxim Cournoyer]

Hi Christopher,

Christopher Baines  writes:

> Ludovic Courtès  writes:
>
>> What’s the status of ‘core-updates’?  What are the areas where help is
>> needed?
>>
>> I know a lot has happened since the last update¹, which is roughly when
>> I dropped the ball due to other commitments, but I’m not sure where we
>> are now.
>
> I haven't really been following core-updates, but I have had a look
> since there's a request to merge it now [1].
>
> I'm really concerned by the commits on the branch though, assuming I'm
> using Git right, there are 6351 commits on the branch:
>
>   git log --pretty=oneline core-updates ^master | wc -l
>
> Somehow, I think there's been a couple of pushes of commits to
> core-updates that have partially duplicated lots of commits from master,
> I put some more details in:
>
>   https://issues.guix.gnu.org/70456#3
>
> I think keeping the Git commit history clean and representative is
> really important, so to me at least this means core-updates can't be
> merged to master in it's current form, even if the changes overall from
> these 6351 commits are reasonable.
>
> I'm really not sure how to move forward though, I had a go at trying to
> rebuild the branch without introducing the thousands of duplicate
> commits and that produced a branch with 765 commits over master, which
> still seems a lot, but a big improvement over 6351:
>
>   https://git.cbaines.net/guix/log/?h=chris-core-updates-no-duplicates-attempt
>
> That was really hard going though, as there's plenty of merge conflicts
> along the way, and I'm pretty sure I solved some of them
> incorrectly. The resulting branch also differs from core-updates.

I also think Git commit history is important, but in this case I weigh
the value of removing ~5000 duplicated rust commits against the risks of
resolving merge conflicts wrong or forgetting commits upon attempting to
recreate the branch from scratch lower than the benefit.

> Maybe someone with more time, care and attention could do a better job,
> but it might be more worthwhile just starting fresh and rather than
> trying to produce a like for like branch just without the thousands of
> duplicate commits, effectively manually rebase the branch (without the
> duplicate commits) on master and try to get the commits in to a usable
> state.

Given the little attention core-updates is currently receiving, I doubt
someone is willing to put the effort to recreate the branch from scratch
to clean its git history; at least speaking for myself I'd rather spend
the little hack time I have to work on it toward getting it finalized.

I believe how these duplicates came to exist was probably two separate
master -> core-updates merge commits, with one of them ending up being
rebased on top of the other, probably so that it could be pushed.
Perhaps we could capture in our contribution guidelines that rebasing a
merge commit should never be done to keep the history clean, and that in
a situation where:

1. a merge has been prepared locally (with conflicts resolved and all)
2. a new commit has appeared on the remote branch

the solution should be to merge the remote branch into the local one
instead of rebasing the local one on the remote one (as is usually
done).  Disclaimer: I haven't actually tried this suggested approach,
which should be done before documenting it, if there's a consensus to do
so.

In other words, I suggest we document what *not* to do to avoid
repeating the same mistake in the future, and move on.

-- 
Thanks,
Maxim

Welcome to Zheng (z572) as a new committer!

[Maxim Cournoyer]

Hi comrades,

Zheng has joined the committers to help improving cross-compilation,
riscv64, and KDE, among others.

Let's wish them a warm welcome!

Happy hacking!

-- 
Thanks,
Maxim

Re: Creating a documentation team?

[Maxim Cournoyer]

Hi,

Ludovic Courtès  writes:

> Hi Florian and all,
>
> I figure you’ve been doing a lot of review and writing of the manual.
> Should we create a documentation team, of which you could be a honorary
> member?  :-)
>
> I feel like ensuring doc consistency, be it regarding the content,
> terminology, typography, or use of markup, is a job in its own that
> could be best reviewed by people familiar with and interested in those
> issues.
>
> WDYT?
>
> Ludo’.
>
> diff --git a/etc/teams.scm b/etc/teams.scm
> index d537e83efc..4d65a5476e 100755
> --- a/etc/teams.scm
> +++ b/etc/teams.scm
> @@ -434,6 +434,16 @@ (define-team core
>(make-regexp* "^guix/scripts/")
>(make-regexp* "^guix/store/"
>  
> +(define-team documentation
> +  (team 'documentation
> +#:name "Documentation"
> +#:description "Documentation: the manual and cookbook."
> +#:scope (list (make-regexp* "\\.texi$")
> +  "doc/build.scm"
> +  "gnu/system/examples/bare-bones.tmpl"
> +  "gnu/system/examples/lightweight-desktop.tmpl"
> +  "gnu/system/examples/desktop.tmpl")))
> +
>  (define-team core-packages
>(team 'core-packages
>  #:name "Core packages"

I second this.  I'd be happy to be added to it.

-- 
Thanks,
Maxim

Re: Should we include nss-certs out of the box?

[Maxim Cournoyer]

Hi Fabio,

Fabio Natali  writes:

> Hi,
>
> Here's my attempt at adding 'nss-certs' to '%default-packages'.
>
> https://lists.gnu.org/archive/html/guix-patches/2024-04/msg01187.html
>
> I've removed the 'nss-certs' entry from the installer, as suggested by
> Ludo, and I've updated the docs, hopefully all the relevant parts. Can
> you think of anything else that may need to be updated?

It seems we were working on a similar thing at a similar time :-).
Could you please take a look at
'65e8472a4b6fc6f66871ba0dad518b7d4c63595e', which I hope didn't leave
no longer useful 'nss-certs' doc/examples behind ?

-- 
Thanks,
Maxim

Re: Status of ‘core-updates’

[Maxim Cournoyer]

Hi Ludo,

Ludovic Courtès  writes:

> Hi Maxim,
>
> Maxim Cournoyer  skribis:
>
>> Since branches were merged in, I believe the problem we are facing at
>> the moment is librsvg failing its test suite with a segfault (!).  Could
>> be the glibc upgrade, or rust itself, I'm not sure.  I was trying to
>> upgrade librsvg, which needs an update anyway, but it pulls many rust
>> crates updates.  I'll get there eventually, if nobody beats me to it.
>
> Ouch, OK.  I guess it doesn’t hurt if several of us take a look and we
> share our findings here.
>
>>> Josselin, Maxim: could you explain what problems there are around
>>> pkgconf and what you would recommend?
>>
>> Nothing in particular to point at the moment, but remaining problems
>> would manifest in the form of missing inputs, due to transitive libtool
>> dependencies causing overlinking and the new pkgconf being smart enough
>> to optimize away some previously captured link directives that would be
>> baked in the RUNPATH and sastify libtool overlinking needs.
>>
>> The solution is to hunt the libtool .la files from the transitive
>> dependencies causing the problem and removing them. See commit
>> b6540bd285cbe5920ad379ddfc6256359ee7204c for an example.
>
> Good.  So it seems like we can move forward after all and just do the
> “normal” job and finding and fixing build failures along these lines.
>
> Do we need ci.guix to build more packages to facilitate testing and
> debugging?  That’s something I can help with (though I’ll be away for a
> week).

If we do this, and I believe it'd be useful, I think it'd help to fork a
'core-updates-frozen' and have the CI rebuild this one fully, leaving
'core-updates' open to business as usual.

-- 
Thanks,
Maxim

Sustainability fund application ongoing

[Maxim Cournoyer]

Hello Guix,

It was pointed to us that the "Free and Open Source Software
Sustainability Fund" [0] is currently receiving applications.  The fund
aim to sponsors free software projects for their
maintenance/organisational activities.

I'm tempted to apply myself, but I thought I'd share it here: the more
submissions they get the more likely Guix is to receive support.

-- 
Thanks,
Maxim

Re: policy for packaging insecure apps

[Maxim Cournoyer]

Hi Attila,

Attila Lendvai  writes:

> the context:
> 
>
> there's an app currently packaged in guix, namely 
> gnome-shell-extension-clipboard-indicator, that has a rather questionable 
> practice: by default it saves the clipboard history (passwords included) in 
> clear text, and the preferences for it is called something obscure. its 
> author actively defends this situation for several years now, rejecting 
> patches and bug reports.

Are there no forks yet?

[...]

> my question:
> 
>
> how shall we deal with a situation like this?
>
>  1) shall i create a guix patch that makes the necessary changes in
> this app, and submit it to guix? this would be a non-trivial, and
> a rather hidden divergence from upstream, potentially leading to
> confusion.

Perhaps enough people are interested in getting this in order to have
created a fork already?  We could use it.

>  2) is there a way to attach a warning message to a package to explain
> such situations to anyone who installs them? should there be a
> feature like that? should there be a need for a --force switch, or
> an interactive y/n, to force installing such apps?

For now, I think a simple mention of the issue in the description could
be enough.

>  3) is there a point where packages refusing to address security
> issues should be unpackaged? and also added to a blacklist until the
> security issue is resolved? where is that point? would this one
> qualify?

Is the issue is severe enough (I don't think it is the case here -- it
seems a note to its description would be sufficient -- but I haven't
looked closely into it), I think the package could be dropped, discussed
as a patch.

>  4) is this the responsibility of a project like guix to address
> situations like this?

When the security risks introduced are severe, I'd say so (by excluding
such package from our collection) -- but it would need to be something
truly problematic.

-- 
Thanks,
Maxim

Re: Python's native-inputs

[Maxim Cournoyer]

Hi Nicolas,

Nicolas Graves via "Development of GNU Guix and the GNU System
distribution."  writes:

> Hi Guix,
>
> On some languages, there are a lot of unused native-inputs that are
> development & linting dependencies much more than packages that are
> actually used to build or test a package (I'm thinking at least Python
> and Rust). These fall in the category of tools "useful" at run time, but
> unecessary at build time.

Indeed.

> Is there a clear policy about their removal? I've seen the case of
> pre-commit in Python, and I've commited a series yesterday regarding
> pylint, but there are a whole lot of them in Python, at least :

[...]

> These packages make a lot of sense when considering things like
> `guix shell -D` but they are hampering some progress on Python packages
> since they are everywhere and a small update in their inputs rebuilds
> the whole python world (even though it has NO influence on the
> functionality of any other package).
>
> What are the guidelines in this case?

There aren't really any guidelines.  It's easy to avoid the linters, it
makes sense to avoid them, but sometimes upstream makes it difficult to
avoid running a linter as part of there test suite, in which case I
think it's acceptable to keep them as native-inputs rather than come up
with more patches to maintain.

> I can propose a huge patch series (currently ~300 patches, and not
> finished), to remove them, lint against them and remove them from the
> importer as a default, but that's a big decision to make. IMO we should
> have a dev-inputs field to handle these cases, but that's even more work.

The situation is similar as for other test inputs such as pytest; it has
an enormous amounts of dependents and thus cannot be easily upgraded on
the master branch.  At least more up-to-date variants can be added since
these are not in the transitive closure of any packages.

I don't think we should go out of our way to address annoyances that
upstream have caused -- that'd be too much work to maintain in the long
run.

-- 
Thanks,
Maxim

Re: Should we include nss-certs out of the box?

[Maxim Cournoyer]

Hello,

Ludovic Courtès  writes:

[...]

>> It apparently even makes it impossible to run 'guix pull', if I am to
>> believe bug#62026.
>
> I don’t think that’s the case: see use of ‘le-certs’ in (guix scripts
> pull).

OK, good to know!

>
>> Should we do as in bug#62026 and have this package be part of the
>> recommended basic installation?  It'd be in the basic set of an
>> operating-system packages (via its default %base-packages set).  It
>> could still be manipulated via the Guix API (filtered out/replaced with
>> something else).
>>
>> Is anyone opposed to having nss-certs in %base-packages?
>
> No objection from me.  I’m partly responsible for the initial choice to
> not include nss-certs by default, but as you write, most likely everyone
> installs it these days.
>
> Note that we’ll also need to remove that choice from the installer in
> (gnu installer services).

I went ahead and have now done so, and included a news entry for it.
I've adjusted the OSes templates accordingly, the doc, and the
installer in 65e8472a4b6fc6f66871ba0dad518b7d4c63595e.  Let me know if
I've missed anything.

-- 
Thanks,
Maxim

Re: 01/06: gnu: gnurl: Deprecate in favor of curl.

[Maxim Cournoyer]

Hi Ludo,

Ludovic Courtès  writes:

> Hi,
>
> guix-comm...@gnu.org skribis:
>
>> +(define-deprecated/public-alias gnurl curl)
>
> Just for the record (because it probably doesn’t matter much in this
> case), this creates a deprecated alias for the variable, but not for the
> package.
>
> For package deprecation, I think it’s best to write:
>
>   (define-public gnurl (deprecated-package "gnurl" curl))
>
> to allow “guix install gnurl” to DTRT.  Deprecating the variable itself
> is usually less important for packages.

Ugh.  I always spent some minutes looking at the docstring and they are
too general/vague to be of much practical use, at least to me, and I
often manage to get it wrong, as I did here!  I tried to follow-up with
the recommended '(define-public gnurl (deprecated-package "gnurl"
curl))' alternative, but it seems I'm not hitting a Guile module
top-level dependency cycle, as it won't byte compile, erroring with:

--8<---cut here---start->8---
ice-9/eval.scm:293:34: error: curl: unbound variable
--8<---cut here---end--->8---

Indeed, deprecated-package works by inheritance, so according to our
guidelines defined in (info '(guix) Cyclic Module Dependencies'), it
should be defined in the curl module.  I've now done so in commit
a69e5e5e47b70e3fe14040142544147fbd9239a1.

> As I write this, I realize we should probably document package
> deprecation and removal.

This would greatly help, and/or extended docstrings with practical
examples at the definition sites.

-- 
Thanks,
Maxim

Re: Fix grammar and markup (was Re: Feedback of the GNU Guix manual)

[Maxim Cournoyer]

Hi Matt,

Matt  writes:

>   On Mon, 15 Apr 2024 14:58:50 +0200  pelzflorian (Florian Pelz)  wrote 
> --- 
>
>  > Do you agree that I should commit your docs correction with @pxref?
>  > I believe it is an improvement over current “see @ref”, even though it
>  > looks different in the info reader.
>  
> Yes, I agree and thank you for double checking.  Sorry for not
> answering you more clearly before!  @pxref is the correct command to
> use within parentheses.
>
>  > I believe the Emacs errors are historical; looking at Emacs 29.3 as
>  > packaged in Guix, Emacs-Info displays xref properly as See.  Display
>  > errors had existed, but in the past only.
>
> I reported it to emacs-devel (it's addressed now) and the issue isn't
> technically @xrefs; it's the compilation of @xref and @ref, '*Note'
> and '*note' respectively.  Emacs looks at the context around these and
> renders them as 'See' or 'see' according to what's nearby.  The
> trouble is that the Texinfo documentation intends to show '*Note' and
> '*note' literally which is an exception to the Emacs logic.
>   
> I looked it up and the related Emacs code had been there for like 18
> years...I have a knack for finding these kinds of dumb things.  I
> can't decide if it's a superpower or a super-curse. :)

Awesome!  Definitely a superpower in my book.  Thanks for following it
through with the Emacs folks!  I'm sure I was bothered by that
inconsistency before but couldn't put my finger on the culprit.

-- 
Thanks,
Maxim

Re: Status of ‘core-updates’

[Maxim Cournoyer]

Hello,

Ludovic Courtès  writes:

> Hi Steve,
>
> Steve George  skribis:
>
>> On 10 Apr, Ludovic Courtès wrote:
>
> [...]
>
>>> To be clear (but I guess it’s crystal clear to anyone who’s been around
>>> long enough :-)), what we need most is someone to keep track of changes,
>>> coordinate efforts, decide what goes in the branch and what’s postponed
>>> or moved to a separate branch, and send periodic (weekly) status updates
>>> over the course of a couple of months.  This can (and probably should)
>>> be done without doing any actual hacking on the branch.
>> (...)
>>
>> This sounds like something I can do:
>>
>> - track changes (in branch)
>> - co-ordinate blocking issues (in bug system)
>> - co-ordinate with people doing the actual work :-P
>> - send (periodic) weekly emails
>> - encourage shipping by minimising scope creep ;-)
>
> Awesome, thanks for volunteering!
>
>> Sounds like there's already agreement to revert the 'pkgconf' change
>> and push a new branch without them which becomes
>> 'core-updates'. Josselin on IRC I had the impression you were
>> working on that?
>
> I’m not sure what the situation is (I see Maxim just pushed changes on
> top of current ‘core-updates’, so maybe it’s OK?).

Since branches were merged in, I believe the problem we are facing at
the moment is librsvg failing its test suite with a segfault (!).  Could
be the glibc upgrade, or rust itself, I'm not sure.  I was trying to
upgrade librsvg, which needs an update anyway, but it pulls many rust
crates updates.  I'll get there eventually, if nobody beats me to it.

> Josselin, Maxim: could you explain what problems there are around
> pkgconf and what you would recommend?

Nothing in particular to point at the moment, but remaining problems
would manifest in the form of missing inputs, due to transitive libtool
dependencies causing overlinking and the new pkgconf being smart enough
to optimize away some previously captured link directives that would be
baked in the RUNPATH and sastify libtool overlinking needs.

The solution is to hunt the libtool .la files from the transitive
dependencies causing the problem and removing them. See commit
b6540bd285cbe5920ad379ddfc6256359ee7204c for an example.

-- 
Thanks,
Maxim

Should we include nss-certs out of the box?

[Maxim Cournoyer]

Hi,

It's been Guix policy to let people choose whether to install or not TLS
root certificates and which one to their machine.  While I applaud the
idea to have the users make a conscious decision about it, in practice I
suppose very few of us choose to *not* install any as that basically
breaks using web browsers, especially ones like IceCat which (by
default) ensures HTTPS is used on every page.

It apparently even makes it impossible to run 'guix pull', if I am to
believe bug#62026.

Should we do as in bug#62026 and have this package be part of the
recommended basic installation?  It'd be in the basic set of an
operating-system packages (via its default %base-packages set).  It
could still be manipulated via the Guix API (filtered out/replaced with
something else).

Is anyone opposed to having nss-certs in %base-packages?

-- 
Thanks,
Maxim

Re: guix --container is RAM hungry

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès  writes:

> Hi Edouard,
>
> Edouard Klein  skribis:
>
>> I'm a huge fan of guix --container, and I created a system to use those
>> by default for network services. But the VPS these services run on has
>> only 2GB of RAM, and I just realized that a container, by default,
>> requires at least 200MB.
>
> Ouch, confirmed:
>
> $ \time -v guix shell -C coreutils -- uname 2>&1 |grep 'Maximum resident'
> Maximum resident set size (kbytes): 283048
> $ \time -v guix shell coreutils -- uname 2>&1 |grep 'Maximum resident'
> Maximum resident set size (kbytes): 56588
> $ guix describe
> Generation 297  Mar 24 2024 23:12:25(current)
>   guix 28bc0e8
> repository URL: https://git.savannah.gnu.org/git/guix.git
> branch: master
> commit: 28bc0e870b4d48b8e3e773382bb0e999df2e3611
>
>
> As raingloom and Ricardo wrote, there’s a Guile process that keeps
> waiting.

Is there a technical reason for this?  Couldn't we replace the
current Guix process with 'exec', as hinted by Edouard?

If possible, that'd be the most direct way to avoid any of the memory
cost incurred by Guile/Guix.

-- 
Thanks,
Maxim

Re: Request to add a go-team branch and set it on CI.

[Maxim Cournoyer]

Hi Sharlatan,

Sharlatan Hellseher  writes:

> Hi,
>
> I've set the branch go-team, not checked adding it on CI.
>
> Is there any login precision to it?

If you have admin access to to https://ci.guix.gnu.org (see setup in
info '(cuirass) Authentication') using the TLS certificate I provided to
you (I think?), these actions (e.g. creating a go-team job specification
or restarting failed build, if they failed spuriously) should be doable
via the web interface.

-- 
Thanks,
Maxim

Re: Request to add a go-team branch and set it on CI.

[Maxim Cournoyer]

Hi,

Simon Tournier  writes:

> Hi,
>
> CC: guix-maintainers
> CC: guix-sysadmin
>
> On lun., 22 janv. 2024 at 19:58, Sharlatan Hellseher  
> wrote:
>
>> May I ask someone with admin rights to the build farm to set up
>>  go-team branch, please?
>
> What is the status of this request?  Is it doable?  I do not see the
> specification on .  Do I miss something?

IIRC, I think I had sent a private TLS client certificate to Sharlatan
so they can manage the go-team branch creation/jobs themselves from the
web UI.

-- 
Thanks,
Maxim

Re: (Lx)Qt team in Guix

[Maxim Cournoyer]

Hello,

宋文武  writes:

[...]

> Hello, I have just pushed 2 commits to remove qt.scm from the scope of
> lxqt and add myself to the qt team.
>
> Happy weekend!

Yay!  Thank you, and happy weekend to you as well!

-- 
Thanks,
Maxim

Re: (Lx)Qt team in Guix

[Maxim Cournoyer]

Hi Andreas,

+guix-devel

Andreas Enge  writes:

> Hello,
>
> I am reaching out since I am receiving in cc patches for Qt, and realise
> that I do not feel quite confident about them; I added myself because I
> feel able to work on C code, but I think you are much more involved and
> competent for Qt than I and also sufficiently numerous, so I could safely
> remove myself.
>
> But when trying to do so, I realised that the situation is a bit convoluted:
> In addition to a qt team, there is an lxqt team, and I was actually part
> only of the latter; but the two overlap in claiming responsibility for
> gnu/packages/qt.scm. Moreover Maxim is part of qt, and 宋文武 and I are
> part of lxqt.

I think the reason qt.scm is part of the lxqt team is historical; lxqt-team
predates qt-team.  We should probably just remove qt.scm from
lxqt-team's scope.  What do you think?

> So I would suggest to sort this out somehow; maybe merge lxqt into qt?

I'd keep both team separated; I'm interested in maintaining qt upgrades
but not much lxqt.

-- 
Thanks,
Maxim

Re: Bugs and Patches---or rather, bugs or patches?

[Maxim Cournoyer]

Hi,

Josselin Poiret  writes:

> Hi Felix,
>
> Felix Lechner via "Development of GNU Guix and the GNU System
> distribution."  writes:
>
>> Hi,
>>
>> We track bugs and patches separately.  Does the distinction serve a
>> purpose?  May I combine them?
>
> What do you mean by tracking them separately and combining them?
> They're on two different lists, right?  I don't like the separation
> between both MLs, sure, but that's it.  What would be the pros/cons of
> such a merge?

I like to be able to see a list of just reported bugs, without having to
use any filters.  Combining the trackers would make this impossible/more
difficult.

-- 
Thanks,
Maxim

Re: cmake-build-system: build tests only if #:tests? is true?

[Maxim Cournoyer]

Hi Hartmut,

Hartmut Goebel  writes:

> Hi,
>
> I found an old and unfinished patch in my pile. It optimizes building
> with cmake by not building the test if "#:tests?" is false. (Basically
> it passes -DBUILD_TESTING=OFF/ON" depending on "#:tests?".)
>
> Is this of interest? Then I would take my time and finish the patch.

Yes, this looks like a neat improvement.

-- 
Thanks,
Maxim

Re: LUKS2 support in Guix

[Maxim Cournoyer]

Hello,

Fabio Natali  writes:

> On 2024-03-02, 10:41 +0300, Oleg Pykhalov  wrote:
>> I also lost some time due to this incorrect statement.
>
> Brilliant, thanks Felix and Oleg.
>
> FWIW, micro-patch submitted here⁰.
>
> Let's see what the patch reviewer thinks. I'm not terribly happy about
> "downgrading" the manual's recommendation from LUKS2 to LUKS1, but if
> LUKS2 is really not supported, then of course that needs to be reflected
> in the docs.
>
> Best wishes, Fabio.
>
>
> ⁰ Ticket 69507 on debbugs.gnu.org. I wanted to link to the relevant page
> on https://lists.gnu.org/archive/html/guix-patches/2024-03/index.html
> but it seem it's taking a while for the mail to pass moderation.

Is there something preventing us from updating GRUB to benefit from full
LUKS2 support?

-- 
Thanks,
Maxim

Re: Supporting sssd, preparing for nscd sunset

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès  writes:

> Hello Guix!
>
> Distros are increasingly relying on sssd, in particular Fedora and
> derivatives, as a replacement for nscd, which is either unavailable or
> deprecated.  The documented interface of Guix binaries to the host’s
> name service switch (NSS) is currently nscd:
>
>   
> https://guix.gnu.org/manual/en/html_node/Application-Setup.html#Name-Service-Switch
>
> If sssd becomes dominant, we
> might just as well arrange to have NSS always load libnss_sss.so.
>
> Thoughts?

With nscd removed from glibc, can't we assume foreign distributions will
rely on sssd instead?  Do I understand correctly that the above
(configuring NSS to always load libnss_sss.so) would be a fine
replacement for our use of nscd?  If so, that seems the simplest/best
option to pursue to me.

-- 
Thanks,
Maxim

Re: Feedback of the GNU Guix manual

[Maxim Cournoyer]

Hi Matt,

Matt  writes:

>   On Wed, 21 Feb 2024 18:20:19 +0100  Maxim Cournoyer  wrote --- 
>
>  > Thanks for the follow-up.
>
> Thank you!  Seems like we were looking at it at about the same time :)
>
>  > Like Josselin, I prefer to keep the mention that the tarball archive
>  > includes the transitive dependencies of Guix (it's explicit; "archived
>  > binaries" is a bit vague to my taste).
>
> I tried to address that in the diff I wrote up before I saw your message.  I 
> agree, "archived binaries" is awkward.  I changed it in my update.
>
>  > I've pushed your adjusted suggestions with commit ec9c8b0c1a.
>  
> Cool.  I'll work on the next item in the list.  Please let me know if there's 
> something more regarding this item based on the v2 update.

Great, sounds good.  I've checked your v2 update, but opted to keep
things as they are (following my own edition of your initial work, which
was committed).

Thank you for your efforts!

-- 
Maxim

Re: Proposal to turn off AOT in clojure-build-system

[Maxim Cournoyer]

Hi Maxime!

Maxime Devos  writes:

>>On Thu, Feb 22 2024, Andreas Enge wrote:
>>> Am Thu, Feb 22, 2024 at 03:57:41PM +0100 schrieb Maxime Devos:
 Yes. It appears you are unfamiliar with (...)
 It also appears you are unfamiliar with (...)
>>>
>>> May I suggest to not make assumptions about what other people are familiar
>>> with or not? There is no point in claiming that others are less knowledge-
>>> able than you; they may know as much or even more than you
>>
>>Asking questions can avoid any such conclusions. It's my favorite
>>thing. Here, I might have asked: "Do you know about...?"
>
> OK. I don’t think I’ll do that.

Your sharp replies came off as harsh, which is unfortunate because you
seem to bring points of technical merits.  Please smooth out your
interactions/communication style, to ensure it remains pleasant for
everyone involved, taking these code of conduct items into account:

* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences

-- 
Thanks,
Maxim

Re: [Cuirass] JavaScript work

[Maxim Cournoyer]

Hi,

Ricardo Wurmus  writes:

> Hi,
>
> I noticed that Cuirass bundles minified JavaScript.  I’ve started a new
> branch that replaces the minified JavaScript with readable source code
> and minifies the files as part of the build.
>
> I also tried to remove the need for jQuery, at least in our own
> JavaScript code.  (Datatables.js still uses jQuery.)
>
> Eventually, I’d like to remove Bootstrap (both CSS and JS) all together
> and switch to Pico.css for conventional default styles without the need
> for bootstrap classes everywhere.  I’d like to remove JS for thing that
> HTML+CSS can do just fine.
>
> (Pico is also used in mumi.)
>
> What do you think?

Sounds good to me.  Like Janneke, I also think we can do without JS
obfuscation (minification).

-- 
Thanks,
Maxim

Re: Feedback of the GNU Guix manual

[Maxim Cournoyer]

Hi Matt,

Thanks for the follow-up.

Matt  writes:

[...]

> #+begin_src diff
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 4af830aed7..16349d4ec1 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -732,14 +732,16 @@ ready to use it.
>
>  @cindex installing Guix from binaries
>  @cindex installer script
> -This section describes how to install Guix on an arbitrary system from a
> -self-contained tarball providing binaries for Guix and for all its
> -dependencies.
>
>  This is often quicker than installing from source, which
> -is described in the next sections.  The only requirement is to have
> -GNU@tie{}tar and Xz.
> +This section describes how to install Guix from archived binaries.  Such
> +installations are often quicker than building from source, which is
> +described in the next sections.  Binary installations require a Linux or
> +Hurd-based system with GNU@tie{}tar and Xz.

Like Josselin, I prefer to keep the mention that the tarball archive
includes the transitive dependencies of Guix (it's explicit; "archived
binaries" is a bit vague to my taste).

As for the Linux or Hurd-based system, that seems like a good idea to
make things explicit.

I've pushed your adjusted suggestions with commit ec9c8b0c1a.

Thank you!

-- 
Maxim

Re: Golang mudules to follow common grouping

[Maxim Cournoyer]

Hi Sharlatan,

Sharlatan Hellseher  writes:

> Hi Guix!
>
> I've pushed split IV
> https://issues.guix.gnu.org/69042
>
> Now there are all base golang-* modules which I'm about to populate on demand
> during patch review.
>
> - golang-build
> - golang-check
> - golang-compression
> - golang-crypto
> - golang-web
> - golang-xyz
>
> These two may be added as well if I see enough packages related to that topic.
> - golang-graphics
> - golang-maths / golang-science

Great work!  Thank you for seeing it through!

-- 
Thanks,
Maxim

Re: Guix Days: Patch flow discussion

[Maxim Cournoyer]

Hi Clément,

Clément Lassieur  writes:


[...]

>>> I also agree! :-) What appears to me “difficult” is that most of the
>>> tools as Email client are poorly supporting Message-ID.
>>>
>>> For instance, debbugs.el (Gnus).  To my knowledge, there is not easy way
>>> to get the Message-ID when reading an article (bug/patch) from Debbugs.
>>> There is other means for applying patches.  But still each time appears
>>> to me weird. :-)
>
> May I add too, that you can add "Message-ID" in gnus-visible-headers.

Thanks, that sounds useful!

-- 
Thanks,
Maxim

Re: Guix Days: Patch flow discussion

[Maxim Cournoyer]

Hi Simon,

Simon Tournier  writes:

> Hi,
>
> On dim., 11 févr. 2024 at 11:38, Maxim Cournoyer  
> wrote:
>
>> 'b4 shazam' is probably the most trouble-free way to apply patches;
>
> I agree*!
>
>> it
>> even selects the latest revision it finds in the issue thread.  To make
>> finding a message-id easier, I've also recently added a 'Copy
>> Message-ID' button to the Mumi interface; try it visiting any issue,
>> e.g. <https://issues.guix.gnu.org/68990>.  The message-id of any message
>> can be easily copied to your clipboard via the new button.
>
> I also agree! :-) What appears to me “difficult” is that most of the
> tools as Email client are poorly supporting Message-ID.
>
> For instance, debbugs.el (Gnus).  To my knowledge, there is not easy way
> to get the Message-ID when reading an article (bug/patch) from Debbugs.
> There is other means for applying patches.  But still each time appears
> to me weird. :-)

I usually used C-u g to see the message source, then regexp-searched for
^Message-ID, but that's not exactly convenient.

-- 
Thanks,
Maxim

Re: Guix Days: Patch flow discussion

[Maxim Cournoyer]

Hi,

Kyle Meyer  writes:

> Hi Ricardo,
>
> Ricardo Wurmus writes:
>
>> Hi Josselin,
>
>>> They both can co-exist with debbugs, and for now the patchwork instance
>>> of QA is not usable for status tracking (because it is not meant to be
>>> used as such for now).  One can already use both of them, but using both
>>> supercedes debbugs, and gets rid of its limitations.  I've been using
>>> b4/lei with the yhetil public-inbox instance, with piem.el as an
>>> interface, and it's really useful.  With a properly configured b4, one
>>> could simply run `b4 shazam some-msg-id` and it would automatically
>>> apply the corresponding patchset.
>>
>> I’m interested in adopting this workflow.  Where can I find more
>> information on how to configure this?
>
> In 889a6204f8 (doc: Add some guidelines for reviewing, 2023-11-07),
> Maxim added some b4 configuration to Guix's etc/git/gitconfig that
> points at yhetil.org.  With that setup, running 'b4 shazam MESSAGE-ID'
> from a Guix checkout should work.

Indeed!  It should perhaps be more prominently documented, probably as
an new 'Reviewing patch work flows' section in the Guix Cookbook that
could be linked from our already lengthy Contributions section.

'b4 shazam' is probably the most trouble-free way to apply patches; it
even selects the latest revision it finds in the issue thread.  To make
finding a message-id easier, I've also recently added a 'Copy
Message-ID' button to the Mumi interface; try it visiting any issue,
e.g. .  The message-id of any message
can be easily copied to your clipboard via the new button.  From your
Guix checkout, you'd then run:

--8<---cut here---start->8---
$ b4 shazam 'cover.1707383694.git.h.goe...@crazy-compilers.com'
Looking up 
https://yhetil.org/guix/cover.1707383694.git.h.goe...@crazy-compilers.com
Grabbing thread from 
yhetil.org/guix/cover.1707383694.git.h.goe...@crazy-compilers.com/t.mbox.gz
Checking for newer revisions
Grabbing search results from lore.kernel.org
Nothing matching that query.
Analyzing 29 messages in the thread
---
  [PATCH 1/28] gnu: Add ruby-test-unit-ruby-core.
  [PATCH 2/28] gnu: Add ruby-excon.
  [PATCH 3/28] gnu: Add ruby-ipaddr.
  [PATCH 4/28] gnu: Add ruby-net-ftp.
  [PATCH 5/28] gnu: Add ruby-fake-ftp.
  [PATCH 6/28] gnu: Add ruby-net-sftp.
  [PATCH 7/28] gnu: Add ruby-net-telnet.
  [PATCH 8/28] gnu: Add ruby-pairing-heap.
  [PATCH 9/28] gnu: Add ruby-stringio.
  [PATCH 10/28] gnu: Add ruby-stream.
  [PATCH 11/28] gnu: Add ruby-rgl.
  [PATCH 12/28] gnu: Add ruby-sfl.
  [PATCH 13/28] gnu: Add ruby-specinfra.
  [PATCH 14/28] gnu: Add ruby-serverspec.
  [PATCH 15/28] gnu: Add ruby-time.
  [PATCH 16/28] gnu: Add ruby-google-protobuf.
  [PATCH 17/28] gnu: Add ruby-googleapis-common-protos-types.
  [PATCH 18/28] gnu: Add ruby-grpc.
  [PATCH 19/28] gnu: Add ruby-vagrant-cloud.
  [PATCH 20/28] gnu: Add ruby-vagrant-spec.
  [PATCH 21/28] gnu: Add ruby-vagrant-spec-helper-basic.
  [PATCH 22/28] gnu: Add ruby-hashicorp-checkpoint.
  [PATCH 23/28] gnu: ruby-childprocess: Update to 4.1.0.
  [PATCH 24/28] gnu: Add ruby-libvirt.
  [PATCH 25/28] gnu: Add ruby-fog-core.
  [PATCH 26/28] gnu: Add ruby-fog-json.
  [PATCH 27/28] gnu: Add ruby-fog-xml.
  [PATCH 28/28] gnu: Add ruby-fog-libvirt.
---
Total patches: 28
---
 Base: using specified base-commit a4464bd0975c811f18af98f69032b29bddda5b81
Application de  gnu: Add ruby-test-unit-ruby-core.
Application de  gnu: Add ruby-excon.
Application de  gnu: Add ruby-ipaddr.
Application de  gnu: Add ruby-net-ftp.
Application de  gnu: Add ruby-fake-ftp.
Application de  gnu: Add ruby-net-sftp.
Application de  gnu: Add ruby-net-telnet.
Application de  gnu: Add ruby-pairing-heap.
Application de  gnu: Add ruby-stringio.
Application de  gnu: Add ruby-stream.
Application de  gnu: Add ruby-rgl.
Application de  gnu: Add ruby-sfl.
Application de  gnu: Add ruby-specinfra.
Application de  gnu: Add ruby-serverspec.
Application de  gnu: Add ruby-time.
Application de  gnu: Add ruby-google-protobuf.
Application de  gnu: Add ruby-googleapis-common-protos-types.
Application de  gnu: Add ruby-grpc.
Application de  gnu: Add ruby-vagrant-cloud.
Application de  gnu: Add ruby-vagrant-spec.
Application de  gnu: Add ruby-vagrant-spec-helper-basic.
Application de  gnu: Add ruby-hashicorp-checkpoint.
Application de  gnu: ruby-childprocess: Update to 4.1.0.
Application de  gnu: Add ruby-libvirt.
Application de  gnu: Add ruby-fog-core.
Application de  gnu: Add ruby-fog-json.
Application de  gnu: Add ruby-fog-xml.
Application de  gnu: Add ruby-fog-libvirt.
--8<---cut here---end--->8---

and done!

-- 
Thanks,
Maxim

Re: TODO Delete branch snapper.

[Maxim Cournoyer]

Hi,

Nicolas Graves via "Development of GNU Guix and the GNU System
distribution."  writes:

> The snapper branch is here with no activity for quite some time.
>
> There's a patch for snapper in 57311, maybe we can start by deleting the
> branch. 

Done (the branch is now deleted).

-- 
Thanks,
Maxim

Re: bug#68920: Issues with issues.guix.gnu.org (502 Bad Gateway)

[Maxim Cournoyer]

Hi Christina,

Christina O'Donnell  writes:

> Hi Guix,
>
> From my machine[1] connecting to https://issues.guix.gnu.org/ results
> in, after 130 seconds[2], a 502 bad gateway. It's been having issues
> for over a week, but I only just found a need to test it.
>
> I couldn't see an issue about it on debbugs so I thought it prudent to
> raise an issue.
>
> Reproduced with curl, firefox, and ungoogled-chromium on the main page
> and on specific issues.
>
> It appears to be under a lot of load.

Thanks for the report.  It occurred a few times in the past weeks, where
the 'mumi' service had to be restarted on Berlin.  Let's keep this open
to see if it'll occur again.  Otherwise I'll close it in a week or two.

-- 
Thanks,
Maxim

Re: Mechanism for helping in multi-channels configuration

[Maxim Cournoyer]

Hi,

Simon Tournier  writes:

> Hi,
>
> Well, using Guix bdab356 from a little bit more than one month old, then
> associating the channel guix-science 0b3d4a2f last week, I get the
> failure:
>
> $ guix build /gnu/store/g3aa5rh7bs5pyxd3q1gvhwz1s9z1vh3z-guix-science.drv
> The following derivation will be built:
>   /gnu/store/g3aa5rh7bs5pyxd3q1gvhwz1s9z1vh3z-guix-science.drv
> building /gnu/store/g3aa5rh7bs5pyxd3q1gvhwz1s9z1vh3z-guix-science.drv...
> (repl-version 0 1 1)
> WARNING: (guix-science build bazel-build-system): imported module (guix build 
> utils) overrides core binding `delete'
> (exception unbound-variable (value #f) (value "Unbound variable: ~S") (value 
> (python-nr-stream)) (value #f))
> builder for `/gnu/store/g3aa5rh7bs5pyxd3q1gvhwz1s9z1vh3z-guix-science.drv' 
> failed to produce output path 
> `/gnu/store/qzgj4vig3vklbznz1i0pgy11nr3z4rv9-guix-science'
> build of /gnu/store/g3aa5rh7bs5pyxd3q1gvhwz1s9z1vh3z-guix-science.drv failed
> View build log at 
> '/var/log/guix/drvs/g3/aa5rh7bs5pyxd3q1gvhwz1s9z1vh3z-guix-science.drv.gz'.
> guix build: error: build of 
> `/gnu/store/g3aa5rh7bs5pyxd3q1gvhwz1s9z1vh3z-guix-science.drv' failed
>
> Well, that’s expected!  Guix bdab356 does not contain python-nr-stream
> introduced by commit 7dfe41aa71a4a4a9d6065a44e9c6271717215b3e.
>
> The wishlist is: provide a machine-readable description on guix-science
> channel side in order to help in finding the good overlap between
> commits of different channels.
>
> It could be nice if instead of an hard error, “guix pull” could say:
> « the channel ’guix’ needs to be at least at commit 1234abc ».

Anything is better than an obscure failure/backtrace, so I'd say it's a
good idea, especially if you are motivated to hack on it.

-- 
Thanks,
Maxim

Re: [shepherd] several patches that i deem ready

[Maxim Cournoyer]

Hi,

Zheng Junjie  writes:

> Maxim Cournoyer  writes:
>
>> Hi Attila,
>>
>> Attila Lendvai  writes:
>>
>>>> About "cheaper code path when a log level is disabled at runtime",
>>>> perhaps it can be improved in guile-lib, but otherwise that's a nice
>>>> list. I just wish we had a good logging library in Guile and could stop
>>>> reinventing the wheel left and right.
>>>
>>>
>>> i've made my judgement that the logger in guile-lib was never applied
>>> seriously when i relized that it stores the enabled state in a
>>> hashtable (which must be looked up for every log statement).
>>>
>>> i made sure the log statements have a unique syntax, so the underlying
>>> machinery can be replaced easily later, and then i moved on.
>>
>> I guess if we were serious at giving Scheme/Guile a great logging
>> library we would try to design it with the help of everyone as a SRFI;
>> it could then be integrated to Guile and any other Scheme, benefiting
>> the ecosystem at large.  Is this something you could be interested in
>> trying?  I've found the Scheme community around SRFIs very knowledgeable
>> and helpful.
>>
>
> maybe can see srfi-215 Central Log Exchange
> [https://srfi.schemers.org/srfi-215/srfi-215.html]?

That's somewhat related, but is not an actually usable logging library,
just some standard means to dispatch them to various sinks, if I
recall my read of it.

-- 
Thanks,
Maxim

Re: [bug#68606] role of core-updates

[Maxim Cournoyer]

Hi Andreas,

Andreas Enge  writes:

> Hello,
>
> Am Sat, Dec 09, 2023 at 11:33:54AM +0100 schrieb Andreas Enge:
>> Am Sat, Dec 09, 2023 at 11:16:14AM +0100 schrieb Ludovic Courtès:
>> > With that in mind, ‘core-updates’ would effectively become the branch of
>> > the ‘core-packages’ team: the branch where we update packages in these
>> > files (primarily the toolchain and Guile), perhaps also (guix build
>> > utils), and that’s all.
>> > How does that sound?
>> Sounds good, thanks to you and Maxim for thinking it through!
>
> is the current core-updates branch ready for building and merging?
> I am looking at an issue:
>https://issues.guix.gnu.org/68606
> that adds a patch (updating patchelf) that, as far as I understand, is
> already available on core-updates. So I feel somewhat blocked for the
> issue.
>
> The last merge was in spring of 2023, I think, and my patch updating wget
> is lingering in the branch since last July. So I am afraid we are reenacting
> the problems we had with the historic core-updates branch. It would be nice
> to merge and to move the branch to its new purpose.
>
> Also,
>https://issues.guix.gnu.org/65200
> from last August is blocked by a core-updates merge (it should probably go
> to the new-style core-updates branch, and would be the starting point of
> working on bootstrapping from a newer GMP).

Since patchelf is core material, if the rest of the series depend on
that update, it should go to core-updates as well.  Now is as good a
time as any, since work has picked to get it into a mergeable state.

-- 
Thanks,
Maxim

Re: [shepherd] several patches that i deem ready

[Maxim Cournoyer]

Hi Attila,

Attila Lendvai  writes:

>> About "cheaper code path when a log level is disabled at runtime",
>> perhaps it can be improved in guile-lib, but otherwise that's a nice
>> list. I just wish we had a good logging library in Guile and could stop
>> reinventing the wheel left and right.
>
>
> i've made my judgement that the logger in guile-lib was never applied
> seriously when i relized that it stores the enabled state in a
> hashtable (which must be looked up for every log statement).
>
> i made sure the log statements have a unique syntax, so the underlying
> machinery can be replaced easily later, and then i moved on.

I guess if we were serious at giving Scheme/Guile a great logging
library we would try to design it with the help of everyone as a SRFI;
it could then be integrated to Guile and any other Scheme, benefiting
the ecosystem at large.  Is this something you could be interested in
trying?  I've found the Scheme community around SRFIs very knowledgeable
and helpful.

>> OK. For levels greater than debug, they I see them as glorified
>> comments (executable comments as yo wrote), so I don't see a strong
>> reason to attempt to hide them or treat them specially. In Python
>> (which strives to be readable), we typically break logging lines (which
>> are concatenated for free inside the parens -- default Python behavior),
>> and that doesn't hurt readability in my opinion, and means we can just
>> follow the usual style rules, keeping things simple.
>
>
> my experience is different. i found myself only ever looking at log
> statements when i'm debugging something, regardless of the level, and
> including other people's code. and then i just toggle line wrap with
> the press of a button.
>
> must be related to my habit that i usually put more effort into making
> the code more self-documenting (readable) than i put into writing
> informal comments and documentation. and rethinking my "executable
> comment" metaphore: these log statements serve much less as comments
> than reporting the temporal state and program flow.
>
> but my primary aim is to color it all gray, and i don't immediately
> know how to do that in emacs for multiline sexp's (i.e. balanced
> parens). this is the primary reason our team just kept them on one
> line, but the flexibility of toggling word wrap as needed is also
> nice: the essetial part is always within a reasonable margin, and the
> rest can be read when word-wrap is enabled.

I'm sure there's a way; have you consulted in #emacs on Libera?

> if requested, then i'm willing to re-format the log statements if i
> can find a way to still color it all gray. it's important that logging
> stays out of sight while reading the code.

OK; I'll let other voices their preference (because it's down to that,
and current conventions).

>
>> Thanks for working on this, I'm sure it'll help many, myself included,
>> following the execution of Shepherd more easily.
>
>
> my pleasure!
>
> in my experience when a project doesn't have proper logging,
> backtraces, error handling hygene, and warning-free compilation, then
> inefficient debugging quickly eats up more time than it would take to
> implement these features properly.
>
> unfortunately, guix and guile is not very good on this front, so i
> found myself working on these, too. such investment rarely pays off
> for the first bug, but it pays off very well in the long run.

I wholly agree.  Sadly, I find the debugging facilities of Guile are
also lacking compared to what other programming languages have, which
further exacerbate that situation.

-- 
Thanks,
Maxim

Re: update darktable version

[Maxim Cournoyer]

Hi Alex,

Alex Devaure  writes:

> Hi all,
> There is a new version of the RAW developer darktable. The patch #67719
> modifies the generation of it by replacing clang with gcc. The patch is
> not yet merged, should I base my patch (updating darktable version) to
> #67719?

That'd be fine, yes.

-- 
Thanks,
Maxim

Re: Greetings!

[Maxim Cournoyer]

Hello!

Raghav Gururajan  writes:

> Hello Guix!
>
> Happy holidays and a new year everyone! I wish you all a continued
> success in your endeavour(s), health, and life.
>
> Regards,
> Raghav "RG" Gururajan.

Thanks Raghav!  I wish you a great 2024 as well.

-- 
Thanks,
Maxim

Re: How to answer all people in a thread?

[Maxim Cournoyer]

Hi Christian,

Christian Miller  writes:

> Let's see if this works.  Does everyone get an email?

Ack.  I use 'S W' in Gnus, which is the shortcut to send a wide reply
(it CCs everyone from the message being wide replied to).

-- 
Thanks,
Maxim

Re: [shepherd] several patches that i deem ready

[Maxim Cournoyer]

Hi Attila,

Attila Lendvai  writes:

> hi Maxim,
>
>> > > - a lightweight logging infrastructure together with plenty of log
>> > > lines throughout the codebase, and some hints in the README on how
>> > > to turn log lines gray in emacs (i.e. easily ignorable).
>>
>>
>> Are you using guile-lib's logging library for it? I've used it in
>> guile-hall if you want to have an example. We should maximize its
>> users, refine it and aim to have it builtin Guile at some point.
>
>
> i looked at that lib first (IIRC by your recommendation), but i ended
> up rolling my own for the cost of two additional pages of code in
> shepherd. i think the main issue i had was the amount of unconditional
> computation that happens on the common code path, and its complexity
> in general, including its API.
>
> shepherd has some non-trivial machinery regarding logging output being
> captured and redirected through sockets to herd and whatnot; i.e. most
> of the handler machinery in guile-lib's logger would be just an
> impedance mismatch instead of being helpful.

Since we can define a custom logger for guile-lib, I'm a bit surprised
by this, but it's true that this logging library is quite minimal/naive.

> for those two pages it's:
>  - one less external dependency
>  - less resource use
>  - more flexibility
>  - cheaper code path when a log level is disabled at runtime
>  - compile-time log level to drop entire log levels
>  - and most importantly much less code complexity.

About "cheaper code path when a log level is disabled at runtime",
perhaps it can be improved in guile-lib, but otherwise that's a nice
list.  I just wish we had a good logging library in Guile and could stop
reinventing the wheel left and right.

> you can find the relevant commit at:
>
> https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila
>
> FWIW, it's a partial bort of a CL lib of mine:
>
> https://github.com/hu-dwim/hu.dwim.logger
>
>
>> > a quick note on the log statements: they are essentially noise when it
>> > comes to reading the code, hence the gray coloring i suggest in
>> > emacs. (although they may often serve also as "executable" comments).
>>
>> > i'd also like to propose to relax the 80 column limit for log lines
>> > for the same reason that i've explained above.
>>
>>
>> I don't think an exception is deserved here; the logging library from
>> guile-lib for example concatenates message strings by default, which
>> makes it easy to brake long messages on multiple lines.
>
>
> my ultimate goal here is to minimize the disruption of code readability. only 
> some emacs (editor) magic and/or code formatting can help with that.
>
> log lines are only relevant when you're debugging something, or when you're 
> trying to understand a codebase. all other times they are essentially noise.
>
> my proposal is what our CL team settled with: always one line per log 
> statement, and setting the foreground color of the entire line gray in emacs.

OK.  For levels greater than debug, they I see them as glorified
comments (executable comments as yo wrote), so I don't see a strong
reason to attempt to hide them or treat them specially.  In Python
(which strives to be readable), we typically break logging lines (which
are concatenated for free inside the parens -- default Python behavior),
and that doesn't hurt readability in my opinion, and means we can just
follow the usual style rules, keeping things simple.

Thanks for working on this, I'm sure it'll help many, myself included,
following the execution of Shepherd more easily.

-- 
Thanks,
Maxim

Re: Golang check phase skipping some tests?

[Maxim Cournoyer]

Hi Sharlatan,

Sharlatan Hellseher  writes:

> Hi Maxim,
>
> Thank you for detailed replay.
>
>> The branch workflow for teams is to use a *-team branch that is short
>> lived, e.g. for the time needed to do the integration work; with an
>> associated job spec in Cuirass (ci.guix.gnu.org) to build it.
>
> Am I ok to push new go-team branch? And I'll push patches related to
> split v3 soon this week.

I'm not in the go-team, so the safest bet to get everyone onboard would
be to send your series to guix-devel first via 'git send-email'; go team
members will be notified.

After a two weeks (or before that if you get it reviewed by other
members), you could then merge to go-team and have it built.

-- 
Thanks,
Maxim

Re: [core-updates] Native build of make-boot0 fails on missing zstd

[Maxim Cournoyer]

Hello,

Janneke Nieuwenhuizen  writes:

[...]

> As mentioned on IRC, this fixes it for me; thanks!

Great!  I'm tentatively including many other patches I had lined up for
the next world rebuild that I've picked from guix-patches, to turn this
wasted computing cycles into an opportunity.

-- 
Thanks,
Maxim

Re: [core-updates] Native build of make-boot0 fails on missing zstd

[Maxim Cournoyer]

Hi Janneke,

Janneke Nieuwenhuizen  writes:

> Maxim Cournoyer writes:
>
> Hi,
>
>> Efraim Flashner  writes:
>>
>>> On Sun, Jan 21, 2024 at 10:33:37AM +0100, Janneke Nieuwenhuizen wrote:
>>>> Hi!
>>>> 
>>>> On core-updates, running
>>>> 
>>>> ./pre-inst-env guix build --system=i586-gnu -e '(@@ (gnu packages 
>>>> commencement) gnu-make-boot0)'
>>>> 
>>>> fails for me with
>>>> 
>>>> sh: zstd: command not found
>>>> 
>>>> See log below.  FWIW, using --system=i686-linux for example, works fine.
>>
>> I don't know what it'd only affect non-x86 systems, but I've noticed one
>> bug in my recent "default to zstd" repack logic, which would name a xz
>> tarball as .tar.zst.  Perhaps tar then try to use zstd to decompress it,
>> even if it's really a xz compressed archive?
>
> Ah, that could be...

My previous diff included yet another bug... here's one that seems to
work fine:

--8<---cut here---start->8---
modified   guix/packages.scm
@@ -949,10 +949,7 @@ (define* (patch-and-repack source patches
  (bzip2   (lookup-input "bzip2"))
  (lzip(lookup-input "lzip"))
  (xz  (lookup-input "xz"))
- (zstd(or (lookup-input "zstd")
-  ;; Fallback to xz in case zstd is not available, such as
-  ;; for bootstrap packages.
-  xz))
+ (zstd(lookup-input "zstd"))
  (patch   (lookup-input "patch"))
  (comp(and=> (compressor source-file-name) lookup-input))
  (patches (map instantiate-patch patches)))
@@ -1033,10 +1030,13 @@ (define* (patch-and-repack source patches
   locale (system-error-errno args)
 
 (setenv "PATH"
-(string-append #+zstd "/bin"
-   (if #+comp
-   (string-append ":" #+comp "/bin")
-   "")))
+(string-join
+ (map (cut string-append <> "/bin")
+  ;; Fallback to xz in case zstd is not
+  ;; available, such as for bootstrap packages.
+  (delete-duplicates
+   (filter-map identity (list #+zstd #+xz #+comp
+ ":"))
--8<---cut here---end--->8---

I'll push this to core-updates along a list of cherry-picked changes
from our tracker destined for core-updates, if it tests good on your
side.

-- 
Thanks,
Maxim

Re: [shepherd] several patches that i deem ready

[Maxim Cournoyer]

Hi Attila,

Attila Lendvai  writes:

>> - a lightweight logging infrastructure together with plenty of log
>> lines throughout the codebase, and some hints in the README on how
>> to turn log lines gray in emacs (i.e. easily ignorable).

Are you using guile-lib's logging library for it?  I've used it in
guile-hall if you want to have an example.  We should maximize its
users, refine it and aim to have it builtin Guile at some point.

>
> a quick note on the log statements: they are essentially noise when it
> comes to reading the code, hence the gray coloring i suggest in
> emacs. (although they may often serve also as "executable" comments).
>
> i'd also like to propose to relax the 80 column limit for log lines
> for the same reason that i've explained above.

I don't think an exception is deserved here; the logging library from
guile-lib for example concatenates message strings by default, which
makes it easy to brake long messages on multiple lines.

-- 
Thanks,
Maxim

Re: [core-updates] Native build of make-boot0 fails on missing zstd

[Maxim Cournoyer]

Hello,

Efraim Flashner  writes:

> On Sun, Jan 21, 2024 at 10:33:37AM +0100, Janneke Nieuwenhuizen wrote:
>> Hi!
>> 
>> On core-updates, running
>> 
>> ./pre-inst-env guix build --system=i586-gnu -e '(@@ (gnu packages 
>> commencement) gnu-make-boot0)'
>> 
>> fails for me with
>> 
>> sh: zstd: command not found
>> 
>> See log below.  FWIW, using --system=i686-linux for example, works fine.

I don't know what it'd only affect non-x86 systems, but I've noticed one
bug in my recent "default to zstd" repack logic, which would name a xz
tarball as .tar.zst.  Perhaps tar then try to use zstd to decompress it,
even if it's really a xz compressed archive?

I have the following commit that fixes this, haven't pushed it yet (it's
a world rebuild).

--8<---cut here---start->8---
modified   guix/packages.scm
@@ -949,10 +949,7 @@ (define* (patch-and-repack source patches
  (bzip2   (lookup-input "bzip2"))
  (lzip(lookup-input "lzip"))
  (xz  (lookup-input "xz"))
- (zstd(or (lookup-input "zstd")
-  ;; Fallback to xz in case zstd is not available, such as
-  ;; for bootstrap packages.
-  xz))
+ (zstd(lookup-input "zstd"))
  (patch   (lookup-input "patch"))
  (comp(and=> (compressor source-file-name) lookup-input))
  (patches (map instantiate-patch patches)))
@@ -1033,10 +1030,16 @@ (define* (patch-and-repack source patches
   locale (system-error-errno args)
 
 (setenv "PATH"
-(string-append #+zstd "/bin"
-   (if #+comp
-   (string-append ":" #+comp "/bin")
-   "")))
+(string-join (fold (lambda (x path)
+ (if (and x (not (member x path)))
+ (cons x path)
+ path))
+   '()
+   ;; Fallback to xz in case zstd is
+   ;; not available, such as for
+   ;; bootstrap packages.
+   (list #+zstd #+xz #+comp))
+ ":"))
 
 (setenv "ZSTD_NBTHREADS" (number->string (parallel-job-count)))
--8<---cut here---end--->8---

-- 
Thanks,
Maxim

Re: Golang mudules to follow common grouping

[Maxim Cournoyer]

Hi Christina,

Christina O'Donnell  writes:

> Hi Oleg,
>
> On 13/01/2024 21:05, Sharlatan Hellseher wrote:
>> Hi Guix,
>>
>> I'm about to prepare split and aggregation of all golag packages
>>  related to cryptography. The process would be the same as for
>>  golang-check and golang-web.
>>
>>
>> In progress:
>> golang-cryptography
>>
>> Planned:
>> golang-compression
>> golang-build
>
> I just wanted to offer my labor to this task. I'm very new to guix
> development
> (I've only packaged one package so far), but I've got a fair amount of
> free time
> and I've been looking for ways to contribute to Guix. (Plus I've
> already got my
> head in golang.scm trying to package gitleaks and all its dependencies.)

Thanks for offering to help!  The most important part in doing so is to
synchronize with the other members of the Go team to avoid duplicating
the work, I think.

> I know that there's a lot to do already, but I was also wondering whether
> they're going in any particular order once they're moved. There's a
> comment at
> the end of golang.scm that says,
>
>> ;;; Avoid adding new packages to the end of this file. To reduce the
>   chances
>> ;;; of a merge conflict, place them above by existing packages with
>   similar
>> ;;; functionality or similar names.
>
> Would it be more organized if they was just one order: either in
> alphabetical
> order or grouped by function? My suggestion would be to use the file
> split to
> group by function and then sort each file alphabetically. Do you know
> how it is
> arranged for other languages?

That'd be neat, but in practice is hard to maintain as the module
becomes quite large and people easily (and understandably) miss the
intended grouping when adding new packages.

> Another question I have: Is there any tooling that can help big package
> migrations like this go faster? Eg. a script to split one big diff into
> individual package moves with appropriate change-log entries.

Not that I'm aware of.

-- 
Thanks,
Maxim

Re: 07/07: gnu: Add python-docspec.

[Maxim Cournoyer]

Hi Ricardo,

Ricardo Wurmus  writes:

[...]

>> This commit and a few others mention there are no tests; perhaps that's
>> true of the PyPI archive, but in this case it's worth fetching from Git
>> in my opinion to run the tests.  Here for example, there appear to be a
>> Pytest test suite:
>> https://github.com/NiklasRosenstein/python-docspec/tree/develop/docspec/test
>>
>> Fetching from git and adding pytest no native-inputs may be enough to
>> have it run.
>>
>> Something to keep in mind for future submissions!
>
> Yes, I agree.  These recent Python additions are a little special in
> that they had been misplaced and forgotten in the guix-science channel.
> I’ve recently taken some time to clean up the guix-science channel and
> move things that don’t belong there to the default Guix channel,
> reviving and updating packages as I went along.

I understand, thanks for doing this!

-- 
Thanks,
Maxim

Re: Golang check phase skipping some tests?

[Maxim Cournoyer]

Hi Sharlatan,

Sharlatan Hellseher  writes:

> Hi Maxim,
>
> You mentioned go-team branch which I tried to find :-)
>
> Is there any formal procedure to push new branches?
> I might need a branch to push changes from the split task
> instead of sending patches.

The branch workflow for teams is to use a *-team branch that is short
lived, e.g. for the time needed to do the integration work; with an
associated job spec in Cuirass (ci.guix.gnu.org) to build it.

Patches should still ideally be reviewed on the guix-patches mailing
list before merging to the branch, especially if multiple people are
working on the branch.

A request for merge message is then sent to guix-patc...@gnu.org as
explained in (info "(guix) Managing Patches and Branches"), and we wait
for the branch to be scheduled and built by QA, which should hopefully
catch any problems with it before it's merged.

At least that's the spirit; when the QA is lagging too much behind and
nothing happens for a long time, we can fallback to a heavier local
testing as we were doing previously, back when we didn't have these
handy tools.  I use some shell procedures you can find here [0] that may
be of use to build dependent packages and such.

[0]  
https://notabug.org/apteryx/guix-api-examples/src/master/command-line-hacks.sh

-- 
Thanks,
Maxim

Re: Feedback of the GNU Guix manual

[Maxim Cournoyer]

Hi Matt!

Matt  writes:

> I care about this.  How can I help?
>
> - Convert the notes into patches?
> - Proofread any patches that derive from Christian's efforts?

That would be a good way to help, yes!  It'll be easier to review if
each distinct problem is fixed in its own commit (patch).

-- 
Thanks,
Maxim

Re: Golang mudules to follow common grouping

[Maxim Cournoyer]

Hi Oleg,

Sharlatan Hellseher  writes:

> Hi Guix,
>
> I'm about to prepare split and aggregation of all golag packages
>  related to cryptography. The process would be the same as for
>  golang-check and golang-web.
>
>
> In progress:
> golang-cryptography
>
> Planned:
> golang-compression
> golang-build

This sounds like a good idea to me!  Thanks for volunteering to do this
laborious task.

-- 
Maxim

Re: Golang check phase skipping some tests?

[Maxim Cournoyer]

Hi Oleg,

Sharlatan Hellseher  writes:

> Hi,
>
> I can't say that I'm an expert in Golang :-), but I've got some experience in
> building and deploying Glang daily for some time.
>
> First things first the default behaviour of `go test ./...` is like this:
>
> find * -type f -name *_test.go | 
>
>
> Internally Golang tries to find any files with _test.go suffix in
> provided module's
> location and evaluate defined tests in them. If the directory of the module 
> does
> not have test files it's just ignored.
>

[...]

> I'm currently in review and split some packages from (gnu packages golang) 
> into
> (gnu packages golang-crypto) to simplify the maintenance. I try to play with
> that option and see which packages are missed to satisfy passing all tests.

A simple thank you for doing this!

-- 
Maxim

Re: 07/07: gnu: Add python-docspec.

[Maxim Cournoyer]

Hi,

I'm not in the Python team, but I thought I'd give some feedback on
recent Python packages added.

guix-comm...@gnu.org writes:

> gnu: Add python-docspec.
> 
> * gnu/packages/python-xyz.scm (python-docspec): New variable.
> 
> Change-Id: I3103bde3483273a335156b38de742f493fd366f1
> ---
>  gnu/packages/python-xyz.scm | 25 +
>  1 file changed, 25 insertions(+)
>
> diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
> index 61958d5eac..58a1a2b3cd 100644
> --- a/gnu/packages/python-xyz.scm
> +++ b/gnu/packages/python-xyz.scm
> @@ -622,6 +622,31 @@ Python dataclasses.")
>  from JSON payloads using the @code{databind.core} framework.")
>  (license license:expat)))
>  
> +(define-public python-docspec
> +  (package
> +(name "python-docspec")
> +(version "2.2.1")
> +(source
> + (origin
> +   (method url-fetch)
> +   (uri (pypi-uri "docspec" version))
> +   (sha256
> +(base32 "1zqmdrc6k8pprra8p3wpzq2ml2gph1cfjmsyg07f8b8fvizffm28"
> +(build-system pyproject-build-system)
> +(arguments (list #:tests? #false))  ;there are none

This commit and a few others mention there are no tests; perhaps that's
true of the PyPI archive, but in this case it's worth fetching from Git
in my opinion to run the tests.  Here for example, there appear to be a
Pytest test suite:
https://github.com/NiklasRosenstein/python-docspec/tree/develop/docspec/test

Fetching from git and adding pytest no native-inputs may be enough to
have it run.

Something to keep in mind for future submissions!

-- 
Thanks,
Maxim

Re: Commit Access: Sharlatan Hellseher

[Maxim Cournoyer]

Hi!

Kyle Meyer  writes:

> Hi Maxim,
>
> Maxim Cournoyer writes:
>
>> Another easy option is to retrieve the Message-ID of any message in the
>> series (via the source HTML of the mail archives, or directly from the
>> mail headers if you have the mail locally), and then use B4, Linux
>> style [0].  Example: suppose I wanted to apply and review this Perl
>> series [1], then I could apply locally to my Guix git checkout with:
>
> I was going to add that a key part of using b4 with public-inbox
> instances outside of the default lore.kernel.org is setting b4.midmask.
> But then I noticed that Guix's etc/git/gitconfig now has
>
> [b4]
> attestation-check-dkim = off
> attestation-policy = off
> linkmask = https://yhetil.org/guix/%s
> linktrailermask = https://yhetil.org/guix/%s
> midmask = https://yhetil.org/guix/%s
>
> Neat.  Thanks for adding that!

This auto-config thing is powerful :-).  Perhaps we should add a word or
two about b4 and other workflows to our cookbook.  I also use patman for
managing submissions with ease (info "(u-boot) Patman patch manager").

-- 
Thanks,
Maxim

Re: Golang check phase skipping some tests?

[Maxim Cournoyer]

Hi Tomas,

Tomas Volf <~@wolfsden.cz> writes:

> On 2024-01-14 22:12:38 +0100, Troy Figiel wrote:
>> Hi everyone,
>> 
>> When looking into the Go build system, I noticed the default check phase
>> runs (invoke "go" "test" import-path), which only runs the tests in the
>> root directory of the source. Running the tests in all subdirectories
>> would require something like this instead:
>> 
>> --8<---cut here---start->8---
>> (define* (check #:key tests? import-path #:allow-other-keys)
>>   "Run the tests for the package named by IMPORT-PATH."
>>   (when tests?
>> (invoke "go" "test" (string-append import-path "/...")))
>>   #t)
>> --8<---cut here---end--->8---
>> 
>> For example, if the -v flag is added (which might be a better default?)
>> to the check phase of go-github-stretchr-testify, it can be seen that
>> only `TestImports' runs, none of the tests in assert, http, etc.
>> However, the source code in these subdirectories is still recursively
>> copied to out during the install phase.
>> 
>> Is this desired behaviour? I assumed it isn't, because it looks like we
>> are skipping a lot of tests during the check phase. However, I might
>> also simply be overlooking something here as I am new to packaging
>> Golang with Guix.
>
> I tend to agree (on both accounts).  I am not aware of anything Guix specific
> here that would prevent it, but maybe someone from golang team will chime in.

I'm not much of a Go person, but your suggestions sound reasonable.
They'd have to be tested on the go-team branch to see whether it breaks
many things (it probably would, since many more tests would be run).

-- 
Thanks,
Maxim

Re: Using the pyproject-build-system

[Maxim Cournoyer]

Hi,

Troy Figiel  writes:

> Hi Guix/Python team,
>
> My fix for python-requests-kerberos was pushed today (thanks Oleg!) and
> I thought it would be an appropriate moment to ask about the
> pyproject-build-system. In short, is the pyproject-build-system a
> preferable default over the python-build-system? The manual states
> "experimental", but "encouraged to try it", leaving me wondering which
> one to use when.
>
> And in long:
>
> Although not fully PEP 517-compliant according the documentation, the
> pyproject-build-system does seem to fall back to setuptools.build_meta
> if the pyproject.toml is missing. Contrary to what the name implies to
> me, it can therefore also be used for packages with only a setup.py file.
>
> This usually leads to slicker definitions, since quite a few packages
> seem to only use pytest as their testing suite and in these cases the
> pyproject-build-system does not require an override of the check phase.
>
> Should I therefore always try to use the pyproject-build-system instead
> of the python-build-system, or is there some different guideline to
> follow?

I think pyproject-build-system is often more convenient, such as for
running the test suite, where it automatically figures out whether
pytest or something else should be run.

It's fine to use it as a replacement for python-build-system until it
becomes the default, it you want!

-- 
Thanks,
Maxim

Re: Guix wiki

[Maxim Cournoyer]

Hi Attila,

Attila Lendvai  writes:

>> 1. People find the [data] service provides value (can someone restate what 
>> that
>> value is exactly? Is it needed e.g. to power
>
>
> if you allow hijacking the above into the wiki discussion:
>
> this is a good example where a wiki page (central, easily editable,
> capturing the current state) would tremendously help this
> discussion. who, where, why, what, etc...

This assumes there would be people willing to write information there
and maintain it up to date, which is far from granted.  Other than that,
I get your point.

-- 
Thanks,
Maxim

Re: Commit Access: Sharlatan Hellseher

[Maxim Cournoyer]

Hi,

Efraim Flashner  writes:

> On Fri, Jan 12, 2024 at 05:21:51PM +0100, Clément Lassieur wrote:
>> On Thu, Jan 11 2024, Sharlatan Hellseher wrote:
>> 
>> > Hi Guix!
>> >
>> > I am happy to have been granted commit access and I am ready to help
>> > review pending issues and prepare queued packages for GNU packages in
>> > astronomy. I would like to concentrate on the packages covered by the
>> > Go, Lisp, Python, and Science teams.
>> >
>> > I would like to thank the Guix team for allowing me to become a
>> > committer member. I am looking forward to continuing our collaboration.
>> >
>> > If anyone has a good patch review workflow using Emacs, Gnus, and Magit,
>> > I would appreciate it ;-)

Another easy option is to retrieve the Message-ID of any message in the
series (via the source HTML of the mail archives, or directly from the
mail headers if you have the mail locally), and then use B4, Linux
style [0].  Example: suppose I wanted to apply and review this Perl
series [1], then I could apply locally to my Guix git checkout with:

--8<---cut here---start->8---
b4 shazam 
'a6492643b9e32409b8420e7d54ba290d0937b838.1705443194.git.felix.lechner@leaseup.com'
--8<---cut here---end--->8---

It should get the latest revisions if there are, and apply the full
series in one go.

[0]  https://b4.docs.kernel.org/en/latest/maintainer/am-shazam.html
[1]  https://lists.gnu.org/archive/html/guix-patches/2024-01/msg01180.html
-- 
Thanks,
Maxim

Re: An update on ‘core-updates’

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès  writes:

> Hello Guix!
>
> Several of us have been fiddling with the ‘core-updates’ branch for a
> while.  I think there’s now consensus that the branch is really
> dedicated to core packages and (guix build …) modules, as embodied in
> the new ‘core-packages’ team¹.
>
> We’ve updated GCC 11.x, glibc, binutils, and various packages from (gnu
> packages base).  Notable exceptions are Coreutils, Findutils, sed, and
> tar; I tried but that’s a bit more work, notably because their variants
> in commencement.scm would no longer build because their build scripts
> use sed patterns not supported by Gash-Utils.
>
> Long story short: I’d like us to freeze and merge the branch ASAP,
> notably because the glibc graft on ‘master’ leads to a bad user
> experience.  I’m happy with the current state of the branch and wouldn’t
> mind postponing remaining upgrades for the next cycle.
>
> Thoughts?

Sounds good!

> Remaining work includes: checking that cross-compilation targets still
> work after the recent Binutils updates, checking i586-gnu (GNU/Hurd) and
> other platforms, and possibly addressing the Gawk non-determinism
> issue².

There's a non-determinism issue in Python that is supposed to be fixed
in bug#68394; we should apply it if possible.

-- 
Thanks,
Maxim

Re: Building and caching old Guix derivations for a faster time machine

[Maxim Cournoyer]

Hi Simon,

Simon Tournier  writes:

> Hi Maxim,
>
> On Thu, 30 Nov 2023 at 08:28, Maxim Cournoyer  
> wrote:
>
>> I'd like to have a single archive type as well in the future, but I'd
>> settle on Zstd, not lzip, because it's faster to compress and
>> decompress, and its compression ratio is not that different when using
>> its highest level (19).
>
> When running an inferior (past revision), some past Guile code as it was
> in this past revision is launched.  Hum, I have never checked: the
> substitution mechanism depends on present revision code (Guile and
> daemon) or on past revision?
>
> Other said, what are the requirements for the backward compatibility?
> Being able to run past Guix from a recent Guix, somehow.

We're only impacting the future, not the past, I think.  The inferior
mechanism still relies on the same daemon, as far as I know, and the
currently available gzipped nars would remain available according to
their current retention policy (6 months when unused).

>>>  1. Keep for as longer as we can all the requirements for running Guix
>>>  itself, e.g., “guix time-machine”.  Keep all the dependencies and all
>>>  the outputs of derivations.  At least, for all the ones the build farms
>>>  are already building.
>>>
>>>  2. Keep for 3-5 years all the outputs for specific Guix revision, as
>>>  v1.0, v1.1, v1.2, v1.3, v1.4.  And some few others.
>>
>> That'd be nice, but not presently doable as we can't fine tune retention
>> for a particular 'derivation' and its inputs in the Cuirass
>> configuration, unless I've missed it.
>
> That’s an implementation detail, a bug or a feature request, pick the
> one you prefer. ;-)

I'd say it's a feature request :-).

> We could imagine various paths for these next steps, IMHO.  For
> instance, we could move these outputs to some specific stores
> independent of the current ones (ci.guix and bordeaux.guix).  For
> instance, we could have “cold” storage with some cooking bakery for
> making hot again, instead of keeping all hot.  For instance, we could
> imagine etc. :-)
>
> Well, I do not have think much and I just speak loud: Cuirass (and Build
> Coordinator) are the builders, and I would not rely on them for some NAR
> “archiving“, instead maybe “we” could put some love into the tool
> nar-herder.  Somehow, extract specific NAR that the project would like
> to keep longer than the unpredictable current mechanism.

It seems the nar-herder would perhaps be well suited for this, if
someone is inclined to implement it, given it keeps each nars in a
database, which should make it fast to query for all the 'guix' packages
substitutes.  Perhaps it even has (or could have) hooks when registering
a new nars which could define what is done to it (send to another
server).

Otherwise good old 'find' could be used to rsync the 'guix' named nars
and their .narinfo metadata files to a different location, but that'd
probably be less efficient (IO-intensive) on the huge multi-terabytes
collection of nars we carry.

-- 
Thanks,
Maxim

Re: When is check-system run?

[Maxim Cournoyer]

Hi Tomas,

Tomas Volf <~@wolfsden.cz> writes:

> Hello,
>
> I would like to inquire regarding system tests.  There is check-system make
> target, however it seems like at least some of them are broken.  Can someone
> check whether that is case even on their machine?  Do they all pass for you?

The system tests are run for new Guix revisions periodically by this job
set: https://ci.guix.gnu.org/jobset/tests

Otherwise we depend on people touching e.g. services running them
manually to ensure they pass.  Accumulated test failures that have
slipped in (we do not have a CI infrastructure gating merges on test
failures yet) should ideally be reported and fixed (thanks for working
on it!).

There is typically an effort to ensure all (or most) tests pass before a
new release is made.

> I already have patches fixing 4 of them, and debugging 5th, but I would like 
> to
> ask when exactly are the those tests run and where are the failures
> reported to?

Thank you!  You can report the failures to bug-g...@gnu.org, if they
weren't already; these are Guix bugs.

> Since the breakage sneaked into the master, I assume they are not run as part 
> of
> the regular patches?  Are there any measures the project is planning to take 
> to
> prevent this from happening again?

It'd be nice to have our CI infrastructure act more like a CI (currently
it's more of a build farm than a CI), but I don't know of anyone working
on it (though Christopher may have long term plans toward that goal with
QA [0]).

[0] qa.guix.gnu.org/

-- 
Thanks,
Maxim

Welcome Oleg (sharlatan) as a new committer

[Maxim Cournoyer]

Hello Guix!

I'm happy to start the 2024 year with a new committer onboard: Sharlatan
(Oleg).

Sharlatan is maintaining a growing collection of astronomy packages in
Guix, among others.

Let's Wish them a warm welcome!

-- 
Thanks,
Maxim

Proposition to streamline our NAR collection to just zstd-compressed ones

[Maxim Cournoyer]

Hello Guix, and Happy New Year!

It's been on my head for quite a bit of time (about 2 years, according
to [0]), to streamline our offering of cached nars.  Letting go of gzip
2 years ago, along a more aggressive garbage collection policy allowed
us to reduce our storage needs by at least 6.5 TiB.  I'm proposing to do
the same with our lzip compressed nars, to let go of an additional 3.9
TiB:

--8<---cut here---start->8---
$ du -sh /var/cache/guix/publish/{lzip,zstd}
3.9T/var/cache/guix/publish/lzip
4.1T/var/cache/guix/publish/zstd

$ find /var/cache/guix/publish/lzip -name '*.nar' | wc -l
4484645
$ find /var/cache/guix/publish/zstd -name '*.nar' | wc -l
4461195
--8<---cut here---end--->8---

The above suggests that zstd compressed nars are about 5% larger than
the lzip ones, which is not big enough to justify carrying both, in my
opinion.  In exchange for a little bit more bandwidth, users would have
the nars decompressed much faster with less CPU overhead locally.

Having our complete nars collection fit in around 4 TiB would also open
the door for simple rsync-based mirroring, which I have started working
on.

What do you think?  Should we go ahead and effect the following simple
change for the Berlin build farm?

--8<---cut here---start->8---
modified   hydra/modules/sysadmin/services.scm
@@ -683,7 +683,7 @@ to a selected directory.")
;; 

;; for the compression ratio/decompression speed
;; tradeoffs.
-   (compression '(("lzip" 9) ("zstd" 19)))
+   (compression '(("zstd" 19)))
(cache-bypass-threshold cache-bypass-threshold)
(workers publish-workers)))
--8<---cut here---end--->8---

-- 
Thanks,
Maxim

Re: Discontinuing data.guix.gnu.org?

[Maxim Cournoyer]

Hi,

Ludovic Courtès  writes:

> Hello Christopher,
>
> Christopher Baines  skribis:
>
>> Ludovic Courtès  writes:
>
> [...]
>
>>> Christopher Baines  skribis:
>>>
 As previously set out, I'm planning to stop hosting the data service
 instances this year. While I would like to stop hosting the server for
 data.guix.gnu.org,
>>>
>>> I forgot the outcome of previous discussions, but it seems to me that
>>> the service itself and all the data it accumulated over the years are
>>> super valuable.  I would be sad to see it go!
>>
>> There was a discussion back in April, but no action came directly from
>> it.
>>
>> Just having data.qa.guix.gnu.org was discussed, and at least
>> concentrating on getting to a sustainable hosting situation there seemed
>> like a sensible priority. The longer history provided by
>> data.guix.gnu.org does have value though in my view.
>>
>>> Is there something we can do to not lose it all?  It could be
>>> distributing responsibility, reducing the scope, ensuring hosting is
>>> managed collectively by the project, etc.  WDYT?
>>
>> Since that discussion, I have disabled the database dumps and backups,
>> which has reduced (to 62€ per month) the hosting costs (although
>> obviously not having backups isn't ideal). It's possible to further
>> reduce the hosting costs as well by switching away from a VM to a
>> physical machine at Hetzner.
>>
>> But yeah, given that having at least one data service instance is a key
>> part of keeping the bordeaux build farm running, managing the hosting
>> through the project seems to be the way to go. I'm just not sure how we
>> can get there, or what I can do to move things in that direction.
>
> Like you, I would have hoped for more reactions.  Unfortunately, I’m not
> offering to help here because I have more than enough on my plate and
> even a 1+ month backlog on guix-devel…
>
> Maintainers, what’s your take on this?

I don't have a particularly clear view of the matter, but observing
that:

1. People find the service provides value (can someone restate what that
value is exactly?  Is it needed e.g. to power
https://qa.guix.gnu.org/patches ?)
2. Are willing to help with paying for it.

Given the above, it seems to me that the ball is in the camp of the Guix
Foundation to sort how to transfer ownership of the Hetzner
accounts/payments from Christopher to them.

It could be interesting, as Gabor mentioned, to review if a machine
already part of our fleet (Milano, perhaps?) could be used for it, to
keep our operating costs low, but this can be done later.

My 2 cents,

-- 
Thanks,
Maxim

Re: How to answer all people in a thread?

[Maxim Cournoyer]

Hi,

Yarl via "Development of GNU Guix and the GNU System distribution."
 writes:

> I think 'In-Reply-To' and 'References' headers are important.

The safest/best way is to use 'reply all'; this preserves filtering
using Return-Path working as expected for the recipients.

-- 
Thanks,
Maxim

Re: How to answer all people in a thread?

[Maxim Cournoyer]

Hi,

Yarl via "Development of GNU Guix and the GNU System distribution."
 writes:

> I think 'In-Reply-To' and 'References' headers are important.

The safest/best way is to use 'reply all'; this preserves filtering
using Return-Path working as expected for the recipients.

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Apoorv  writes:

> OK, it was my bad I had an older copy of guix.scm file I was passing
> that, it didn't have cargo-build-system. But now I'm getting so many
> other errors related to cargo deps missing or version mismatch, I
> tried adding those deps spent like 1-2 hours but something dep was
> still missing or version mismatch, finally I gave up. I'm not sure how
> many deps I have to add like this.. this is getting way to complicated
> then it needs to be.

cargo/rust is poorly integrated in Guix, as there's some impedance
mismatch between package handling by Guix and Cargo itself, which would
rather take care of it.

I'd expect most Rust developpers using Guix only do so to provide
Rust/Cargo, and let Cargo take care of the rest.  Otherwise you'd need
to replicate the pain the cargo-build-system goes through to make it
work.

A more future-proof approach if you'd like to improve Rust integration
into Guix would be to continue integrating the 'antioxydant' build
system effort started by Maxime Devos [0], which doesn't rely on Cargo
to build Rust crates (and supports dynamic linking of Rust libraries, I
think?).

[0]  https://notabug.org/maximed/cargoless-rust-experiments

-- 
Thanks,
Maxim

Outreachy, Google Summer of Code Guix participation

[Maxim Cournoyer]

Hi Guix,

The FSF has reached out to let us know that signing up and submission
for Outreachy is starting this month [1] and that from January 22 to
February 6, 18:00 UTC mentoring organizations can submit applications
for Google Summer of Code.

Would someone like to suggest projects or support potential
participants?  The FSF also offered to help if we need anything.

[1]  https://www.outreachy.org/communities/cfp
[2]  https://developers.google.com/open-source/gsoc/timeline

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Apoorv  writes:

> I can add rust:cargo but the package definition in the guix.scm is already 
> using `cargo-build-system` shouldn't it include rust and cargo etc 
> automatically?

It should, but if it doesn't you may want to review how rust:cargo input
gets added to the cargo-build-system; perhaps it's done in a ad-hoc
fashion that causes that problem.

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Apoorv  writes:

> Sorry, this is my first time using some mailing list.
>
> I tried using `--container` and `--emulate-fhs` first it doesn't seem
> to use `guix.scm` automatically I have to specify it using
> `--file=guix.scm`. Second, the container is missing so much more stuff
> like cargo for example.

Yes, that's a somewhat surprising behavior designed to avoid bad
surprises, in that if you start using arguments with 'guix shell', the
'do what I mean' magic is cancelled and you need to explicit each
arguments.

About cargo missing, what prevents you from adding rust:cargo to your
guix.scm file?

> This is my own project in Rust lang, I basically define `guix.scm` for
> all my projects since I started using Guix.

I guess this being rust everything is statically compiled, and the usual
dynamic loader is not useful here (the RUNPATH baked in elf binaries is
not used).

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Apoorv  writes:

> There are a lot of libs that need to be added to LD_LIBRARY_PATH for
> my project to compile successfully. I do have gcc-toolchain added in
> the guix.scm file. I had to do a lot of trickery for the shell
> environment to work for me properly, using direnv,

[...]

Did you try the 'guix shell --container --emulate-fhs' trick I suggested
earlier?  I guess it should just work.  What is it that you are
building/attempting to run?

Also, please keep guix-devel@gnu.org and participants in CC, so that
everyone can follow/participate (the easiest way to do so is to "wide
reply" from your email client).

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Tobias Geerinckx-Rice  writes:

> Hiyall,
>
> On 29 December 2023 03:58:27 UTC, Maxim Cournoyer
>  wrote:
>>Guix doesn't/shouldn't make use of LD_LIBRARY_PATH, except in rare cases
>>to wrap binaries.  It's better to patch the dlopen calls to use the
>>absolute shared library file name.
>
> Fully agree.
>
> Furthermore, '--ld-library-path' sounds like an inflexible alias of a
> nonexistent '--ad-hoc-search-path=FOO=/bar' option.
>
> So why not instead write a cute little (name "LD_LIBRARY_PATH") fake
> package that does nothing but declare a search path for /lib?  It
> could live in any channel rather than further fatten 'guix shell'.
>
>>Perhaps you are missing the package configuring LIBRARY_PATH and other
>>useful environment variables for finding libraries?  That'd be
>>gcc-toolchain, if I recall correctly.
>
> I might be mistaken but I assumed this was more about running
> 'pre-existing' (cough cough nudge nudge) software not built from
> source.

Ah, if this is the case, the 'guix shell --container --emulate-fhs' may
help, by providing some FHS layout with libraries under e.g. /usr/lib/.

-- 
Thanks,
Maxim

Re: guix installation why internet connection required?

[Maxim Cournoyer]

Hi,

v...@mail-on.us writes:

> x86 x64 gnu guix system 1.4.0 iso requires internet connection in order to get
> installed. Same goes for i686 iso.
>
> Why is that so? Why is there no
> iso option for installing off line? Thanks.

There's this ticket about the same: #43049.  If I remember correctly it
may be related by the Guix binary inside the ISO image being from the
Guix package of the Guix used to generate it, perhaps.

That seems like a tricky problem to solve.

-- 
Thanks,
Maxim

Re: Suggestion for a guix shell feature.

[Maxim Cournoyer]

Hi,

Apoorv via "Development of GNU Guix and the GNU System distribution."
 writes:

> I want to make a suggestion for a feature for guix shell.
>
> I was recently writing guix.scm files for my projects and was having
> problem with the libraries I added, not being found inside the
> shell. After struggling for hours I finally learned that that the
> LD_LIBRARY_PATH  is not being updated automatically inside the  guix
> shell environment so the libraries can be found easily.
>
> So my feature suggestion is that maybe we can have a flag or a option
> for guix shell, something like --add-ld-path or --update-ld-path for
> example, to add all libraries specified in the guix.scm or even as
> sub-command guix shell --add-ld-path mesa as an example, to
> automatically update the LD_LIBRARY_PATH environment variable to
> contain the correct paths to these libraries so they can be found
> easily inside the shell.

Guix doesn't/shouldn't make use of LD_LIBRARY_PATH, except in rare cases
to wrap binaries.  It's better to patch the dlopen calls to use the
absolute shared library file name.

Perhaps you are missing the package configuring LIBRARY_PATH and other
useful environment variables for finding libraries?  That'd be
gcc-toolchain, if I recall correctly.

-- 
Thanks,
Maxim

Re: rust-team branch merged

[Maxim Cournoyer]

Hi Efraim,

Efraim Flashner  writes:

> The rust team is pleased to announce that the rust-team branch has been
> merged back into master.  There are 570 commits across the branch.
> Cross-compiling support for the cargo-build-system was added, including
> for librsvg.  Cross-compiling was tested for (nearly) all architectures
> supported in `guix build --list-targets`.  Upstream rust added
> 'i686-unknown-hurd-gnu' as a target in rust-1.74, so that will need to
> wait until another time.  In the meantime, x86_64-w64-mingw32 at least
> compiles correctly and tests well under wine.
>
> Notable notes:
>
> * New rust version is 1.73.
> * 4001 packages use the cargo-build-system
> * Rust crates produced in the 'package' phase of the cargo-build-system
> should now be reproducible (for real this time)
> * Packages added: eza, kibi, libgit2-1.6, libgit2-1.7, spotifyd, stgit-2
> * Packages updated: alfis, rust-analyzer, rust-cargo-c
> * Rust-analyzer is built from the rust sources now, updating it to 1.73.
> It can be installed as 'rust-analyzer' or as 'rust:tools'
> * Cross-compiling support for the cargo-build-system was added

Sounds good!  Many thanks for working toward a better Rust in Guix.

-- 
Thanks,
Maxim

Re: Why bash-minimal is part of sbcl package

[Maxim Cournoyer]

Hi Felix,

Felix Lechner  writes:

> Hi Maxim,
>
> On Tue, Dec 12 2023, Maxim Cournoyer wrote:
>
>> it means bash-minimal should be explicitly added to the inputs,
>> otherwise when cross-building the package for another architecture the
>> native bash captured wouldn't be executable on the target.
>
> The core-updates branch has a lot of new 'bash-minimal' additions,
> presumably for cross-building. Maybe this is a good opportunity to ask
> about something I have been working on.
>
> As part of a broader effort to make development in scripting language
> like GNU Guile easier, I rewrote the executable wrapper so that it uses
> Guile instead of Bash.  My initial hope was that the Guile wrapper could
> use /run/current-system/profile/bin/guile (and thereby avoid an explicit
> Bash prerequisite) but it is only available on Guix System.

Yes, the problem with that would be portability when using anything but
a Guix System machine.

> My Guile wrapper will eventually reach you as part of a proposal to
> migrate Guix from the Automake build system to my very own Bespoke,
> which is likewise written in Guile. Meanwhile, please let me know if
> there is any use for a Guile wrapper (instead of Bash). Thanks!

Sounds fun!  You may be interested in looking at 'wrap-script', which
already leverages Guile for use in wrappers.

-- 
Thanks,
Maxim

Re: bug#67790: New signing key

[Maxim Cournoyer]

Hi,

Maxim Cournoyer  writes:

> Hi,
>
> Leo Famulari  writes:
>
>> Hello,
>>
>> I'm changing my Guix signing key from
>> B0515948F1E7D3C1B98038A02646FA30BACA7F08 to
>> 68407224D3A64EE53EAC6AAC1963757F47FF.
>>
>> Patches to follow. Testing is appreciated!
>
> Thanks for the heads-up!

Note that I believe you can simply update to your new key yourself.
You'll want to add your new key to the keyring branch, then adjust the
.guix-authorizations file with its new keygrip.

Your new key will become mandated after your .guix-authorizations change
is pushed.

-- 
Thanks,
Maxim

Re: New signing key

[Maxim Cournoyer]

Hi,

Leo Famulari  writes:

> Hello,
>
> I'm changing my Guix signing key from
> B0515948F1E7D3C1B98038A02646FA30BACA7F08 to
> 68407224D3A64EE53EAC6AAC1963757F47FF.
>
> Patches to follow. Testing is appreciated!

Thanks for the heads-up!

-- 
Thanks,
Maxim

Re: Why bash-minimal is part of sbcl package

[Maxim Cournoyer]

Hi,

Pan Xie  writes:

> Hello
>
> I find this interesting thing but I don't have an explanation. When
> query the "references" of my Gnu Store item "sbcl", it shows that sbcl
> references bash-mininal, as the following output shows:

[...]

> So the question is, which part of sbcl's package definition tells Guix
> it need to add bash-minimal as part of sbcl? Is there a practical
> method to figure that out?

A bunch of core commands such as bash, sed, make etc. are implicitly
included in the build environment by the GNU build systems (which most
other build systems simply extend).

So if there's a bash shebang somewhere, the patch-source-shebang phase
may substitute it to the *native* implicit bash command found in the
environment, thus keeping a reference to it.

That's a bug, as it means bash-minimal should be explicitly added to the
inputs, otherwise when cross-building the package for another
architecture the native bash captured wouldn't be executable on the
target.

-- 
Thanks,
Maxim

Re: Shutting down qa.guix?

[Maxim Cournoyer]

Hi,

Christopher Baines  writes:

> Tobias Geerinckx-Rice  writes:
>
>> Christopher Baines 写道：
>>> it's not the most cost effective setup
>>
>> Has this been explained in more detail before?
>
> Probably not, beid is currently a CPX51 Hetzner cloud server costing
> €65.33 a month. This has been useful as it's enabled scaling the
> resources dynamically, but it would be possible to reduce the costs and
> still have sufficient RAM/disk space by using a Hetzner server auction
> machine for example.
>
> It's not all about cost though, given the data service is one of the
> slow points of QA, if we want QA to get faster at giving feedback, it's
> probably important to not try and cut costs on this part of the system.

Isn't QA mostly slow because of the lack of x86 build machines?  Does
the head node needs to be powerful itself?  What kind of resources does
it likes having the most?  CPU?  RAM?  Storage?

-- 
Thanks,
Maxim

Re: issues.guix.gnu.org seems stop updating

[Maxim Cournoyer]

Hello,

Andy Tai  writes:

> https://issues.guix.gnu.org/recent, for example, only shows issues up to Dec 
> 6.
>
> Not sure if this is due to some data services stopping running or such...

Perhaps the machine was restarted and the manual rsync job not
restarted?  We still don't have a service for it (bug #59180).

CC'ing Ricardo, who is typically the person restarting that job
manually.

-- 
Thanks,
Maxim

Re: Divvying up service definitions

[Maxim Cournoyer]

Hi Bruno,

Bruno Victal  writes:

> Hi Efraim,
>
> On 2023-11-09 07:15, Efraim Flashner wrote:
>> I assume the define-maybe's aren't public, so I'd guess that shouldn't
>> cause a problem as long as they aren't exported.
>
> They're not public but they override definitions within the same file
> if more than one (define-maybe foo) is present (e.g. when (prefix bar-)
> is used)
>
>> Or moving the define-maybes to the top of the file and reusing them
>> between services. Is that a possibility?
>
> Due to their non-hygienic nature and the (prefix foo-) argument this
> can't work. IMO we should look into replacing this define-maybe business
> with something like SRFI-189 (by integrating it into Guile).

Haven't given much thought to the idea, but I've recently tried my hand
at adding new SRFIs to Guile, and developed some tooling such as a
'snarfi' script to aid with converting its HTML spec to Texinfo doc;
you'll find the result at:

and the script at
.

I hope these are useful in your (or someone's) quest to add SRFI 189 or
others.

-- 
Thanks,
Maxim

Re: Building and caching old Guix derivations for a faster time machine

[Maxim Cournoyer]

Hi Guillaume,

Guillaume Le Vaillant  writes:

> Maxim Cournoyer  skribis:
>
>> Hi Simon,
>>
>> Simon Tournier  writes:
>>
>>> Hi,
>>>
>>> On mer., 22 nov. 2023 at 19:27, Ludovic Courtès  wrote:
>>>
>>>> For long-term storage though, we could choose to keep lzip only (because
>>>> it compresses better).  Not something we can really do with the current
>>>> ‘guix publish’ setup though.
>>>
>>> It looks good to me.  For me, the priority list looks like:
>>
>> I'd like to have a single archive type as well in the future, but I'd
>> settle on Zstd, not lzip, because it's faster to compress and
>> decompress, and its compression ratio is not that different when using
>> its highest level (19).
>
> Last time I checked, zstd with max compression (zstd --ultra -22) was
> a little slower and had a little lower compression ratio than lzip with
> max compression (lzip -9).
> Zstd is however much faster for decompression.

I think when we talk about performance of NARs, we mean it in the
context of a Guix user installing them (decompressing) more than in the
context of the CI producing them, so zstd beats lzip here.

> Another thing that could be useful to consider is that lzip was designed
> for long term storage, so it has some redundancy allowing fixing/recovering
> a corrupt archive (e.g. using lziprecover) if there has been some bit
> rot in the hardware storing the file.
> Whereas as far as I know zstd will just tell you "error: bad checksum"
> and will have no way to fix the archive.

That's an interesting aspect of lzip, but in this age of CRC-check file
systems like Btrfs, we have other means on ensuring data integrity (and
recovery, assuming we have backups available).

I'm still of the opinion that carrying a single set of zstd-only NARs
makes the most sense in the long run.

-- 
Thanks,
Maxim

Re: what is the status of the core-updates now

[Maxim Cournoyer]

Hi,

Maxime Devos  writes:

> Op 23-11-2023 om 13:02 schreef Z572:
>> Maxime Devos  writes:
>> 
>>> [[PGP Signed Part:Undecided]]
>>>
>>>
>>> Op 21-11-2023 om 18:21 schreef Maxime Devos:
   > [PATCH] gnu: ephemeralpg: Fix cross-compilation.
 There is already a patch for that:
 
 and (a rebased version of) it effectively has already been applied:
 
>> ok, close it.
 I guess you could copy it from c-u to master if you don't want to
 wait for the c-u merge, but please use the search function at
 :
>> what is the status of the core-updates now,it on ci.guix.gnu.org
>> only build
>> 'core', is this a configuration error?
>
> I don't know.

The idea was to build only a little of it while it's still receiving
world rebuilding changes.

I haven`t been able to dedicate much time to it recently, but I`ll get
back to it eventually.

-- 
Thanks,
Maxim

Re: Building and caching old Guix derivations for a faster time machine

[Maxim Cournoyer]

Hi Simon,

Simon Tournier  writes:

> Hi,
>
> On mer., 22 nov. 2023 at 19:27, Ludovic Courtès  wrote:
>
>> For long-term storage though, we could choose to keep lzip only (because
>> it compresses better).  Not something we can really do with the current
>> ‘guix publish’ setup though.
>
> It looks good to me.  For me, the priority list looks like:

I'd like to have a single archive type as well in the future, but I'd
settle on Zstd, not lzip, because it's faster to compress and
decompress, and its compression ratio is not that different when using
its highest level (19).

>  1. Keep for as longer as we can all the requirements for running Guix
>  itself, e.g., “guix time-machine”.  Keep all the dependencies and all
>  the outputs of derivations.  At least, for all the ones the build farms
>  are already building.
>
>  2. Keep for 3-5 years all the outputs for specific Guix revision, as
>  v1.0, v1.1, v1.2, v1.3, v1.4.  And some few others.

That'd be nice, but not presently doable as we can't fine tune retention
for a particular 'derivation' and its inputs in the Cuirass
configuration, unless I've missed it.

-- 
Thanks,
Maxim

Re: Better support remote deployment

[Maxim Cournoyer]

Hi,

Tomas Volf <~@wolfsden.cz> writes:

[...]

> I was following the debate, but maybe I have missed something, so I will put
> forward my use case as well.  I have two properties I would like to have from 
> a
> "remote deploy" mechanism, and they are not satisfied by SSH-ing to the 
> machine
> and running the commands there.
>
> 1. No need for `guix pull'.  I run (a little) custom Guix, so running `guix
> pull' is somewhat expensive.  I do not want to do it just to deploy something
> (be it a system, or home, or profile).
>
> 2. No need to copy over the definitions.  My configurations are part of a git
> repository, which I would like to keep local to just my development machine.  
> So
> rsync-ing it over to the remote machine in order to run command there is
> something I would like to avoid.
>
> If, however, I can build a profile locally, `guix copy' it over, and *then* 
> SSH
> into the machine and somehow activate the profile (be it home or regular
> profile), that would work for me.  Is that currently possible?

That's possible if the remote machine runs Guix System and you use 'guix
deploy'.  Otherwise, if the remote machine is a foreign distribution,
you'd have to manually copy things with 'guix copy', like Ricardo has
been doing and wanting to improve here.

-- 
Thanks,
Maxim

Re: Turning off tests leads to a different store item

[Maxim Cournoyer]

Hi,

Simon Tournier  writes:

> Hi Maxim,
>
> On Thu, 09 Nov 2023 at 10:04, Maxim Cournoyer  
> wrote:
>
>>> I agree.  On a side note, one of the issue is the time of some tests.
>>> Sometimes, packaging is frustrating: build takes ages, then you fix some
>>> tests, think it will be good, re-launch “guix build”, another test
>>> failing, repeat.  It could nice to be able to cache the result of the
>>> build phase.
>>
>> I usually end up working in the /tmp/guix-build-* failed build of a
>> package for these situations; then I don't need to rebuild the whole
>> thing for each test suite run, and can test changes directly without
>> proper patches while working toward a fix.
>
> Do you know all the command lines equivalent for each phase run by all
> the build systems? ;-)

It depends of the build system, but for gnu-build-system, it's typically
just sourcing environment-variables then running 'make check', I think.
Note that 'live' environment variables are captured in the
'environment-variables' file, which is handy.

> Well, this workflow seems appealing but it never works for me on
> concrete situations.  Most of the time, my issue does not come from
> “what to do from command line” but “how to do it inside the Guix
> recipe”.
>
> For example, this test suite adjustment:
>
>   (add-before 'install 'disable-network-tests
> (lambda _
>   (substitute* "test/runtests.jl"
> (("\"async.jl") "# \"async.jl")
> (("\"client.jl") "# \"client.jl"))

[...]
> (("@testset.*Set-Cookie casing.*" all)
>  (string-append all "return\n"
>
> To have the correct replacement rules, I almost never get it right at
> the very first try, and the feedback loop is poor.  In some case, the
> replacement is done after 'unpack, so I can just kill the build process
> and check inside /tmp/guix-build-* if the output is the expected one.

I've used 'guix repl' in a failed build before, running the same
commands directly in the tree to observe their effects (e.g.,
substitutions) directly.

-- 
Thanks,
Maxim

Re: Upgrading Guix's security team

[Maxim Cournoyer]

Hi,

Ludovic Courtès  writes:

[...]

> Yes, we definitely need a rotation here!  I for one have my name there
> but regardless of my interest, I have to admit that I’ve been unable to
> be sufficiently responsive.  It’s time to let new folks take
> responsibility.
>
> I think we should make this a fixed-term position, to make it easier for
> people to commit to actually being active when needed, with the
> understanding that it’s not a commitment for life.
>
>> - currently we are not on the OS security distribution contact list:
>> ; this
>> had been discussed before but we will need commitment from people
>>
>> - clear roles will be helpful; to me this includes at least a couple
>> of people to coordinate (the majority of security issues will be
>> handled through package upgrades/grafts) and people to help review
>> and/or contact needed experts, like for Guix internal issues; we
>> should make this more precise
>
> We could distinguish security issues in packages provided by Guix from
> security issues in Guix itself.
>
> That said, the security team could redirect things to members of the
> “core” team for security issues in Guix itself; maybe we don’t need to
> formally separate the two.
>
>> - likewise, a clear fixed timeframe for who is on this team; keeping
>> people fresh and engaged for what can suddenly be a time sensitive and
>> critical job; I think this will also help spread institutional
>> knowledge for better security practices in general
>
> +1!
>
>> - members need not be experts but should be active in the community as
>> committers (already a round of vetting), familiar with what issues and
>> processes may arise, and willing to learn; perhaps we need a list of
>> experts to consult though the current teams are a good starting point
>
> +1
>
>> - what are your thoughts? what are the goals and outcomes we as a
>> distro want in security?
>>
>> - finally, I think an internal discussion with maintainers and long
>> time active committers would be helpful to get the improvements
>> started and moving, in addition to this wider discussion here
>>
>> And to get things started, I'm happy to volunteer myself to help
>> coordinate on security, if deemed okay by our current security team,
>> maintainers, and anyone else that's been helping to handle security. A
>> coordinating role with a term of say 6 months to a year? Happy to
>> provide more information and discuss here or privately; in short I'm
>> not a security expert but have time and bandwidth to keep things
>> moving and want to learn.
>
> Thank you for getting the ball moving!
>
> I’m all for having you on board and, to set an example, to leave as you
> join.
>
> If maintainers agree (Cc’d), I invite you to add your name and a
> termination date to the security page, remove my name, and subscribe to
> guix-security.  We should add a term for other people on the team too.
>
> How does that sound?

Sounds good to me!

-- 
Thanks,
Maxim

Re: Turning off tests leads to a different store item

[Maxim Cournoyer]

Hi Simon,

Simon Tournier  writes:

[...]

>> I think the lower fruits are in looking at making the test suite of the
>> few common offenders more robust (using libfaketime or the likes) to
>> prevent (re)occurrences of time bombs in the future.
>
> I agree.  On a side note, one of the issue is the time of some tests.
> Sometimes, packaging is frustrating: build takes ages, then you fix some
> tests, think it will be good, re-launch “guix build”, another test
> failing, repeat.  It could nice to be able to cache the result of the
> build phase.

I usually end up working in the /tmp/guix-build-* failed build of a
package for these situations; then I don't need to rebuild the whole
thing for each test suite run, and can test changes directly without
proper patches while working toward a fix.

-- 
Thanks,
Maxim

Re: Better support remote deployment

[Maxim Cournoyer]

Hi,

Efraim Flashner  writes:

[...]

>> 'guix package' is already the command we use to create profiles; maybe
>> it could accept a '--remote' argument to operate on a remote machine?
>> Or is this not what 'GUIX_DAEMON_SOCKET=ssh://your-target guix package
>> -m manifest.scm' can already accomplish? :-)
>
> Currently there's no ssh connection caching, so using GUIX_DAEMON_SOCKET
> can be quite slow, I normally allow 5-10 minutes for a profile. That
> works best when you can't use guix copy to send derivations over.
>
> The profile effectively already exists, it doesn't need to be computed
> on each machine.  It does seem more like deploy: take this profile, send
> it to that machine, and activate it there.  `guix deploy` already takes
> a file, the file could have a profile (or home config?) and a location
> for the profile. And a user.

That's some idea I suggested earlier (add integration form 'guix home'
configs to an operating system definition).  That has merit on its own,
but wouldn't solve Ricardo's use case, which is to deploy a profile on a
*foreign* distribution (not Guix System).

-- 
Thanks,
Maxim

Re: Turning off tests leads to a different store item

[Maxim Cournoyer]

Hi,

Simon Tournier  writes:

> Hi,
>
> On Wed, 8 Nov 2023 at 20:20, Saku Laesvuori  wrote:
>
>> There is another way: simply preventing the tests from changing the
>> resulting store item. For example, the package could first be built
>> without tests and then that build tree could be copied to the build tree
>> of the build with tests enabled.
>
> Somehow, the store would need to keep all the build intermediary
> artifacts produced, right?  For instance, consider the extreme case
> where the build phase produces .o artifact files and the tests for
> whatever reasons relies on these temporary artifacts.
>
> Well, we had a quick chat with Josselin and Andreas about separating
> the tests at https://hpc.guix.info/events/2023/workshop/program/
> And my understanding of the rough conclusion we had: it is not easy
> and the evil are in all the details.  For example autotools: somehow
> "make check" is connected in one way or the other to "make" and/or
> "make install".  Somehow, the complete build tree (with intermediary
> artifacts as .o) should keep in the store.
>
> From a pragmatical point of view, there is packages where the tests
> cannot be totally separated from from the temporary build, therefore
> the question seems: how do these cases compare to the other regular
> cases?  What is the ratio?  Is the rule about many corner cases
> without a clear "regular"?  Or are they just few corner cases?

One easy-ish way, which would be kind of ugly because coupled to the
specific file system capabilities (e.g Btrfs), would be to leverage CoW
features and create a Btrfs snapshot at the beginning of the test suite,
and reverting to it after it's run.

But even that is not fullproof; that'd only protect the build directory,
say, not the store location (some check phases are moved after
installation).

I agree it looks tricky to get it right (and even trickier to prove/test
for it) :-).

I think the lower fruits are in looking at making the test suite of the
few common offenders more robust (using libfaketime or the likes) to
prevent (re)occurrences of time bombs in the future.

-- 
Thanks,
Maxim

Re: Better support remote deployment

[Maxim Cournoyer]

Hello,

Felix Lechner  writes:

> Hi,
>
> On Tue, Nov 07 2023, Maxim Cournoyer wrote:
>
>> Then your suggestion to extend 'guix package' to be able to
>> install a profile from a store profile sounds useful, or perhaps a new
>> 'guix deploy-profile' (or a better name / sub-command?) that would
>> automate the copy and installation in one step.
>
> We have a lot of sub-commands already. Could 'guix deploy' instead be
> fed something other than a 'machine' record?

Perhaps too much of an overload of that command; not sure.

> How about a 'remote-user-profile'?

'guix package' is already the command we use to create profiles; maybe
it could accept a '--remote' argument to operate on a remote machine?
Or is this not what 'GUIX_DAEMON_SOCKET=ssh://your-target guix package
-m manifest.scm' can already accomplish? :-)

-- 
Thanks,
Maxim

Re: Patch Review Flow

[Maxim Cournoyer]

Hi jgart,

"jgart"  writes:

> Hi Guixers,
>
> Does anyone follow this workflow for reviewing patches?
>
> git clone https://git.guix-patches.cbaines.net/guix-patches/
> git checkout issue-x
> git format-patch ...
> # then in the development checkout of Guix:
> git am ...; make; ./pre-inst-env guix build
>
> Should we document it in the manual?

There are many flows... perhaps we should illustrate them via examples
in the Guix Cookbook manual?

> Does anyone follow a workflow that is more efficient than the above?

You can use the 'b4' tool to apply a series via a message's Message-ID
that is from that series.  Or you can use Emacs-Debbugs patch buffer
(which is powered by Gnus), ensure your current directory for the main
buffer is set to your Guix checkout (e.g. 'M-x cd RET ~/src/guix'), then
apply the patches pressing the '|' to pipe it to 'git am -3' command
with it.

-- 
Thanks,
Maxim

Re: Divvying up service definitions

[Maxim Cournoyer]

Hi Bruno,

Bruno Victal  writes:

> Hi,
>
> As the gnu/services and gnu/home/services grow, I think we should
> consider divvying the services into stand-alone modules or
> subdirectories.
>
> Consider the ⌜dovecot-service-type⌝ in gnu/services/mail.scm: as of
> commit 'd22d2a05c389207f8cdcf824be7738b1499a987c' this service
> definition is nearly 1600 lines long, with the remainder of the file
> comprising of four other services with rudimentary support.
>
> It becomes troublesome working with such amalgamations as it makes it
> hard to keep track of the used modules and bindings, especially when
> define-configuration is used since the serializing procedures might be
> used by various service definitions. Further complicating things is
> 'define-maybe', whose use monopolizes the predicate and serializers for
> a particular service definition.
>
> Now, I'm not saying that we should go and split everything into its own
> module, I'm saying that we should be allowed to split some of them if
> convenient (all subjective but I believe we can see that working with a
> monolithic file in the kilolines where the interactions aren't obvious
> is not fun, and that's without bringing in the hygienic issues
> surrounding define-configuration and define-maybe).
>
> Some considerations (using dovecot-service-type as an example):
> * Splitting this as gnu/services/mail/dovecot.scm.
>   We preserve the logical grouping of the services (with the addition
>   that, for extremely comprehensive definitions, these can be neatly
>   organized into subdirectories. (same structure seen with gnu/*.scm)
>   A drawback is that 'use-service-modules' might not work with this
>   although I wonder whether 'use-service-modules' & co. provide any
>   value if we are already doing '(use-modules (gnu) …)' to begin with.
>   They look redundant IMO.
>
> * Splitting this as gnu/services/dovecot.scm.
>   We keep it compatible with 'use-service-modules' at the cost of having
>   a multitude of files under gnu/services, without any logical grouping
>   (messy).

That's a great initiative!  I agree that multiple 'define-configuration'
services per file can be a bit messy, having to use prefixes everywhere,
making the definitions more verbose.

I don't have a strong preference of the caterogization of services, but
would perhaps prefer the first one (gnu/services/mail/dovecot.scm),
which could then make it easy to offer some interface as
gnu/services/mail.scm that'd re-export all that is needed (would that
work, or reintroduce the same top-level clashes?).

-- 
Thanks,
Maxim

Re: RFI response: Strengthening the Free Software Supply Chain via Guix and GNUnet

[Maxim Cournoyer]

Hi Sergio,

Sergio Pastor Pérez  writes:

> Hi, Maxim.
>
> This has been an interesting read. I would like to encourage you to keep
> us updated.
>
> Thanks for sharing!

My pleasure!  I will keep you updated, if anything happens from it.

-- 
Thanks,
Maxim

RFI response: Strengthening the Free Software Supply Chain via Guix and GNUnet

[Maxim Cournoyer]

Hello,

Just for your info, I'm relaying the response I've sent for
consideration to the US government [0] in reply to their RFI (Request
For Information) [1].  It's about adding support for GNUnet for
sharing Guix substitutes.

I had no prior experience authoring these so it is probably written in a
naive style :-).  Reading the other comments, it's interesting to note
how the other submissions cover topics all over the places.  It seems a
common theme is 'rewrite the world in Rust' :-).

[0]  https://www.regulations.gov/comment/ONCD-2023-0002-0033
[1]  https://www.regulations.gov/document/ONCD-2023-0002-0001

-- 
Thanks,
Maxim

Re: Expressing system test dependencies

[Maxim Cournoyer]

Hi Bruno,

Bruno Victal  writes:

> Hi,
>
> There are system tests that would benefit from being able to express
> that their results are dependent on the result of other tests, when
> these constitute independent units, especially when there are
> configuration variants present. (e.g. gnu/tests/gdm.scm)
>
> Consider the following files/system-tests [1]:
> * gnu/tests/gdm.scm
>   Right now this module implements two tests: %test-gdm-x11 and
>   %test-gdm-wayland. (whose names are self-explanatory)
>
> * gnu/tests/vnc.scm
>   This module implements one test, %test-xvnc, in particular it does so
>   by testing the XDMCP feature using GDM.
>   Note: this module predates gnu/tests/gdm.scm.
>
> Here's where we would benefit from being able to express system-test
> dependencies. Clearly some of the “test-assert” clauses in
> gnu/tests/vnc.scm would be better placed in gnu/tests/gdm.scm as they
> are GDM specific tests. Since they constitute configuration variants of
> the service, it would be placed into a %test-gdm-autosuspend variable.
>
> The rationale for a test-dependency mechanism is as follows:
> Suppose we split the GDM specific test to (gnu tests gdm). Now let's say
> that we run %test-xvnc and it fails. Is the failure due to GDM or is it
> caused by something else? (within the %test-xvnc)
>
> By splitting the test to (gnu tests gdm) we would have to run the GDM
> tests first, which isn't obvious. If we could express a dependency here,
> the debugging experience is improved as we now have a way to know that a
> integral component involved in the test failed. (or could be used to
> rule out other parts)
>
> Since the system-test results are expressed as derivations, successful
> tests shouldn't result in duplicate runs so perhaps we could make use of
> this fact for the effect?

Ideally, any kind of tests would be independent from each other, but for
system tests where it's an integration of various components, we do rely
on the individual components used in a test working.

I'm not sure it's worth adding more complexity though; a failure in GDM
should have been caught at the time packaging/upgrading GDM with its own
test suite; it seems these cases would be very rare, and we have actual
real problems to fix (c.f. our bug tracker) :-).

-- 
Thanks,
Maxim