Potential security issue with make authenticate and mitigation

[John Kehayias]

Hi Guix-ers,

Please see the below message (and attached report for further details)
of a potential security issue and mitigation in Guix, from Skyler
Ferris. The very short version: 'make authenticate' is a potential
attack vector, which can be mitigated by using 'guix git authenticate'
in a development workflow.

This was raised to guix-security some months ago, but unfortunately
went through various stalls and got lost for a bit. My apologies to
Skyler and thanks for their patience. At least one other person on
guix-security also recommended we open this up for public comment,
which I agree would be good as we can discuss the potential workflow
and documentation changes quickly and transparently.

Please also see the proposed changes to make 'guix git authenticate'
easier to use, which could also help make easier to use in the Guix
workflow: 

(As an aside, some new members for guix-security would be helpful. I'm
happy to continue but will be away for about a month soon.)

>From Skyler, lightly edited for formatting:

In 2020, the "guix git authenticate" tool was added in order to secure
updates (1). This protection is still intact! The tool also had the
secondary effect of protecting developers against malicious commits
while we are developing. In fact, the manual currently recommends that
all developers run "make authenticate" after every pull for this
purpose (2).

Unfortunately, it turns out that "make authenticate" can itself be
used as an attack vector. The core of the problem is that "make
authenticate" uses the Makefile before the commits have been
authenticated, allowing an attacker to replace the Makefile with a
malicious version. The attacker would need the ability to inject the
malicious commit into the data you pull: for example, by compromising
the Savannah server or poisoning a DNS cache. The attached report
contains full details and a proof of concept.

Fortunately, preventing this problem is fairly straightforward:
instead of running "make authenticate", run "guix git authenticate"
manually. This will detect the malicious commit before the Makefile is
used, alerting you to the problem. As a first step towards a long-term
solution we can stop recommending the use of "make authenticate" in
the manual. We would like to remove the "authenticate" target entirely
because it is easy to misuse. However, this could break existing
workflows. Hopefully this is trivial to solve by running the tool
manually as mentioned above but maybe there are some situations we
have not considered - let us know! We are also aware that the
post-push hook installed by Guix runs "make authenticate". However,
this is only done as a reminder to the developer to avoid breaking
"guix pull"; it is not done for the developer's security. The
post-push hook can be changed to use "guix git authenticate" instead
of "make authenticate" in the same set of patches to avoid breaking
anything.

(1) 

(2) 

Thanks everyone!
John

(Apologies is this went through to some people, but I only realized
days later it wasn't sent to the correct guix-devel address.)


security-report.md
Description: Binary data

Re: Sustainability fund application ongoing

[John Kehayias]

Hi Maxim, Ludo’, and everyone,

On Mon, Apr 22, 2024 at 01:52 PM, Maxim Cournoyer wrote:

> Hi Ludovic,
>
> Ludovic Courtès  writes:
>
>> Hi Maxim,
>>
>> Maxim Cournoyer  skribis:
>>
>>> It was pointed to us that the "Free and Open Source Software
>>> Sustainability Fund" [0] is currently receiving applications.  The fund
>>> aim to sponsors free software projects for their
>>> maintenance/organisational activities.
>>
>> Nice!  You forgot the link but I guess it’s
>> .
>> Deadline is May 17th.
>
> Indeed, thanks for pointing it out.
>

Thanks Maxim for sending this out, this looks like a great opportunity
(I'm sure quite competitive though)!

>>> I'm tempted to apply myself, but I thought I'd share it here: the more
>>> submissions they get the more likely Guix is to receive support.
>>
>> Agreed, it’s great to have people apply there (even better if it works,
>> of course).
>>

I'm still looking into the details of this group and their process,
but would it make sense to have small teams in applying (or in mind as
part of a proposal)? I'm thinking in terms of some input and
responsibility, for instance, in a proposal that wants to improve
things like new contributions, community, and related infrastructure
(Cuirass, Mumi, interfacing with debbugs more generally). I guess one
person applies, but in trying to figure out scope and who might be
involved should a proposal get funded it could be good to have some
people in mind.

>> Maybe we should provide a contact point for people who want to
>> coordinate when applying for Guix-related funding, to avoid overlapping
>> or competing applications?
>
> I guess we can use guix-devel for that purpose; the traffic would not justify 
> a new
> mailing list I guess (and we already have too many to keep track of :-)).

I am very interested in this and would love to discuss with some
people for needed information, input, things like scope and what would
have the most impact for Guix. So, I may be soliciting some help on
IRC, here, and reach out individually, but would love to put in a
strong proposal. (I'm at a time where I'm contemplating a career
shift, so this could fit in particularly nicely should I be so lucky.)

Thanks again Maxim for sending this!
John

Re: System can no longer be reconfigured

[John Kehayias]

Dear Felix et al,

On Sun, Apr 21, 2024 at 08:54 AM, Felix Lechner via \"Development of GNU Guix 
and the GNU System distribution.\" wrote:

> Hi Ludo'
>
> On Fri, Apr 19 2024, Ludovic Courtès wrote:
>
>> Is it not hanging during Shepherd service upgrade?
>
> Yes, thank you!  It was hanging during the Shepherd system upgrade due
> to an invalid calendar-event specification.  In a service that uses both
> #:days-of-month as well as #:days-of-week, I got confused because the
> count starts with one in one, and with zero in the other. [1]
>
> Thank you for your kind pointer in another forum! The issue is now
> solved.
>
> On this occasion, please allow me to mention for everyone's benefit that
> your support level for the Shepherd is very generous---especially when
> the complaints involve repeated user errors by the people bothering you.
>
>> If that is the case, (1) that is not preventing upgrade from happening
>> since that’s the very last step (it just means you’ll have to reboot),
>
> Yes, but 'reboot' and 'halt' also stop working, which meant that I
> issued 'sync' a few times and then, crossing my fingers, pressed the
> reset button a few hundreds of a second later.  I wish there were a way
> to force a reboot.
>

Not sure if this is what you did, but for anyone else that didn't know
this (like me, for many years!) there is the Magic SysRq Key to talk
directly to the kernel:


I always remember "BUSIER" backwards as the sequence to push to safely
stop everything, sync disks, and reboot, without needing to use the
power button. Or the mnemonics: "raising elephants is so utterly
boring" or "reboot even if system utterly broken." Hopefully rarely
needed, but really handy when it is.

(I wrote a whole basic guide to killing things and these last keys:
)

>> (2) it’s a shepherd-related bug for which perhaps there are clues in
>> /var/log/messages?
>
> Thank you for that pointer, too.  I'm still learning about where to look
> for error messages, whose destination is not always obvious to me.
>
> Kind regards
> Felix
>
> [1]
> 

Re: bug#63267: gcc-toolchain is missing libstdc++.so

[John Kehayias]

Hi everyone,

Apologies for the long delay on this.

On Tue, May 09, 2023 at 07:07 PM, Simon Tournier wrote:

> Hi,
>
> I am proposing patch#63393 [1] which adds the output lib to
> gcc-toolchain.  Well, quoting the comment:
>
>   ;; The main raison d'être of this "meta-package" is (1) to conveniently
>   ;; install everything that we need, and (2) to make sure ld-wrapper 
> comes
>   ;; before Binutils' ld in the user's profile.
>
> I think, this package gcc-toolchain should be the visible package and
> battery included.
>
> WDYT?
>
> 1: https://issues.guix.gnu.org/issue/63393
>
> Cheers,
> simon

I've just pushed, as b47ae1ecc43baaf726701ab2d2f810ecfaa75428, the
patches from that issue (the first from simon, the second my simplified
one just adding gcc:lib to gcc-toolchain).

This should fix the original bug here, so I am closing. However, it was
raised here and in 63393 about alternatives in how we use gcc-toolchain
outputs. As well as the issue I ran into about make-libstdc++.

So, if anyone would like to change anything from the new status quo,
please open a new issue. At least now we are working from a better
default I would say, with gcc-toolchain including the libraries from
gcc:lib.

Thanks all,
John

Recent security issues (guix-daemon and xz)

[John Kehayias]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512


-BEGIN PGP SIGNATURE-

iQJRBAEBCgA7FiEEpCB7VsJVEJ8ssxV+SZCXrl6oFdkFAmYIhx0dHGpvaG4ua2Vo
YXlpYXNAcHJvdG9ubWFpbC5jb20ACgkQSZCXrl6oFdmOLw//dXLjX82YYaQeEpbp
a0xNswh4OyHKDpMwy8rE2sWbQ7Ckivu79e+0m8wGbzf1+K4SB37CaxT2g9/98uRB
Vm5XiztOmnx83aqin4DS141FR8mVFUz+7YzqNP5iZNq+x+s3OBlWE3D3fxiD5dnj
9ZFNfGev8tgNyYrQ6Sa6ftXBvK61O20kWMa23BsFojZiSldyZO+ELP6fNIg0Pc9z
pZdbP3l5Y5sInPe+mNNJG/SLgOXnovGA/Cg/4N+JciF490bwWnMR6HrtSHAONJKk
VtBPgKBYoIUtQFTD0922+2ykfBKGL4R4KdlXXChwqLLeFYZ6K4NHK4RD4tXLbW2H
KH3z72dHennuLAeyOxvUu7BBJnEKNGJ8DWdXF0g4HnbbaCADxuLvM70i3ddOBLuq
cQuHuzLZH/anynCGHclwj4I0ZPP8i8tZVUhGdRQ+skXTVySx5me6aG7Yc9bRYNyy
v7Aafdf4jdBGIEEy1oBvrZFLnp1VUOBEcWRWTOQNlmBOjy52kxpUD/bzSVSQErft
yLBd58VBicOsrm/hPZZ9NVvOfaxZn4LY1/fmMKX58JQJUv+OaepwVE42icX5EqT6
JVCHXNhoJ0xqBCIhfid+KHwO7ePjXJoVaShzs864OVwx3IyPaphbGH3/XL6sutJb
ncmpTQzJJllME60zLO+4fQzNtq4=
=SYeT
-END PGP SIGNATURE-
Hi Guix-ers,

Two security issues I would like to briefly draw attention to:

First, a belated (sorry!) note about a security issue that was
originally found in Nix but also affects the guix-daemon. All users
are strongly encouraged to update their guix-daemon. For details about
this security issue, how to check if you are on an impacted version,
and most importantly how to upgrade, please see the blog post:

<https://guix.gnu.org/en/blog/2024/fixed-output-derivation-sandbox-bypass-cve-2024-27297/>

Secondly, perhaps many have heard of the recent security issue
(backdoor) in the xz project:

- <https://www.openwall.com/lists/oss-security/2024/03/29/4> (original
  disclosure)

- <https://nvd.nist.gov/vuln/detail/CVE-2024-3094> (CVE-2024-3094)

As far as I, and those I've discussed with, can tell, Guix is *not*
affected. For one, we are currently on an older version, 5.2.8, which
I believe also predates most or all of the contributions made by the
identity associated with the backdoor. We also don't fit what we
currently know about when the backdoor is enabled in the build, due to
our packaging not being one of the targets, as well as not using
systemd (which provided a link between sshd and xz), among other
factors.

This is an evolving situation with many current discussions online. I
also just noticed that the xz project has a page identifying this
backdoor and what they are currently doing:
<https://tukaani.org/xz-backdoor/>. Though given how this exploit has
come about, we should remain skeptical and vigilant.

Let me stress that there is much we don't know. There certainly
remains the possibility of other exploits or malicious code to be
discovered, as well as looking at contributions made via the same user
identifier to other projects. We will be keeping a close eye on this,
but please report any security issues to .

I hope this was helpful and assuring but I welcome feedback on any of
this. While I am on guix-security, please note I wrote this message
independently to be timely and hopefully assuage any questions.

I hope otherwise everyone is having a great weekend and that your Guix
machines (and all the others!) are humming along happily!

John Kehayias

Re: Backdoor in upstream xz-utils

[John Kehayias]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512


-BEGIN PGP SIGNATURE-

iQJRBAEBCgA7FiEEpCB7VsJVEJ8ssxV+SZCXrl6oFdkFAmYHK0sdHGpvaG4ua2Vo
YXlpYXNAcHJvdG9ubWFpbC5jb20ACgkQSZCXrl6oFdkFRA//WaJMegtHd88wlq0V
QovAYD7+d6zj5DxgVTiGKXckyKWx7AceVJ0WVp9MB+WxU8dEXepEnd9AHOA4v/Fb
HLy4prms+noIpXqHW5y6EDgbMiBUX2rk6UVq7qnLCPujfv3hrJl4S7B5fJxjLSM/
M++F40YKc6PNSjQHi9BH5+Vl70jGCIzXNcomvEanu4SAsXLSlEwvOlnAPD57mb4k
n4Tg4d7ExXjdi7/qdq/OnF2RGQjiLQ4qX7AeSu8kIaEaK3WdMy1JO1fy9vaZNuSg
oCuUGJYCFj60BEYDQdUM8NiNe76zVzXvP/wKrR1XpqsnK9keKKEZpuZCQmJApgCJ
dvVbrU8OfKPJ/B7CwNJu32FyrdgQt53ytYjNxs/cNNjB2ciDeIGszCzxwytRZz4k
JEbE8VZrUACNvQXCdRbr1Jse1+FuM2hjTwILdia/A8GcWn9tfmfGdqlqOuw6c8qG
hYX7l3+3t0c7VzLhgs2iE/BEKtUAYCrwRf+10J9dOm4TzmbEbg7+1j7FJcYhmIgJ
qeEXistWXx7FY2Yl0UjrNtxi3UGR5rnx2hAb3zEcMoqcHHKuKz/X8aeMfIHryn23
rQms/cVwAPeR908xwbJgqkzQhY5A9DrU+0VGssILyXKvMYp6xTXJ6cf2gGLyhAFF
VerlLVFCEHunNyWr94ZTeXr3p00=
=dUKI
-END PGP SIGNATURE-
Hi Ryan, Felix, and guix-devel,

On Fri, Mar 29, 2024 at 01:39 PM, Felix Lechner via Reports of security issues 
in Guix itself and in packages provided by Guix wrote:

> Hi Ryan,
>
> On Fri, Mar 29 2024, Ryan Prior wrote:
>
>> I'm reading today that a backdoor is present in xz's upstream tarball
>> (but not in git), starting at version 5.6.0. Source:
>> 
>
> Thanks for sending this!  This is an extremely serious vulnerability
> with criminal intent.  I cc'd guix-secur...@gnu.org just in case you
> haven't.
>

At least me (as part of guix-security) is aware and have been reading
the analysis and further investigation.

Both clever and interesting, but also worrisome. I think we were
rather lucky this was found relatively quickly, though it seems to
point to a bad actor and throws into question other projects (like
libarchive) which have contributions from the same identity. Likely
other accounts are involved too, so maybe on a positive side this
unravels other issues.

The discussion on Hacker News has also been informative (though rather
long now): 

>> Guix currently packages xz-utils 5.2.8 as "xz" using the upstream
>> tarball. [...] Should we switch from using upstream tarballs to some
>> fork with more responsible maintainers?
>
> Guix's habit of building from tarballs is a poor idea because tarballs
> often differ.  For example, maintainers may choose to ship a ./configure
> script that is otherwise not present in Git (although a configure.ac
> might be).  Guix should build from Git.
>

We discussed a bit on #guix today about this. A movement to sourcing
more directly from Git in general has been discussed before, though
has some hurdles. I will let someone more knowledgeable about the
details chime in, but yes, something we should do.

Unfortunately in this case, while it seems the older versions don't
have *this* exploit, given the perpetrator either is or has control over
a maintainer account, it throws into question a lot more than the most
recent version. We will have to keep a careful eye on this. I'm not
currently aware of anything untoward for our current version, so far.

>> Is there a way we can blacklist known bad versions?
>

I'm not sure what you mean, but I don't think so. The main danger is
in guix time-machine to the past, as you are (purposefully) going to
older versions of software. This is warned in the manual

though we should perhaps do this at runtime as well.

Even better would be if we can warn about known bad versions. Such a
tool was started (guix health) here:
 Anyone up for reviving it, now
that we have some changes that should make this more doable (based on
a quick glance of more recent messages)?

> Having said all that, I am not sure Guix is affected.
>
> On my systems, the 'detect.sh' script shows no referece to liblzma in
> sshd.  Everyone, please send additional reports.
>

Pretty sure we are not affected, at least with what is known: the
exploit targets particular systems and things like argv[0] being
/usr/sbin/sshd. A combination perhaps of who or what was being
targeted as well as trying to make this harder to discover.

Still, we should have an abundance of caution and pay close attention,
as there is much we don't know and a history of commits to go through.
As well as being suspicious in general of things like binary files
added to a release tarball (as a project we always try to make sure
there are no binary files anyway), this is a clear example of a
clever/malicious way of causing harm.

Please do feel free to report privately any concerns or potential
affected packages to guix-secur...@gnu.org as well. And if you are
interested in helping with these things, I'm sure we could rotate in
some people for that team.

Thanks all! An action-packed Friday.

John

Re: Should commits rather be buildable or small

[John Kehayias]

Hi,

Apologies for the delay. I would like to get things rolling on
mesa-updates and building, including the vulkan updates, so a choice
will have to be made :)

Thanks for the input so far!

On Tue, Mar 05, 2024 at 06:19 AM, Liliana Marie Prikler wrote:

> Hi,
>
> Am Montag, dem 04.03.2024 um 21:38 + schrieb John Kehayias:
>> [...]
>> 1. Essentially squash to one commit where all of vulkan is updated in
>> one commit. The main upside is that nothing should break (within
>> vulkan, dependents to be fixed as needed) and it shows as "one"
>> change; the main downside is that the proposed changes are not just
>> trivial version bumps. Harder to then disentangle as needed.
>>
>> 2. Make each commit updating a package, but don't use the variable
>> %vulkan-sdk-version, updating each package with a version as it is
>> done. Then do a commit where all the versions are replaced by the
>> variable. This seems like unnecessary work to me and while it stops
>> the obvious breaking (source hashes don't match once variable is
>> updated but package hasn't yet) versions are still mixed which is
>> likely a problem.
>>
>> 3. Go with the series as proposed: this means after the first commit
>> for sure all other vulkan packages and dependents don't build, as the
>> source hashes won't match until the commit that updates that package.
>> Along with version mixing, this perhaps doesn't give you a helpful
>> git bisect either?
>>
>> None are perfect. What do people think?
> I think 1 would be workable if the changes to the packages are minimal.
> You should also check whether you can just do the version bumps and
> then the other changes – or flip the order.
>

As currently proposed, the changes are not minimal.

dan: Do you know if just a version/hash bump is at least buildable? Or
are the changes necessary for the packages to build/function at all?
Or I guess if the non-version changes are applicable to the current
version first?

> I don't really see the benefit with 2.  Normally, we'd have "-next"
> variants to catch nontrivial updates (among other things), but those
> don't seem a good approach here.
>
> If nothing else works, 3 is indeed an option to fall back to, albeit
> begrudgingly.  As noted for 1, you could check whether bumping all the
> hashes and then only fixing whatever else for the builds is an option
> here.
>

That's what I'll have to do I think, unless indeed the versions
changes can be made separately and still build. I can mark each patch
in the commit log that it is part of a series updating all the vulkan
packages. That might be something worth doing in general for cases
like this, to help out future time travelers and when e.g. searching
the log and finding a commit.

> Alternative 4 would be to build those -next variants and then replace
> the base vulkan all at once.  This has the advantage of not doing any
> version mixing in-between IIUC.
>

That's also an idea. Add a %vulkan-version-next or something like
that, and -next variants of all the packages using that version
instead. A bit clumsy and perhaps convoluted with the extra work for
maybe minimal gain.

I'll wait to see if dan has any information of what changes can be
made independently, but I guess I'll just have to make a decision on
mesa-updates.

Thanks!

John

Re: doc: installation: fix ~root confusion (was Re: doc: Removing much of Binary Installation)

[John Kehayias]

Hi vagrant,


On Sunday, March 10th, 2024 at 9:58 PM, Vagrant Cascadian  
wrote:

> 
> 
> On 2024-03-10, Suhail Singh wrote:
> 
> > Vagrant Cascadian vagr...@debian.org writes:
> > 
> > > but "guix pull" does not update the running guix-daemon;
> > 
> > Just to be clear, however, if one were to do =sudo -i guix pull=
> > instead, followed by =systemctl restart guix-daemon.service= it /would/
> > update the running guix-daemon on Debian, correct? Or is that not the
> > case?
> 
> 
> No, out of the box guix-daemon is provided by the Debian guix package.
> 

That means the instructions to update the guix daemon in the manual, 
 is 
incorrect or doesn't work? Or am I misunderstanding what you meant here?

(I know in the past some discussions have come up about older guix-daemon on 
foreign distros, presumably because the packages there don't get updated and a 
user wouldn't think to upgrade guix separately? But it seems you are saying you 
can't upgrade without modifying the e.g. systemd service definition? This is 
also important for security updates to guix itself, of course.)

> > In other words, on Debian, does the systemd unit reference
> > =/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon= ?
> 
> 
> But you could provide an override pointing at whatever guix-daemon you
> want, of course! :)
> 
> Once you do that, you may as well remove the Debian packaged guix,
> although users that have not yet run "guix pull" would need to guess
> where to find guix, as there will be no guix on PATH.
> 
> 
> live well,
> vagrant

Re: Should commits rather be buildable or small

[John Kehayias]

Hi everyone,

And sorry for reviving an old thread, but I am faced with a similar issue for 
updating vulkan, with the patch series submitted by dan (cc'ed): 
. I thought I would get some opinions here, 
please see below:

On Mon, Dec 11, 2023 at 12:51 PM, Ricardo Wurmus wrote:

> Attila Lendvai  writes:
>
>> i myself also had headaches multiple times when i fixed something that
>> needed to touch several different packages, and they would only work
>> when applied in one transaction:
>>

In this case all the vulkan packages share a version through a variable name. I 
would assume packages wouldn't like mixed versions, but maybe some would work 
(I haven't tried). I'll be taking this series on mesa-updates with related 
changes, so the plan is that when it hits master there are no/few broken 
packages and full substitute coverage. So perhaps this makes this more of a 
style and convention question.

Some options:

1. Essentially squash to one commit where all of vulkan is updated in one 
commit. The main upside is that nothing should break (within vulkan, dependents 
to be fixed as needed) and it shows as "one" change; the main downside is that 
the proposed changes are not just trivial version bumps. Harder to then 
disentangle as needed.

2. Make each commit updating a package, but don't use the variable 
%vulkan-sdk-version, updating each package with a version as it is done. Then 
do a commit where all the versions are replaced by the variable. This seems 
like unnecessary work to me and while it stops the obvious breaking (source 
hashes don't match once variable is updated but package hasn't yet) versions 
are still mixed which is likely a problem.

3. Go with the series as proposed: this means after the first commit for sure 
all other vulkan packages and dependents don't build, as the source hashes 
won't match until the commit that updates that package. Along with version 
mixing, this perhaps doesn't give you a helpful git bisect either?

None are perfect. What do people think?

My instinct is to go with the series as proposed (after review) accepting that 
there will be for sure builds failing if time traveling to the middle of the 
series. I don't think we can really avoid that anyway, as sometimes we only see 
an issue after a commit and it is fixed some time later. We could have a note 
in the first commit that this requires the next n commits to update vulkan 
packages. That might help if someone is on an intermediate commit and can see 
quickly in git log this note.

Or perhaps we can note something is part of a dependent series when we make 
commits so this is easier for someone to tell in general?

Thanks!
John

Re: Packaging Hyprland

[John Kehayias]

Hi Efraim,

On Sun, Feb 25, 2024 at 12:42 PM, Efraim Flashner wrote:

> On Sat, Feb 24, 2024 at 10:32:29PM +0000, John Kehayias wrote:
>> Slightly off topic, but for anyone wondering about my emacs keys issue:
>>
>> On Sat, Feb 24, 2024 at 04:01 PM, John Kehayias wrote:
>>
>> > Seems xremap can do it (which we have packaged) except it doesn't
>> > pick up different applications for where keys apply on Hyprland. I
>> > do miss in Stump how easy that was right in the config.
>>
>> It does work! Slight difficulty since I had already set capslock to
>> control in my Hyprland settings (via xkb I believe). So I still had to
>> map capslock to control in xremap as well, and then the provided
>> example of emacs keybindings works (with the modification that on my
>> system at least it is 'emacs' (lowercase) for the application name to
>> make sure xremap doesn't apply there as well.)
>>
>> Guess I can continue with Hyprland!
>
> The README and Cargo.toml disagree about what commands to call, but I
> can add a hyprland variant of the xremap package if there's interest.

Yeah, I'm not sure the difference of xremap wlroots or hyprland
variant. I'm just using our packaged wlroots one and it is working
great now. I suppose we should add the hyprland one, at least when we
have hyprland. Thanks in advance Efraim!

On the xremap side, a service would be great, and luckily we have one
pending: <https://issues.guix.gnu.org/66932>.

John

Re: Packaging Hyprland

[John Kehayias]

Hi everyone,

Just a note on cairo below:

On Sun, Feb 25, 2024 at 10:39 AM, Hilton Chain wrote:

> Hi everyone,
>
> On Sun, 25 Feb 2024 08:32:27 +0800,
> Lucy Coleclough wrote:
>>
>>> On Sat, 24 Feb 2024 at 20:48, hutzdog  wrote:
>>>
[snip]
>>>  # New Patches
>>>  The following new patches will need to be created (I intend to submit these
>>>  at some point in the near future):
>>>  Cairo -> 1.18.0 (requires moving to Meson, I have a mostly complete set of
>>>  changes to make it work)
>
> I didn't take a closer look at cairo update, but yes, this may require some
> work, at least to be able to finish its tests.
>

I submitted the patches as 69495: 
It works for me locally in some limited tests (e.g. icecat).

Hutzdog: you mentioned getting the docs built and put in the output,
but I didn't see that in your linked repo. The issue is that gtk-doc
is needed which depends on cairo. I worked around the dependency cycle
by hiding cairo and making a new cairo-with-documentation (the first
is used for packages and hidden, the second with docs and public).
Thanks to lilyp on #guix for pointing me to glib with a similar issue
and workaround.

But maybe I missed something if you didn't need to do this?

Once cairo is approved I'll get mesa-updates building with libdrm,
cairo, vulkan, and mesa updates. It would be nice to grab the libinput
update so we don't have to wait for core-updates, but I didn't look if
that make sense. Otherwise we could have just a libinput-next for
hyprland, for instance (though I don't know where it fits in the
dependency graph here).

More updates soon, and thanks everyone! Cool to see about plugin
systems and services, haven't looked at that yet.

John

Re: Packaging Hyprland

[John Kehayias]

Slightly off topic, but for anyone wondering about my emacs keys issue:

On Sat, Feb 24, 2024 at 04:01 PM, John Kehayias wrote:

> Seems xremap can do it (which we have packaged) except it doesn't
> pick up different applications for where keys apply on Hyprland. I
> do miss in Stump how easy that was right in the config.

It does work! Slight difficulty since I had already set capslock to
control in my Hyprland settings (via xkb I believe). So I still had to
map capslock to control in xremap as well, and then the provided
example of emacs keybindings works (with the modification that on my
system at least it is 'emacs' (lowercase) for the application name to
make sure xremap doesn't apply there as well.)

Guess I can continue with Hyprland!

Re: Packaging Hyprland

[John Kehayias]

Hi Hutzdog,

On Fri, Feb 23, 2024 at 01:20 AM, hutzdog wrote:

> Hi all,
>
> I've been working on moving over to GNU Guix recently, and have hit a

Welcome to Guix! (For the first part, not the roadblock part...)

> roadblock: there is no package for Hyprland (the one WLRoots based
> compositor with single window capture and automatic window swallowing
> that I know of). I've taken the liberty of packaging the latest
> version (see
> 

Funny enough, I just decided to take the plunge and try out Wayland
finally with Hyprland, as a sucker for eye candy. I do miss using Lisp
more (coming from lots of StumpWM time) and easy emacs keybindings
everywhere.

I'm using a package from a channel, I believe, of one of our
committers, who I'm cc'ing as I was just about to ask them about
upstreaming.



Works great on my end, thanks to them!

> for the package), but there are some changes that need to happen in
> order for it to be upstreamed (as of v0.35.0).
>
> # Pending Patches
> The following existing patches need to be merged:
> LibInput -> 1.25.0 ()

I could take this on mesa-updates as I was going to be doing some
updates so might as well take this. I'll add it locally and send a
message to the bug number. Part of my motivation was to be able to
upstream Hyprland as well.

If this fits better on another existing branch, please let me know,
someone.

> LibDRM -> 2.4.120 ()
>

I have this locally on my mesa-updates. I'll send a message to that
bug number.

> # New Patches
> The following new patches will need to be created (I intend to submit
> these at some point in the near future):
> Cairo -> 1.18.0 (requires moving to Meson, I have a mostly complete
> set of changes to make it work)

I have this locally as well and will send some patches for review. It
took a bit more work, though the final result is pretty simple, it
needed a hidden cairo and a new cairo-with-documentation to build the
docs. I built up to icecat locally and all was good from what I could
see.

Sorry for any duplicate work, perhaps your version can add something
to mine.

> Toml++ (package will be sent as a patch soon)
> Hyprlang (for xdg-desktop-portal-hyprland, will publish after Hyprland)
>

I believe we'll need a more current version than the released wlroots
as well. Possibly others based on the channel I referenced above, but
the big update is libdrm and cairo, in terms of rebuilds.

> ## HWData
> As with packages using the release versions of WLRoots, due to how
> Guix packages HWData a patch is needed to make Meson find it. We have
> a few options: maintain a parallel package which simply farms all
> outputs of HWData as symlinks and adds the pkg-config file, maintain a
> patch on a much more volatile version of WLRoots, or find some other
> solution.

Was this handled already in  for
wlroots? Which has been merged.

>
> # Hyprland
> This will allow me to submit packages for Hyprland and its XDG Desktop
> Portal at version 0.35.0 (the latest release). As it's one of the more
> popular Wayland compositors out there, I think it is worth adding it
> to the repos. For now, the package is available through my Guix
> channel (fair warning, it is still very WIP and I wouldn't recommend
> using it yet outside of maybe pulling the Hyprland packages). I look
> forward to working with Guix (Scheme is certainly a breath of fresh
> air after dealing with Nix for a while) and contributing to its
> ecosystem.
>
> --Hutzdog
>

Thanks for your efforts and helping get this conversation started! I
think we can handle the bigger updates on the mesa-updates branch. I
was planning on pushing that to get builds going once I send the cairo
patch and have people take a look at that. Despite the big changes,
I'm fairly confident in it since it seems pretty much everything we
need is built by default now.

And agreed that Hyprland has been pretty nice so far as my first step
into Wayland. Though if anyone can point me to the best way to get
emacs keys everywhere I will be forever thankful. Seems xremap can do
it (which we have packaged) except it doesn't pick up different
applications for where keys apply on Hyprland. I do miss in Stump how
easy that was right in the config.

John

Re: Update to source-highlight seems to have broken rust

[John Kehayias]

Hi Kaeylyn,


On Monday, January 29th, 2024 at 12:26 PM, Kaelyn  
wrote:

> 
> 
> Hi Efraim and guix-devel,
> 
> Based on my local cuirass instance and some spot testing this morning, it 
> appears the change to source-highlight in commit 367bc2d198 has broken the 
> build of rust 1.73.
> * `guix refresh -l source-highlight` reports modifying the package would 
> require a large number of rebuilds ("Building the following 6129 packages 
> would ensure 9069 dependent packages are rebuilt").
> * I am seeing a consistent test failure for rust starting with that commit.
> * `guix weather rust@1.73` reports 100% availability for both ci.guix.gnu.org 
> and bordeaux.guix.gnu.org at commit fbeae77ae6 but 0% availability at commit 
> 367bc2d198.
> 

This was reverted in 0fb160a5e8b0b800426489c0a2ad387c6934fba so maybe you were 
just unlucky in what commit you were at?

I can't comment on the details of the change or rust, but hopefully just 
pulling to a newer commit will go back to good coverage for you as well.

Hope that helps!
John

> The consistently-reported failure is:
> 
>  metadata::cargo_metadata_non_utf8 stdout 
> thread 'metadata::cargo_metadata_non_utf8' panicked at 
> crates/cargo-test-support/src/paths.rs:155:33:
> failed to mkdir_p 
> /tmp/guix-build-rust-1.73.0.drv-0/rustc-1.73.0-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/tmp/cit/t1684/foo/�/./src:
>  Invalid or incomplete multibyte or wide character (os error 84)
> note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
> 
> 
> failures:
> metadata::cargo_metadata_non_utf8
> 
> test result: FAILED. 2801 passed; 1 failed; 198 ignored; 0 measured; 0 
> filtered out; finished in 92.81s
> 
> error: test failed, to rerun pass `--test testsuite`
> 
> 
> I don't exactly understand how wrapping scripts in source-highlight ended up 
> breaking the above rust test, nor do I use rust, so I wanted to bring it to 
> folks' attention since the failure appears to affect a great many packages (I 
> noticed it in an ffmpeg build failure on my local cuirass instance).
> 
> Cheers,
> Kaelyn

Re: update darktable version

[John Kehayias]

Hi Alex,

On Wed, Jan 24, 2024 at 06:15 PM, Alex Devaure wrote:

> Hi Maxim,
>
> Thank you for your answer. I created the issue #68293 and the
> modification is now on master branch.
> Best regards,
> Alex
>

Looks like Vinicius Monego already did the update in
9c67673d7bb9cdc43c1196b075447b57aff74039 after pushing #67719 starting
with 50731875243428cb11cc17fa712fa8079b307622.

However, both issues were left open, so I'm closing both with this message.

Sorry for any duplicated work and thanks both of you for keeping
Darktable up to date and improving it!

John

> Maxim Cournoyer  writes:
>
>> Hi Alex,
>>
>> Alex Devaure  writes:
>>
>>> Hi all,
>>> There is a new version of the RAW developer darktable. The patch #67719
>>> modifies the generation of it by replacing clang with gcc. The patch is
>>> not yet merged, should I base my patch (updating darktable version) to
>>> #67719?
>>
>> That'd be fine, yes.

Re: Commit Access: Sharlatan Hellseher

[John Kehayias]

On Thu, Jan 11, 2024 at 07:56 PM, Sharlatan Hellseher wrote:

>
> Hi Guix!
>
> I am happy to have been granted commit access and I am ready to help
> review pending issues and prepare queued packages for GNU packages in
> astronomy. I would like to concentrate on the packages covered by the
> Go, Lisp, Python, and Science teams.
>
> I would like to thank the Guix team for allowing me to become a
> committer member. I am looking forward to continuing our collaboration.
>
> If anyone has a good patch review workflow using Emacs, Gnus, and Magit,
> I would appreciate it ;-)
>
> Regards,
> Sharlatan Hellseher (Oleg)
>

Welcome and looking forward to your further Guix contributions!

John

Re: xwayland security updates, to mesa- or core-updates or ?

[John Kehayias]

On Mon, Jan 08, 2024 at 10:32 AM, Efraim Flashner wrote:

> On Mon, Jan 08, 2024 at 05:43:40AM +0000, John Kehayias wrote:
>> Hi all,
>>
>> Forgive the top post and please see below/previous messages for
>> previous updates.
>>
>> TL;DR: I plan to merge mesa-updates into master today-ish (well,
>> tomorrow for me at this point).
>>
>> I've been checking in with Efraim who's been very helpful at trying to
>> nudge along substitute coverage on non-x86_64 platforms. Unfortunately
>> looks like we have plateaued a bit on, e.g., aarch64. We haven't been
>> getting stats from QA for this round, and Berlin looks good for what
>> it covers (x86) but other architectures are down from what we can
>> tell.
>>
>> I don't think there are any fundamental failures at this point but
>> just lots of "missing derivation" errors (I've restarted so many
>> manually for x86_64/i686) and builds not completing without restarts.
>> Or unknown reasons. Given the few weeks I've given this and the risk
>> of just perpetually doing rebuilds to keep catching up (with then more
>> updates to push) I think it would be best to merge to master. Mesa and
>> other bits will continue to move forward as well, so I think it is
>> time so we can be somewhat timely.
>>
>> I'd rather not without complete substitute coverage, but given recent
>> build farm difficulties, and the tools we do have for users (pinning,
>> weather checks, etc.) I think it is best to call this branch so we can
>> move on. Gnome has some updates that will need (re)building as well as
>> trying to move forward with core-updates now too.
>>
>> This is a case where having some better sense of our users and actual
>> substitute needs/wants would be helpful (yes, Guix survey!) as well as
>> recognizing our current infrastructure limits. Here's another vote for
>> prioritizing infrastructure and making sure QA lives and expands.
>>
>> Feel free to object to this merge timing, though with the relative
>> silence in each previous message I take it I can make a call here.
>>
>> Thanks everyone and hope 2024 is off to a good start! Enjoy the new
>> mesa with curl and xwayland security updates (no new grafts!).
>
> To record here more or less what I said on IRC, we're currently at
> rust-1.56 or 1.57 on the mesa-teams branch, and we're looking at
> probably more than a week to build out to rust itself, and then the
> packages which depend on it.  Currently, on master, Berlin already is
> running behind on building rust, and it wasn't until after the previous
> mesa-updates merge that it caught up with building rust.

Thanks again for your help and watchful eye on this Efraim!

Merged in 7a7c8920aeddaf9ab8d68c572780bc34b404711b.

Thanks everyone, apologies for anyone that needs to wait for
substitutes. Feel free to CC me directly on any breakages due to this
merge but hopefully I didn't miss anything major.

John

Re: xwayland security updates, to mesa- or core-updates or ?

[John Kehayias]

Hi all,

Forgive the top post and please see below/previous messages for
previous updates.

TL;DR: I plan to merge mesa-updates into master today-ish (well,
tomorrow for me at this point).

I've been checking in with Efraim who's been very helpful at trying to
nudge along substitute coverage on non-x86_64 platforms. Unfortunately
looks like we have plateaued a bit on, e.g., aarch64. We haven't been
getting stats from QA for this round, and Berlin looks good for what
it covers (x86) but other architectures are down from what we can
tell.

I don't think there are any fundamental failures at this point but
just lots of "missing derivation" errors (I've restarted so many
manually for x86_64/i686) and builds not completing without restarts.
Or unknown reasons. Given the few weeks I've given this and the risk
of just perpetually doing rebuilds to keep catching up (with then more
updates to push) I think it would be best to merge to master. Mesa and
other bits will continue to move forward as well, so I think it is
time so we can be somewhat timely.

I'd rather not without complete substitute coverage, but given recent
build farm difficulties, and the tools we do have for users (pinning,
weather checks, etc.) I think it is best to call this branch so we can
move on. Gnome has some updates that will need (re)building as well as
trying to move forward with core-updates now too.

This is a case where having some better sense of our users and actual
substitute needs/wants would be helpful (yes, Guix survey!) as well as
recognizing our current infrastructure limits. Here's another vote for
prioritizing infrastructure and making sure QA lives and expands.

Feel free to object to this merge timing, though with the relative
silence in each previous message I take it I can make a call here.

Thanks everyone and hope 2024 is off to a good start! Enjoy the new
mesa with curl and xwayland security updates (no new grafts!).

John

On Thu, Jan 04, 2024 at 12:09 AM, John Kehayias wrote:

> Hi Efraim and guix-devel
>
> On Mon, Dec 25, 2023 at 08:44 AM, Efraim Flashner wrote:
>
>> On Fri, Dec 22, 2023 at 09:19:27AM +0200, Efraim Flashner wrote:
>>> On Thu, Dec 21, 2023 at 09:18:50PM +, John Kehayias wrote:
>>> > Hi all,
>>> >
>>> > On Mon, Dec 18, 2023 at 12:57 AM, John Kehayias wrote:
>>> >
> [snip]
>>> >
>>> > I haven't seen QA process this branch, so I'm just going with what I
>>> > see on Berlin. From the branches overview it shows about 61% last I
>>> > saw, compared to 72% for master. Unfortunately, non x86 architectures
>>> > are usually better covered by Bordeaux, but I don't know where to get
>>> > a sense of that coverage. For what it is worth, Efraim has manually
>>> > built xorgproto and mesa at least on powerpc64le, riscv64, without
>>> > issues.
>>>
>>> I had berlin build for powerpc64le and that went without any problems.
>>> Locally I built for riscv64 and powerpc and those both built fine.  I
>>> ran into an issue locally with curl on aarch64 and test 1477(?) which is
>>> weird since it's supposed to be skipped but I'm sending it through
>>> again.  Haven't started armhf yet.
>>>
>>> > Coverage on x86_64 and i686 seems good from what I can tell. I also
>>> > don't think there are any other branches ready to merge, and would
>>> > like to give them time to rebuild once these changes hit.
>>> >
>>> > Any thoughts on when to merge?
>>> >
>>> > Thanks everyone!
>>> > John
>>
>
> Coming back to this point, seems Berlin is doing better with building
> but I don't see mesa-updates on QA so I'm not sure about non
> x86_64/i686-linux coverage. Anyone have any thoughts?
>
> I don't know that I've seen real new failures, as still lots of
> "missing derivation" or other transient issues that resolve on forcing
> a rebuild.
>
> I don't want to merge to master and have issues with substitute
> coverage, but do have to call it at some point or will end up keep
> scheduling/waiting for rebuilds to happen anyway.
>
> Thoughts?
>
>> I've been having trouble with curl on aarch64 again. Looking at this
>> snippet from the build log:
>>
>> test 1477...[Verify that error codes in headers and libcurl-errors.3 are in 
>> sync]
>>
>>  1477: stdout FAILED:
>> --- log/1/check-expected2023-12-22 10:53:51.658667071 +
>> +++ log/1/check-generated   2023-12-22 10:53:51.658667071 +
>> @@ -1 +0,0 @@
>> -Result[LF]
>>
>>  - abort tests
>> test 1475...[-f and 416 with Content-Range: */size]
>> --pd---e--- OK (1247 out of 1472, remaini

Re: xwayland security updates, to mesa- or core-updates or ?

[John Kehayias]

Hi Efraim and guix-devel

On Mon, Dec 25, 2023 at 08:44 AM, Efraim Flashner wrote:

> On Fri, Dec 22, 2023 at 09:19:27AM +0200, Efraim Flashner wrote:
>> On Thu, Dec 21, 2023 at 09:18:50PM +0000, John Kehayias wrote:
>> > Hi all,
>> >
>> > On Mon, Dec 18, 2023 at 12:57 AM, John Kehayias wrote:
>> >
[snip]
>> >
>> > I haven't seen QA process this branch, so I'm just going with what I
>> > see on Berlin. From the branches overview it shows about 61% last I
>> > saw, compared to 72% for master. Unfortunately, non x86 architectures
>> > are usually better covered by Bordeaux, but I don't know where to get
>> > a sense of that coverage. For what it is worth, Efraim has manually
>> > built xorgproto and mesa at least on powerpc64le, riscv64, without
>> > issues.
>>
>> I had berlin build for powerpc64le and that went without any problems.
>> Locally I built for riscv64 and powerpc and those both built fine.  I
>> ran into an issue locally with curl on aarch64 and test 1477(?) which is
>> weird since it's supposed to be skipped but I'm sending it through
>> again.  Haven't started armhf yet.
>>
>> > Coverage on x86_64 and i686 seems good from what I can tell. I also
>> > don't think there are any other branches ready to merge, and would
>> > like to give them time to rebuild once these changes hit.
>> >
>> > Any thoughts on when to merge?
>> >
>> > Thanks everyone!
>> > John
>

Coming back to this point, seems Berlin is doing better with building
but I don't see mesa-updates on QA so I'm not sure about non
x86_64/i686-linux coverage. Anyone have any thoughts?

I don't know that I've seen real new failures, as still lots of
"missing derivation" or other transient issues that resolve on forcing
a rebuild.

I don't want to merge to master and have issues with substitute
coverage, but do have to call it at some point or will end up keep
scheduling/waiting for rebuilds to happen anyway.

Thoughts?

> I've been having trouble with curl on aarch64 again. Looking at this
> snippet from the build log:
>
> test 1477...[Verify that error codes in headers and libcurl-errors.3 are in 
> sync]
>
>  1477: stdout FAILED:
> --- log/1/check-expected2023-12-22 10:53:51.658667071 +
> +++ log/1/check-generated   2023-12-22 10:53:51.658667071 +
> @@ -1 +0,0 @@
> -Result[LF]
>
>  - abort tests
> test 1475...[-f and 416 with Content-Range: */size]
> --pd---e--- OK (1247 out of 1472, remaining: 00:45, took 5.310s, duration: 
> 04:11)
> test 1474...[HTTP PUT with Expect: 100-continue and 417 response during 
> upload]
> --pd---e--- OK (1246 out of 1472, remaining: 00:48, took 22.794s, duration: 
> 04:29)
> Warning: test1474 result is ignored, but passed!
> ...
> TESTFAIL: These test cases failed: 1477
>
> It looks like 1474 is passing locally and the ~1474 is telling the test
> suite to ignore the result.  If that's how ~ is interpreted then
> I'd suggest that 1477 is failing hard enough that it's taking the test
> suite with it, not merely ignoring the result.  I'll continue poking it
> but right now I'm starting to like the hurd plan of disabling the test
> instead of merely ignoring the result.

Thanks for looking at this Efraim. Looks like a good chunk of the curl
rebuilds did get through, did it look okay on aarch64 and anywhere
else you checked?

John

Re: xwayland security updates, to mesa- or core-updates or ?

[John Kehayias]

Hi all,

On Mon, Dec 18, 2023 at 12:57 AM, John Kehayias wrote:

> Hi Kaelyn and everyone,
>
> On Fri, Dec 15, 2023 at 05:25 PM, Kaelyn wrote:
>
>> On Thursday, December 14th, 2023 at 10:21 PM, John Kehayias
>>  wrote:
>>
>>>
>>> Hi Guix,
>>>
>>> In light of (more) CVEs in xwayland, see
>>> <https://lists.x.org/archives/xorg-announce/2023-December/003435.html>,
>>>
>>> with already pending security updates, see
>>> <https://issues.guix.gnu.org/67136>, I would like to prioritize
>>>
>>> getting that fixed in master. The tricky thing is that, according to
>>> 67136, the xwayland update needs newer xorgproto, which corresponds to
>>> many rebuilds. (The related CVEs in xorg-server have been pushed
>>> already as effectively minor version bumps.)
>>>

I also updated curl as it was going to be rebuilt and had a new
version out (with some security fixes). I hadn't grafted it on master
but we could do that if the mesa-updates branch isn't merged to master
first.

[snip]

>
> I've pushed 3 patches (mesa, xorgproto, xorg-server-xwayland) to
> mesa-updates after merging in master. The farm is building away.
>

I also had to skip a failing test (unknown reasons) of gtk with these
updates.

Finally, I also enabled the zink driver in Mesa (zink is for OpenGL on
Vulkan). I remember someone asking about it on #guix recently as well,
and we should have it enabled in general, to support devices which may
not be able to use OpenGL without it.

> The request for merging is at <https://issues.guix.gnu.org/67875> with
> some details. In short, running into some issues with builds "failing"
> because they just die or "missing derivation" errors. I'm restarting
> what I see that seems higher impact, but is there anyway to restart
> all the failed builds or ones with missing dependencies?
>

This is still true though I've tried to manually restart lots of
builds on x86_64 and i686, which has removed many of the failures. Any
idea what is happening to cause this more recently?

[snip]

> Thanks! I saw you had posted the latest version and that's what I
> included. On x86_64-linux at least everything has built fine for
> those, but the larger world remains to be seen.
>
> Would still like confirmation from other branches about what they want
> to do, but we have some time while things build. And builds get
> restarted.
>

I haven't seen QA process this branch, so I'm just going with what I
see on Berlin. From the branches overview it shows about 61% last I
saw, compared to 72% for master. Unfortunately, non x86 architectures
are usually better covered by Bordeaux, but I don't know where to get
a sense of that coverage. For what it is worth, Efraim has manually
built xorgproto and mesa at least on powerpc64le, riscv64, without
issues.

Coverage on x86_64 and i686 seems good from what I can tell. I also
don't think there are any other branches ready to merge, and would
like to give them time to rebuild once these changes hit.

Any thoughts on when to merge?

Thanks everyone!
John

Re: xwayland security updates, to mesa- or core-updates or ?

[John Kehayias]

Hi Kaelyn and everyone,

On Fri, Dec 15, 2023 at 05:25 PM, Kaelyn wrote:

> On Thursday, December 14th, 2023 at 10:21 PM, John Kehayias
>  wrote:
>
>>
>> Hi Guix,
>>
>> In light of (more) CVEs in xwayland, see
>> <https://lists.x.org/archives/xorg-announce/2023-December/003435.html>,
>>
>> with already pending security updates, see
>> <https://issues.guix.gnu.org/67136>, I would like to prioritize
>>
>> getting that fixed in master. The tricky thing is that, according to
>> 67136, the xwayland update needs newer xorgproto, which corresponds to
>> many rebuilds. (The related CVEs in xorg-server have been pushed
>> already as effectively minor version bumps.)
>>
>> Where is the most efficient branch for this, that could take these
>> rebuilds to be merged to master soon (whatever soon is for a scope of
>> something like 22k affected packages)?
>>
>> I was thinking to put that update and mesa, since it had a new stable
>> release after the current one never got updates, on mesa-updates and
>> merge once builds are done assuming no issues. Again, the potential
>> sore spot is xorgproto I would say. I could see about any other
>> pending/urgent related changes, but I'm not aware of any off the top
>> of my head and want to let this move quickly. I also don't want to
>> jump the queue sending other branches to rebuild everything again.
>
> This doesn't seem unreasonable to me, for picking up both the new mesa
> release and the latest xwayland security fixes.
>
>> I'll test things locally in the meantime, but please chime in. If I
>> don't hear anything too urgent I'll update the mesa-updates branch to
>> start builds at least. I've also cc'ed some names I think will be
>> knowledgeable about some current branches.
>>

I've pushed 3 patches (mesa, xorgproto, xorg-server-xwayland) to
mesa-updates after merging in master. The farm is building away.

The request for merging is at <https://issues.guix.gnu.org/67875> with
some details. In short, running into some issues with builds "failing"
because they just die or "missing derivation" errors. I'm restarting
what I see that seems higher impact, but is there anyway to restart
all the failed builds or ones with missing dependencies?

Also, gtk for i686-linux is failing a test and I don't know why. With
a newer version incoming from the gnome team I would just go for
disabling that test if I knew how...

>> And thanks to Kaelyn (also cc'ed) for the pending xwayland patches!
>
> You're welcome! I've been working on updating my patch set to xwayland
> 23.2.3, but it's been taking a while to build the update because most
> of the dependency stack on core-updates apparently needed rebuilding
> locally (presumably from a lack of recent substitutes unrelated to the
> xorgproto-triggered rebuilds, but that's based on my computer churning
> away at the build for the past day or so, and not having checked guix
> weather yet--I even ran into an issue with coreutils-minimal failing a
> test when /tmp was a btrfs partition, that I got past by mounting a
> tmpfs on /tmp).
>
> Cheers,
> Kaelyn
>

Thanks! I saw you had posted the latest version and that's what I
included. On x86_64-linux at least everything has built fine for
those, but the larger world remains to be seen.

Would still like confirmation from other branches about what they want
to do, but we have some time while things build. And builds get
restarted.

Thanks!
John

Possible to separate out tk from python?

[John Kehayias]

Hi Guix,

Quick(?) question if someone happens to know: can we separate out the tk 
dependency from the python package, for instance by making the tk output of 
python a separate package?

I'm asking because I've realized that it is through tk that python, and thus a 
good chunk of all packages, depend on things like libx11 and xorgproto. In 
updating xorgproto for xorg-server-xwayland update, this causes python to be 
rebuilt. I saw the same when ungrafting libx11 but didn't look close enough to 
realize it is through tk.

(As an aside, in using python as a user, it is easy to miss the python:tk 
output when you need it. And without searching I'm guessing many fewer packages 
depend on python:tk than the default output.)

I don't know enough about how python is built (and then used in building) off 
the top of my head to know if this is possible. But if it was, I think it could 
untangle lots of the package graph.

Thanks in advance!
John

xwayland security updates, to mesa- or core-updates or ?

[John Kehayias]

Hi Guix,

In light of (more) CVEs in xwayland, see
,
with already pending security updates, see
, I would like to prioritize
getting that fixed in master. The tricky thing is that, according to
67136, the xwayland update needs newer xorgproto, which corresponds to
many rebuilds. (The related CVEs in xorg-server have been pushed
already as effectively minor version bumps.)

Where is the most efficient branch for this, that could take these
rebuilds to be merged to master soon (whatever soon is for a scope of
something like 22k affected packages)?

I was thinking to put that update and mesa, since it had a new stable
release after the current one never got updates, on mesa-updates and
merge once builds are done assuming no issues. Again, the potential
sore spot is xorgproto I would say. I could see about any other
pending/urgent related changes, but I'm not aware of any off the top
of my head and want to let this move quickly. I also don't want to
jump the queue sending other branches to rebuild everything again.

I'll test things locally in the meantime, but please chime in. If I
don't hear anything too urgent I'll update the mesa-updates branch to
start builds at least. I've also cc'ed some names I think will be
knowledgeable about some current branches.

And thanks to Kaelyn (also cc'ed) for the pending xwayland patches!

Thanks!
John

Re: bug#67790: New signing key

[John Kehayias]

On Thu, Dec 14, 2023 at 11:16 AM, Leo Famulari wrote:

> On Wed, Dec 13, 2023, at 22:17, John Kehayias wrote:
>> And I assume all this was just to use a new key (did I see some
>> mention of subkeys on #guix? that's what I use) and not because of
>> something bad happening to the old one right?
>
> I don't know if anything bad happened to the old key. That's
> fundamentally unknowable. But I decided to start using a new key.

I suppose I should have been more specific than "something bad" :) I
merely meant this wasn't an actual security issue of losing control of
a private key, but merely moving to a new one for other reasons.

In any event, this is a good reminder (to myself) to have backups of
private keys somewhere safe!

Re: bug#67790: New signing key

[John Kehayias]

On Wed, Dec 13, 2023 at 09:10 PM, Leo Famulari wrote:

> On Tue, Dec 12, 2023 at 12:02:33PM -0500, Maxim Cournoyer wrote:
>> Note that I believe you can simply update to your new key yourself.
>> You'll want to add your new key to the keyring branch, then adjust the
>> .guix-authorizations file with its new keygrip.
>
> Thanks, I pushed to 'keyring' as
> 935e3c9e93548a566cf3b3039b0822d4179974e4, and to 'master' as
> 4c4222f32a2906b7bcab74fab70ff2c2f152e8eb.
>

Just saw, thanks for the update.

And I assume all this was just to use a new key (did I see some
mention of subkeys on #guix? that's what I use) and not because of
something bad happening to the old one right?

John

Re: bug#66964: Request for merging "mesa-updates" branch

[John Kehayias]

Guix-ers,

On Wed, Nov 15, 2023 at 08:28 AM, Efraim Flashner wrote:

> On Tue, Nov 14, 2023 at 08:11:08PM +0000, John Kehayias wrote:
>> Hi Kaelyn,
>>
>> On Sun, Nov 12, 2023 at 08:01 PM, Kaelyn wrote:
>>
>> > Hi,
>> >
>> > I've just submitted a pair of patches for the mesa-updates branch:
>> > <https://issues.guix.gnu.org/67136> updating xorgproto and
>> > xorg-server-xwayland. The xorgproto is a high-impact update (guix
>> > refresh reports rebuilding 8710 packages would ensure 22871 dependent
>> > packages are rebuilt), but required to update to the latest xwayland
>> > as xwayland requires a newer version of presentproto than in the
>> > current guix xorgproto package. The updating and ungrafting of mesa
>> > and a number of X.org related libraries seemed like a good time (and
>> > place) to update xorgproto as well.
>> >
>> > Cheers,
>> > Kaelyn
>>
>> Thanks for the patches. I think mesa-updates in this current iteration
>> is set on builds (ended up being a lot more due to the ungrafting but
>> seems done on our main architectures for several days now). I had to
>> make some other changes to fix some larger breakages but at this point I
>> think it will just be taking us back in the build queue too much.
>>
>> So I think it would make more sense on the next big rebuild, either
>> core-updates (talk about doing that with more ungrafts right now) or
>> I'll do mesa-updates again when the next release of mesa hits. Or maybe
>> it makes sense to just do another branch for xwayland?
>>
>> Open to ideas! I'll send a separate message soon on the status of
>> mesa-updates and see what people think, but my thought was to merge this
>> to master in the next day or so if there are no objections.
>
> If the mesa branch is ready to merge so soon then I think we should just
> get that merged and then I'll rebase the rust-team branch on top of new
> master.  The rust-team branch is also ready to merge, but we're way
> behind on aarch64 substitutes.  Either way the substitute servers will
> be rebuilding all of rust so I think it'd be better to merge in
> mesa-updates and then do rust.

Merged as 79765b40fd9b4921b531284c589ace8a2c89a6ea woop!

We got good coverage on x86_64, i686, powerpc64le, aarch64 (all
-linux) especially from Bordeaux. Unfortunately armhf got stuck even
with prodding and waiting, but hopefully it will recover. There may be
some slight catching up across the board with recent issues on Berlin,
but prior to things getting wonky it was looking good (of course all
that happened right when I wanted to merge the other day).

Thanks to Efraim for some fixes and especially getting non-x86 in
better shape.

Feel free anyone to ping me on patches/bugs due to this merge. And
please enjoy updated mesa, fixes to gtk4 applications, some less
grafts, and more.

John

PS: I'll return to mesa-updates soon with next major mesa update and
pending related patches, or in core-updates if that is getting close.

Re: [PATCH] gnu: xorg-server: Update to 21.1.9.

[John Kehayias]

Dear Kaelyn,

On Mon, Nov 27, 2023 at 08:46 PM, Kaelyn wrote:

> Hi,
>
> I wanted to bring folks' attention to
>  which updates xorg-server, including
> a number of security fixes. The patch has been pending for about 17
> days now, and while the QA badge reports "failed" I just spot-checked
> some of the failures and they seem to be unrelated (e.g. a lot of
> builds going from unknown to blocked or vice versa, the one new
> failure for aarch64 being a large download test in the onionshare
> package, etc).
>

Thanks for the update. Yes, QA looked good to me too, all things
considered.

> Is there anything I can do to help the process along? It may also be
> worth noting that "guix refresh -l xorg-server" reports 125 rebuilds.
> I also checked and the update to xorg-server does not appear to alter
> the derivation for the xorg-server-for-tests (which is still at
> version 21.1.1).
>
> Cheers,
> Kaelyn

No, you did exactly what you needed to. I did see this patch when it
came in and was just giving a bit for QA to do the builds. That took
longer, I got distracted hoping I could merge mesa-updates first, then
hit CI delays...all that is to say I should have communicated I had
this on my radar.

Sorry about that! I appreciate the patch and the nudge.

Pushed as 06e0f638abd36f816a221af4542ca4a850d7af2d with a minor tweak
to the commit message to note [security fixes] at the top. I built it
locally for x86_64 with mesa-updates merged.

Which reminds me to make sure we have a way to flagging security
updates just like other teams/tags and get them priority. Now on the
security team, it is a first priority.

Thanks again!
John

Re: Upgrading Guix's security team

[John Kehayias]

Hi Ludo’ and everyone else,

On Wed, Nov 22, 2023 at 07:16 PM, Ludovic Courtès wrote:

> Hello,
>
> Efraim Flashner  skribis:
>
>> On Fri, Nov 17, 2023 at 11:31:41PM -0500, Maxim Cournoyer wrote:
>
> [...]
>
>>> > If maintainers agree (Cc’d), I invite you to add your name and a
>>> > termination date to the security page, remove my name, and subscribe to
>>> > guix-security.  We should add a term for other people on the team too.
>>> >
>>> > How does that sound?
>>>
>>> Sounds good to me!
>>
>> Sounds good to me too.
>
> I added John and removed myself from the security page in guix-artwork
> commit 1bd9d383cc1de4cf0eb220129c065a98332b798b.
>

Thanks and happy to be a part of the team!

> I’ve also unsubscribed Andreas and myself from the list; there are now 4
> people subscribed (we should check whether the 4th person wants to be
> officially involved).
>
> Leo, Tobias, and John: What would be a good end-of-term date for each
> one of you?  As I see it, it wouldn’t mean you cannot do an additional
> term but rather that you’ll have an opportunity to leave and that you’ll
> do your best to be around by then.
>

Seeing as how I'm often away from any Guix computers for a few weeks
at a time over the summer, let me say roughly 6 months, ending on May
15th.

As you say, likely I would be happy to continue, though parts of
summer I tend to be away so it would be good to not have us
shorthanded then. Or maybe staggering when people join/leave with some
overlap is a good plan.

> Thanks again for volunteering, John!
>
> Ludo’.

Welcome and hoping to serve the Guix community well!

John

Re: mesa-updates: call for patches

[John Kehayias]

Hi,

On Sat, Nov 18, 2023 at 11:07 AM, Christopher Baines wrote:

>
> John Kehayias  writes:
>
>> Hi everyone,
>>
>> Update below:
>>
>> On Sun, Nov 05, 2023 at 11:47 PM, John Kehayias wrote:
>> [snippy snip snip]
>>
>> At this point I feel we are just about ready to go, unless there are
>> objections?
>>
>> Substitute coverage, according to
>> <https://qa.guix.gnu.org/branch/mesa-updates> is good on x86_64 and
>> i686 (about 95% and 83%, respectively) while, as usual, other
>> architectures are behind. The next best is aarch64 at 54% on bordeaux,
>> and then falling to 24% for armhf, with others we build in the teens.
>> I think this is to be expected? In any event builds continue very
>> slowly and in the past I think this is about where we merge.
>
> I think some changes have been pushed since this email, since the
> aarch64 substitute availability has dropped from 54% to 25%.
>

Yes, Efraim chimed in to help on some other architectures and some big
rebuilds were/are happening for those. I see them slowly ticking up
but it will still need some time.

>> So, shall I merge this to master in the next couple of days? I've been
>> merging master into mesa-updates smoothly so far. Please do check and
>> feel free to object if this needs more time.
>
> I guess this has been held up by the changes on the 15th, but still, I
> think we need to wait for substitute availability to improve more prior
> to merging, unless there's a specific and significant reason why we
> don't want to wait.
>

Yes, agreed. I'm not as clear on how well we do typically on non-x86
but getting a sense of it now, which is why I wanted to ask.

> Targets are arbitrary, but guix weather defines ☀ as 80%+, so I think
> that's what we should aim for at least for x86_64-linux, i686-linux,
> aarch64-linux and armhf-linux. This isn't just about substitute
> availability though as this is key for discovering what things fail to
> build.
>

I think this is something we could better clarify and quantify as many
of us probably only pay attention to x86_64, where for others we can
be strapped for both hardware and people. So I didn't want to wait for
some substitutes that would never come but also don't want to
inconvenience people on other architectures, especially if builds
there take much much longer in the first place.

Perhaps we can look at some historical data on what we've hit in
substitute coverage and try to at least keep up with that if not set
some goals for better coverage? While we might also expect further
difficulties as some get left behind by upstream (as we've had to work
around rust on i686 so far, I believe).

All that is to say, yes, let's make sure we have good substitute
coverage and are clear on what architectures we want to make sure
users get substitutes for.

> Obviously delays in merging aren't ideal, but we should tackle the
> problems around this, maybe by deciding that testing and providing
> substitutes for ARM isn't a priority and thus isn't something we should
> wait for, or look at getting more ARM hardware to speed up the process
> (we also have a lack of x86_64 hardware on the bordeaux build farm).
>

Agreed. We should have some clear Guix-wide standards and goals. I'm
sure we can get some hardware from Guix-ers and/or funding too,
especially if people know exactly what it will go towards improving.

Thanks for chiming in here and all your work on this front!

In the meantime, I'll go back to refreshing the CI and QA pages every
so often to make sure we keep getting closer...

Re: mesa-updates: call for patches

[John Kehayias]

Hi everyone,

Update below:

On Sun, Nov 05, 2023 at 11:47 PM, John Kehayias wrote:
[snippy snip snip]
>>
>> Happy to! Substitutes will eventually become available, but there's
>> quite a few builds to be done. This takes care of some ungrafts and
>> updates with I hope minimal disruption. I'll be keeping an eye out and
>> using locally as well. Please test and report, thanks everyone!
>>
>> John
>
> An issue was created to track merging the mesa-updates branch here:
> <https://issues.guix.gnu.org/66964>. Please use that bug number as
> needed (and cc me or use wide-reply in emacs debbugs).

At this point I feel we are just about ready to go, unless there are
objections?

Substitute coverage, according to
<https://qa.guix.gnu.org/branch/mesa-updates> is good on x86_64 and
i686 (about 95% and 83%, respectively) while, as usual, other
architectures are behind. The next best is aarch64 at 54% on bordeaux,
and then falling to 24% for armhf, with others we build in the teens.
I think this is to be expected? In any event builds continue very
slowly and in the past I think this is about where we merge.

I should note: please check for any breakages. I didn't expect too
much, but did get more than I thought. It seems the ungrafting version
changes caused some things to fail. Also, the libx11 ungraft mean
python and rust were rebuilt, with the many packages that entails.

I fixed big ones I saw, like QT (unrelated: it was libxkbcommon
upgrade), but other leaf packages I saw had tests failing for reasons
I didn't see. For instance, php fails tests. The current ones are due
to the curl update, but updating php and removing an obsolete patch
had a different test fail. It would be great if someone more familiar
will take a look. With few dependents I figure this can just be done
on master after the merge.

So, shall I merge this to master in the next couple of days? I've been
merging master into mesa-updates smoothly so far. Please do check and
feel free to object if this needs more time.

Thanks everyone,
John


PS: I forgot to email the various patches/issues that are done on
mesa-updates, as listed in a previous message. I will do that too.

Re: bug#66964: Request for merging "mesa-updates" branch

[John Kehayias]

Hi Kaelyn,

On Sun, Nov 12, 2023 at 08:01 PM, Kaelyn wrote:

> Hi,
>
> I've just submitted a pair of patches for the mesa-updates branch:
>  updating xorgproto and
> xorg-server-xwayland. The xorgproto is a high-impact update (guix
> refresh reports rebuilding 8710 packages would ensure 22871 dependent
> packages are rebuilt), but required to update to the latest xwayland
> as xwayland requires a newer version of presentproto than in the
> current guix xorgproto package. The updating and ungrafting of mesa
> and a number of X.org related libraries seemed like a good time (and
> place) to update xorgproto as well.
>
> Cheers,
> Kaelyn

Thanks for the patches. I think mesa-updates in this current iteration
is set on builds (ended up being a lot more due to the ungrafting but
seems done on our main architectures for several days now). I had to
make some other changes to fix some larger breakages but at this point I
think it will just be taking us back in the build queue too much.

So I think it would make more sense on the next big rebuild, either
core-updates (talk about doing that with more ungrafts right now) or
I'll do mesa-updates again when the next release of mesa hits. Or maybe
it makes sense to just do another branch for xwayland?

Open to ideas! I'll send a separate message soon on the status of
mesa-updates and see what people think, but my thought was to merge this
to master in the next day or so if there are no objections.

Thanks!
John

Re: mesa-updates: call for patches

[John Kehayias]

> Details of changes below, which included some more ungrafting:
>
>>> Here is a list of what has been on my radar and I'll be looking to
>>> apply on the mesa-updates branch. If there's something I missed and
>>> you think makes sense here (e.g. lower level graphical/X related
>>> libraries) please let me know. And yes, I need to make a Mesa team. It
>>> is on my list!
>>>
>>> 1. ungraft libx11 and libxpm
>>>
>
> These are done.
>
> Also, since I didn't realize libx11 means building python, and rust,
> and so on, I figured I would ungraft more that was being rebuilt
> anyway. So nghttp2 and curl have been ungrafted.
>
> The only more involved change was in ungrafting curl (to keep another
> test skip for Hurd), in
> 
> and then running parallel tests (very nice speed up for building curl
> I came across in the documentation!) in
> .
> Please do take a look in case I missed something here, but all built
> fine locally on x86_64 at least.
>
>>> 2. update mesa (23.2.1)
>>>
>>> 3.  (libepoxy fix for GTK, see
>>> )
>>>
>
> These are done.
>
>>> 4.  (mesa vulkan search-paths)
>>>
>
> I don't think this is a correct change as written (search-path should
> be in vulkan-loader if I'm understanding what is supposed to happen
> here). Anyway, will follow up on that issue and left it out for now.
>
>>> 5.  (sdl2 vulkan-loader; and update 
>>> sdl2?)
>>>
>>> 6.  (libdrm update; to even newer 
>>> version now)
>>>
>>> 7.  (libxkbcommon update)
>>>
>>> 8.  (pixman update)
>>>
>
> All these done (sdl2 was also updated to latest version).
>
>>> 9. submit patch for mesa team
>>>
>
> But not this one, the easy one :) It will be sent.
>
>>> Thanks everyone! And when you see the updates pushed on mesa-updates
>>> and builds become available, please do test and let me know. Or if
>>> you'd like to join this team of one, happy to have you on board.
>>
>> Sounds like a fine plan!  Thank you for tackling this!
>
> Happy to! Substitutes will eventually become available, but there's
> quite a few builds to be done. This takes care of some ungrafts and
> updates with I hope minimal disruption. I'll be keeping an eye out and
> using locally as well. Please test and report, thanks everyone!
>
> John

An issue was created to track merging the mesa-updates branch here:
. Please use that bug number as
needed (and cc me or use wide-reply in emacs debbugs).

Re: mesa-updates: call for patches

[John Kehayias]

Hi Maxim and guix-devel,

On Tue, Oct 31, 2023 at 09:27 AM, Maxim Cournoyer wrote:

> Hi John,
>
> John Kehayias  writes:
>
[...]
>
> If you're going to put a branch for it and build it whole, I'd simply
> merge it whole after it's done building and hasn't regressed on package
> failures or other things.
>
[...]
>
> I'd say go for a branch!
>

Yes, pushed a bunch of changes to mesa-updates after I built up
through mesa locally. Then I hit the rust compiler chain so I decided
to let the CI take over and build everything else too.

Details of changes below, which included some more ungrafting:

>> Here is a list of what has been on my radar and I'll be looking to
>> apply on the mesa-updates branch. If there's something I missed and
>> you think makes sense here (e.g. lower level graphical/X related
>> libraries) please let me know. And yes, I need to make a Mesa team. It
>> is on my list!
>>
>> 1. ungraft libx11 and libxpm
>>

These are done.

Also, since I didn't realize libx11 means building python, and rust,
and so on, I figured I would ungraft more that was being rebuilt
anyway. So nghttp2 and curl have been ungrafted.

The only more involved change was in ungrafting curl (to keep another
test skip for Hurd), in
<https://git.savannah.gnu.org/cgit/guix.git/commit/?id=00442f15d46cd9d5d02499827946d23426aad0ba=mesa-updates>
and then running parallel tests (very nice speed up for building curl
I came across in the documentation!) in
<https://git.savannah.gnu.org/cgit/guix.git/commit/?id=d70f2b788e56545f93dc24238d2617e20fde7460=mesa-updates>.
Please do take a look in case I missed something here, but all built
fine locally on x86_64 at least.

>> 2. update mesa (23.2.1)
>>
>> 3. <https://issues.guix.gnu.org/65375> (libepoxy fix for GTK, see
>> <https://issues.guix.gnu.org/64981>)
>>

These are done.

>> 4. <https://issues.guix.gnu.org/65155> (mesa vulkan search-paths)
>>

I don't think this is a correct change as written (search-path should
be in vulkan-loader if I'm understanding what is supposed to happen
here). Anyway, will follow up on that issue and left it out for now.

>> 5. <https://issues.guix.gnu.org/65153> (sdl2 vulkan-loader; and update sdl2?)
>>
>> 6. <https://issues.guix.gnu.org/64637> (libdrm update; to even newer version 
>> now)
>>
>> 7. <https://issues.guix.gnu.org/66727> (libxkbcommon update)
>>
>> 8. <https://issues.guix.gnu.org/64639> (pixman update)
>>

All these done (sdl2 was also updated to latest version).

>> 9. submit patch for mesa team
>>

But not this one, the easy one :) It will be sent.

>> Thanks everyone! And when you see the updates pushed on mesa-updates
>> and builds become available, please do test and let me know. Or if
>> you'd like to join this team of one, happy to have you on board.
>
> Sounds like a fine plan!  Thank you for tackling this!

Happy to! Substitutes will eventually become available, but there's
quite a few builds to be done. This takes care of some ungrafts and
updates with I hope minimal disruption. I'll be keeping an eye out and
using locally as well. Please test and report, thanks everyone!

John

Re: mesa@23.1.4: missing symbols

[John Kehayias]

Hi Sergio,

On Fri, Nov 03, 2023 at 06:05 PM, Sergio Pastor Pérez wrote:

> Hi.
>
> I've noticed that the `mesa' package we provide is missing some
> symbols that according to the OpenGL specification should be present on
> the `libGL.so.1` library.
>

I'll put the punchline up top with a few more details below. But the
symbol you want is in the libglvnd package:

--8<---cut here---start->8---
❯ nm -gD $(guix build libglvnd)/lib/libGL.so | grep glBindTextureUnit
0004cb60 T glBindTextureUnit
0004cb80 T glBindTextureUnitParameterEXT
--8<---cut here---end--->8---

> The following commands demonstrate the issue:
>
> $ guix build mesa
> /gnu/store/ds15k8bzqf1m861w17hshd8i54pffig9-mesa-23.1.4-bin
> /gnu/store/hxvp1sp897rnnpbpb0xk887m4822gf77-mesa-23.1.4
>
> $ nm -gD 
> /gnu/store/hxvp1sp897rnnpbpb0xk887m4822gf77-mesa-23.1.4/lib/libGL.so.1 | grep 
> glBindTextureUnit
>
> $ glxinfo | grep "OpenGL version"
> OpenGL version string: 4.6 (Compatibility Profile) Mesa 23.1.4
>
>

Indeed, I found this very puzzling as you did. I looked for other
symbols defined in the same Mesa headers and it was hit or miss
(mostly miss).

> As you can see the grep does not match any symbol. In contrast, if we do
> the same under an Ubuntu installation, we can see the following output:
>
> $ nm -gD /usr/lib/x86_64-linux-gnu/libGL.so.1 | grep glBindTextureUnit
> 00046b60 T glBindTextureUnit
> 00046b80 T glBindTextureUnitParameterEXT
>
> $ glxinfo | grep "OpenGL version"
> OpenGL version string: 4.6 (Compatibility Profile) Mesa 
> 23.0.4-0ubuntu1~22.04.1
>

I have a laptop with Arch on it and I saw the same thing you did on
Unbuntu. However, I decided to double check where libGL comes from and
it is owned not by the mesa package there but libglvnd! And that's how
I found libGL also in our libglvnd package with the symbol you were
looking for.

Unfortunately 'guix locate' didn't help me find other libGL in this
case (I tried that first) since I don't have libglvnd installed in any
profile. But a future upgrade or remote database of files in all our
packages will make this simpler.

>
> This symbol is also present on the `libGL.so.1` provided by the
> proprietary Nvidia driver:
>
> $ guix build nvidia-driver
> /gnu/store/8xhdq3rf9k80n6vz5cwi7z1dg5wjq002-nvidia-driver-515.76
>
> $ nm -gD 
> /gnu/store/8xhdq3rf9k80n6vz5cwi7z1dg5wjq002-nvidia-driver-515.76/lib/libGL.so.1
>  | grep glBindTextureUnit
> 000476c0 T glBindTextureUnit
> 00047700 T glBindTextureUnitParameterEXT
>
> $ glxinfo | grep "OpenGL version"
> OpenGL version string: 4.5 (Compatibility Profile) Mesa 23.1.4
>

Proprietary stuff aside, this comes back to a question raised before
for Guix that we haven't tackled, and that's libglvnd and loading
other gl drivers (yes, could be proprietary, but also more generally
on a foreign distro). Building mesa with libglvnd support is easy
enough, but the proper way to set this up to work well on Guix and
foreign systems I'm not so sure. It has been a while since I looked at
it but happy for someone to bring this up again or propose what we
should do to make it all work seamlessly. That's what libglvnd is for
after all, to dispatch gl calls to the right driver. (I think the idea
is that mesa is a provider of an implementation and the more general
top level for these symbols is elsewhere, libglvnd which could
dispatch it to mesa?)

> I've noticed the problem because I'm packaging a rust up that I wanted to
> contribute as an example on how to build applications that use meson to
> build rust packages. This app uses `rust-minidl' to load `libGL.so.1` at
> runtime. The app tries to use the `glBindTextureUnit` symbol and panics.
>
> Does anyone know why this symbol could be missing? Could we fix the
> `mesa' package? I would like users of the open source drivers to be able
> to run the package.
>
> Thanks,
> Sergio.


So the first thing is to use libglvnd in your packaging, which will
get you the symbol you need. Whether more is needed for this to work
properly (mesa input also or something else) I don't know. And if
other packages in Guix actually need libglvnd (and mesa) to work I
also don't know but I would guess comes up in a bug report if it does
happen.

Hope that helps! And I'm glad it wasn't some huge oversight in how we
build mesa (as far as I can tell) which left me scratching my head at
first.

John

Re: Meet Guix at Capitole du Libre in Toulouse, nov. 18-19

[John Kehayias]

On Fri, Nov 03, 2023 at 01:39 PM, Ricardo Wurmus wrote:

> Luis Felipe  writes:
>
>> El 26/10/23 a las 14:03, Luis Felipe escribió:
>>> El 25/10/23 a las 21:17, Julien Lepiller escribió:
 The print service I usually use has a lot of options for flyers,
 {10,15,20,30}*{10,15,20,30}, 12*12, 21*29.7 and 30*40 cm. I think
 15*20 would be best.

 Visit cards would be 3.5*7.5, 5.5*8.5 or 5*9 cm
>>> Thanks, Julien, I'll hopefully have something ready for next
>>> week. I'll let you all know.
>>
>> Here's a visit card in the same style of the kakemono:
>>
>> 
>>
>> The source is in 
>> (promotional/cards).
>
> Wonderful work, Luis Felipe!  I’d love to have some of these for future
> presentations or conferences.

Seconding this, looks great and adding it to my list of Guix swag I
would like to have one day!

mesa-updates: call for patches

[John Kehayias]

Hi Guix-ers,

Time for another round of Mesa and friends updates! I've been waiting for 
another Mesa release but seems the 23.2 series has stalled out and now is a 
good time before 23.3 or 24. I've been using 23.2.1 for some packages locally 
without issue. I would like to get this branch built and then promptly merged, 
assuming no showstoppers. Actually, probably just cherry-pick since it is 
should just be a few commits and I find that cleaner, personally (but happy for 
advice here).

There's already a CI build job and I'll submit the request to merge for 
hopefully QA to compare. I will also do some ungrafting. Maxim, I'm CC'ing you 
since you mentioned the core-updates cycle about to happen, so if this branch 
takes some time maybe we'll be better off just combining. Let me know. 
Otherwise I hope to have the branch building in a few days and then let it 
churn.

Here is a list of what has been on my radar and I'll be looking to apply on the 
mesa-updates branch. If there's something I missed and you think makes sense 
here (e.g. lower level graphical/X related libraries) please let me know. And 
yes, I need to make a Mesa team. It is on my list!

1. ungraft libx11 and libxpm

2. update mesa (23.2.1)

3.  (libepoxy fix for GTK, see 
)

4.  (mesa vulkan search-paths)

5.  (sdl2 vulkan-loader; and update sdl2?)

6.  (libdrm update; to even newer version 
now)

7.  (libxkbcommon update)

8.  (pixman update)

9. submit patch for mesa team

Thanks everyone! And when you see the updates pushed on mesa-updates and builds 
become available, please do test and let me know. Or if you'd like to join this 
team of one, happy to have you on board.

John

Re: branch master updated: gnu: Add passff.

[John Kehayias]

Hi Clément,

On Sat, Oct 28, 2023 at 05:05 PM, Clément Lassieur wrote:

> On Sat, Oct 28 2023, Christopher Baines wrote:
>
>> This passff-host package looks a bit odd to me, one thing to mention is
>> that guix show says it has no dependencies, but I don't think that's
>> correct:
>>
>>   ./pre-inst-env guix show passff-host
>>   name: passff-host
>>   version: 1.2.3
>>   outputs:
>>   + out: everything
>>   systems: x86_64-linux mips64el-linux aarch64-linux powerpc64le-linux 
>> riscv64-linux
>>   + i686-linux armhf-linux i586-gnu powerpc-linux
>>   dependencies:
>
> I imagine it's a bug in `guix show`?  As doc says:
>
>• Gexps carry information about the packages or derivations they
>  refer to, and these dependencies are automatically added as inputs
>  to the build processes that use them.
>

Right, it uses gexps but I think here the better and more explicit
style would be to use inputs/native-inputs. Then instead of
referencing directly like #$ use
#$(this-package-input "package-name") to get the store path. This I
think is clearer and I believe better allows for inheritance,
input-rewriting, and so on.

Feel free for anyone else to chime in on this point, I'm always
looking to learn to improve my own packaging and review, but this is
what I understand is preferred when possible.

>> Was this change sent as a patch to guix-patches?
>
> No it wasn't.

The mantra I've heard, and agree with, is that the
trivial-build-system is anything but trivial. Not saying it wasn't the
best choice here, or has anything to do with the above points, but
thought it worth mentioning for anyone else.

But this is also why I think it would have been better to have it go
through review. I see there's been several followup commits to improve
the style and fix things which also could have been avoided. Not a
huge deal perhaps, but I would err on the side of review for something
like this.

Of course, thanks for the contribution!

John

Re: core-updates invites to an ungrafting party

[John Kehayias]

Hi Maxim et al,

On Sun, Oct 08, 2023 at 11:12 AM, Maxim Cournoyer wrote:

> Hello Guix!
>
> The core-updates branch is still alive, and has accumulated (or plans
> to) a few changes that cause world-rebuilds, such as fixes to
> git-minimal (bug#65924) as well as docbook improvements (bug#65479) and
> fixes to the build systems so that deep input rewriting works as
> intended (bug#65665).
>
> I think we could also batch ungrafting of all grafted packages, to make
> the most out of this complete rebuild.
>

That sounds good, we have suddenly got a bunch of grafts deep in the
dependency tree.

Speaking of which, I was planning to at least ungraft libx11 and
libxpm, recipients of recent grafts for security reasons, on a
forthcoming mesa-updates branch. I'm just waiting for the next point
release of mesa, since 23.2.1 is actually the first release where
typically a first .1 release is considered the start of the stable
series. (Though 23.2 has had a long release candidate time.)

So, what are we thinking of the time to build/merge core-updates? I
was hoping to do some ungrafting and updating in the mesa-related
ecosystem this week, depending on upstream.

I'll start a separate thread soon to ask for what patches to include
there that I don't already know about, but I'm happy to include
similar scope ungrafting if that makes sense before core-updates.

What does everyone think? I think it is more a question of
timing/resources, either doing some ungrafting earlier but then having
more builds again soon after (e.g. glibc ungraft), or knocking some of
it out earlier with a smaller scope.

> To recall, the policy surrounding what goes to core-updates is still
> unchanged (per the Contributing section of our manual), except for areas
> covered by teams (which is still patchy at best -- have you considered
> joining teams?)
>

And thanks for pointing this out. I do hope we continue building teams
and scopes for them so core-updates doesn't end up getting too
unwieldy. I'm optimistic of a quicker merge timeline here as well, the
ungrafting being a nice immediate reason to do this.

> What do you think?  If you are interested in participating in the
> effort, you can send your ungrafting patches for review with the
> --subject-prefix='PATCH core-updates' prefix or if you are a committer
> you could simple version bumps to core tools that have been posted to
> guix-patches, if any.

Thanks Maxim for getting things rolling here!

John

Re: Fw: Question regarding qmk firmware

[John Kehayias]

Hello,

On Sun, Oct 08, 2023 at 10:34 AM, Ekaitz Zarraga wrote:

> Hi
>
> I want to forward this message to guix-devel because it is a clear
> case of some (actually good) technical decision affecting users in
> unexpected ways.
>
> Now, after the change, a user might run `guix search avr-toolchain`
> and find nothing. Same for ARM.
>

In this case would it have helped to deprecate the package? Or can we
(ab)use this as a way to notify a user that either something has
changed (use a procedure now) or that a package you might expect at
this name is available some other way?

> This is a shame, because we have toolchains for those architectures
> but converting them to a function that returns the package leave many
> users that are not used to read guix's code thinking those packages
> are gone.
>
> Maybe we should create some kind of fake packages that show up in
> `guix show` and `guix search` that have a short tutorial on how to use
> packages that come from a function like these.
> This way providing the same interface for every package regardless
> where they are coming from.
>

At the very least this should be documented, perhaps adding to
information about the kernel in the manual and generally
customizing/building your own.

I like the idea in general of making sure people can find things and
if they are not where you'd expect not having a hard time to find
them. Some tips in a package description about how to use or where to
look in the manual for information would be good, but I don't think
we'd want to get too verbose here, adding another maintenance point
that should be proper documentation (or cookbook).

As an example, we don't need to always say how to add udev rules from
a package, but letting users know (if it is not obvious from the name)
that rules are included and should be added to a system configuration
for something to work (pointing to the manual about udev service) I
think can be helpful. I don't know though, I guess the package's
documentation itself needs to tell a user how to use it, and then one
looks in the Guix manual how to add udev rules.

Anyway, perhaps I run on a tangent here.

John

> I leave it as food for thought.
>
> Thanks,
>
> Ekaitz
>
>
> --- Forwarded Message ---
> From: Fredrik Salomonsson 
> Date: On Saturday, October 7th, 2023 at 21:23
> Subject: Question regarding qmk firmware
> To: help-guix 
>
>
>> Hi,
>>
>> Today I was tweaking a keymap for one of my qmk based keyboards but some
>> of the packages I used when building the firmware has been removed. My
>> commad was as follows:
>>
>> `sh guix shell avr-toolchain dfu-programmer qmk -- qmk flash -kb
>> ergodox_infinity -km plattfot -bl dfu-split-left`
>>
>> But `avr-toolchain` is gone. When I tried to just drop it and see if it
>> worked I get
>>
>> `Ψ Compiling keymap with make --jobs=1
>> ergodox_infinity:plattfot:dfu-split-left QMK Firmware 0.14.19 Making
>> ergodox_infinity with keymap plattfot and target dfu-split-left
>> /gnu/store/rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16/bin/sh:
>> line 1: arm-none-eabi-gcc: command not found`
>>
>> It seems `arm-none-eabi-toolchain` is also removed. Looking at the
>> commit history for guix it looks like they got replaced by
>> [proceduers][0] instead.
>>
>> [0]
>> /?id=35c1df5bd6317b1cd038c1a4aca1c7e4a52d4d93
>>
>> My question is how do I get access to the arm-none-eabi-toolchain from
>> the commandline with guix shell?
>>
>> Thanks
>>
>> --
>> s/Fred[re]+i[ck]+/Fredrik/g

Re: Need people to help with kernel updates

[John Kehayias]

Hi Leo,


On Sat, Oct 07, 2023 at 02:04 PM, Leo Famulari wrote:

> Hello,
>
> For a few years, I've been handling updates of the linux-libre kernel by
> myself. Now I want some more people to help with this.
>

Just wanted to say thanks for this work that goes on mostly behind the
scenes; I've never had any kernel hiccups related to Guix. Let's hope
I didn't just jinx that, but our rollbacks and generation selection at
boot actually make it a lot less painful to experiment here.

John

> The work itself is fairly mechanical and updates occur about once a
> week. It takes about 30 minutes to prepare the patches and push them to
> CI or send them to the mailing list.
>
> When a new major kernel version is released, then we also need to make a
> new kernel config file, which takes a few hours in total.
>
> There is plenty of support for the CI and QA infrastructure to build the
> kernels, so you don't need a powerful computer.
>
> This isn't the sort of the task that needs to be performed by a single
> person. The work could be spread, like most other packages in Guix.
>
> If you want to join in, please reply!
>
> Leo

Upgrading Guix's security team

[John Kehayias]

Hi Guixers!

In light of the several high profile CVEs this month, which were/are being 
handled and more coming (curl joins the chat) some of us were discussing 
improving and systematizing our security team and responses. My thanks to 
Tobias for quick review to help finalize the XOrg CVE grafts, to Liliana for 
the pending glibc fix (see <https://issues.guix.gnu.org/66348>) and updating 
curl in preparation for a critical CVE update, and Ludo for getting this 
discussion started.

Here are some quick thoughts/ideas that came up for comment:

- current security email/people can be found here, which is nicely visible 
<https://guix.gnu.org/en/security/> yet probably in need of a hand and new 
faces for an important but often thankless job; no fault to them or Guix as a 
whole, merely a good time to see how we can keep improving

- currently we are not on the OS security distribution contact list: 
<https://oss-security.openwall.org/wiki/mailing-lists/distros>; this had been 
discussed before but we will need commitment from people

- clear roles will be helpful; to me this includes at least a couple of people 
to coordinate (the majority of security issues will be handled through package 
upgrades/grafts) and people to help review and/or contact needed experts, like 
for Guix internal issues; we should make this more precise

- likewise, a clear fixed timeframe for who is on this team; keeping people 
fresh and engaged for what can suddenly be a time sensitive and critical job; I 
think this will also help spread institutional knowledge for better security 
practices in general

- members need not be experts but should be active in the community as 
committers (already a round of vetting), familiar with what issues and 
processes may arise, and willing to learn; perhaps we need a list of experts to 
consult though the current teams are a good starting point

- what are your thoughts? what are the goals and outcomes we as a distro want 
in security?

- finally, I think an internal discussion with maintainers and long time active 
committers would be helpful to get the improvements started and moving, in 
addition to this wider discussion here

And to get things started, I'm happy to volunteer myself to help coordinate on 
security, if deemed okay by our current security team, maintainers, and anyone 
else that's been helping to handle security. A coordinating role with a term of 
say 6 months to a year? Happy to provide more information and discuss here or 
privately; in short I'm not a security expert but have time and bandwidth to 
keep things moving and want to learn.

Thanks everyone, and here's to hoping the spooky season is full of fun and 
candy and less CVEs!

John Kehayias

Re: Branch (and team?) for mesa updates

[John Kehayias]

Hi Maxim,

On Sun, Jul 30, 2023 at 09:50 PM, Maxim Cournoyer wrote:

> Hi John,
>
> John Kehayias  writes:
>
> [...]
>
>> I'll open a branch merge request issue later today as per new
>> procedure for QA. Though I believe that only builds 2 branches, which
>> is occupied at the moment. Or can someone set a separate build job
>> specifically for mesa-updates, especially if we think it is a good
>> idea to have this going forward?
>
> Do you already have admin access to Cuirass?  We can issue client certs
> for team members needing to create branches on it or restart builds
> there, etc.
>

I do not have access. The mesa-updates branch remains (and still an
active job I think, just nothing pushed since the merge). I plan on
making use of it as soon as 23.2 is out, along with a handful of
pending patches I've seen that will make sense here.

I haven't used Cuirass before but if a hand would be helpful I'm happy
to lend it (let me know if there is someone I should contact directly
or message me off list).

>>> Do we want a "Mesa team" or something a bit larger? Not sure what
>>> exactly, since "graphics" is perhaps too broad. Happy to help
>>> spearhead the Mesa front for Guix (the very package that got me first
>>> involved in the patching process).
>>>
>>
>> This is still a good question I think, of how we want to have a
>> team(s) to handle things like xorg, wayland, mesa, and related
>> packages. They are a bit all over the place in terms of scope and what
>> they touch. For now I'd like to go ahead with a regular mesa-updates
>> branch since that sees regular releases and is pretty self-contained
>> currently.
>
> It seems a 'desktop' team could make sense, covering some of the things
> listed here that makes sense / are already well separated in modules in
> Guix to avoid being added to two teams:
> <https://www.freedesktop.org/wiki/Software/>.

The problem I'm thinking of for a "desktop" team is setting the
correct scope of package files to make use of e.g. auto cc-ing on
patch submissions. Though at least (gnu packages gl) looks pretty
reasonable to start for maybe a graphics team? Maybe with vulkan?

I'm still not sure but I should probably propose something concrete
with at least myself for gl since those patches generally will go to
the mesa-updates branch for convenient building.

Anyone else want in?

John

Re: Updates for Go

[John Kehayias]

Hi Katherine,

On Wed, Aug 23, 2023 at 10:12 AM, Katherine Cox-Buday wrote:

> On 8/22/23 8:24 AM, Felix Lechner via Development of GNU Guix and the
> GNU System distribution. wrote:
>> Hi Attila,
>>
>> On Tue, Aug 22, 2023 at 6:14 AM Attila Lendvai  wrote:
>>>
>>> currently the go build system in guix does not reuse build artifacts
>>
>> Can Golang reuse build artifacts?
>
> I don't think it's recommended right now. See discussion here:
>
> - 
> - 
>
> Summary:
>
> It sounds like due to the way the compiler can optimize code (e.g.
> inlining), `buildmode=shared` is not recommended, but in the future they
> are looking at allowing linking against a single shared library.

I've not been following in detail this discussion, but where do we currently 
stand? Is the proposed Go 1.21 patch basically ready? Should we create a branch 
and build job to start seeing how far we get in making 1.21 the default Go in 
Guix?

Like others, I have a few random Go packages (a bunch locally I really need to 
clean up too) and am not familiar with the language and our packaging much. 
Still, if I can help review/push some patches and get things moving, please let 
me know.

And thanks for all your work here, it is appreciated!

John

Re: A certain new commiter

[John Kehayias]

On Sun, Aug 20, 2023 at 01:40 AM, Hilton Chain wrote:

> Hello Guix,
>
> With the commit [1] made hours ago, I have been granted commit access
> to Guix repository.
>
> Currently, I'm maintaining packages I may use and those I've touched,
> and for now I have no specific plan to move on.  This means I can have
> more time to go through the review process, and I'm glad to do so.
>
> I'm also hako on libera.chat, please contact me if there's anything I
> can help with.
>
> Thanks,
> Hilton Chain
>

Congratulations and welcome!

> ---
> This mail is signed by the key with the following fingerprint, I'll
> use it for commit signing:
> F4C2 D1DF 3FDE EA63 D1D3  0776 ACC6 6D09 CA52 8292
>
> The signing key is a subkey of
> 220F 98D9 5E86 204C 0036  DA7B 6DEC 4360 408B 4185
>
> , which is attached.  And it can be found in [2] as well.
>
> [1]:
> 
>
> [2]:
> 
> 
>

Re: The package/inherit trap

[John Kehayias]

Hi all,

On Wed, Mar 08, 2023 at 10:07 AM, Simon Tournier wrote:

> Hi Tobias,
>
> On Tue, 07 Mar 2023 at 22:11, Tobias Geerinckx-Rice  wrote:
>
>> However, merely documenting something is not enough when we have the
>> chance to fix misleading naming, as we do here.  It would be nice to
>> have, but orthogonal.
>
> I would not say it is orthogonal because renaming would not have been
> enough, at least for me, in order to get the difference with ’inherit’.
>
> For sure, it is a real trap. :-)  For instance,
>
> $ git log --grep='package/inherit' --oneline | grep use
> 5f83dd03a2 gnu: kodi/wayland: Do not use package/inherit.
> 6ecf88a6a1 gnu: poppler-next: Don't use 'package/inherit'.
> 5a5b729d66 gnu: abseil-cpp: Don't use 'package/inherit'.
> dbcf2b61b1 gnu: Fix erroneous uses of 'package/inherit'.
> 2f97a666a5 gnu: python-urllib3: Don't use 'package/inherit' on 
> replacement package.
> 4163b6d855 gnu: avahi: Don't use package/inherit.
>
> and in the light of this discussion, 5f83dd03a2 seems incorrect, no?
>
> Well, I agree that naming is important and probably the most difficult
> task. :-)
>
> Do we go for renaming + deprecated?  Then do we apply the rename to all
> occurrences in Guix?
>
> Cheers,
> simon

I know, reviving an old thread but I assume this wasn't resolved?
Throw my vote for the more explicit renaming with better doc/examples,
as one who never remembers the difference either.

John

Re: poetry not building

[John Kehayias]

Hi all,

On Fri, Jun 23, 2023 at 08:49 AM, Lars-Dominik Braun wrote:

> Hi Reza,
>
>> Poetry is not building on ci.guix.gnu.org [1]. There is a pending patch
>> [2] on the issue tracker. What is missing to apply this patch and how
>> can I help?
>
> both contributors to that issue – John and me – are busy with other
> things right now. That’s all.
>
> My proposal in the python-team branch is rather intrusive and requires
> a world rebuild. I wrote down what needs to be done here:
> 
> Some of that work has been merged into the python-team branch already,
> but I think the 3rd point is missing entirely and the 2nd one is not
> properly tested yet.
>
> Cheers,
> Lars

Yes, it is a rather intrusive change of our python world, requiring
rebuilding of everything that uses python. As noted above, you can use
something like time-machine for older guix with working poetry or use
a local copy with those patches for a newer one. Either will likely
require some local rebuilding.

Anyway, yes, I was distracted with other matters but have just
finished merging mesa-updates which took a bit longer than expected.
I'm not expert on these more internal python and python packaging
matters, so help is welcome. I will work on reviving this next though.

John

Re: Core-updates merge

[John Kehayias]

Hi Pierre!

On Fri, Jul 14, 2023 at 07:12 PM, Pierre Langlois wrote:

> Hi John!
>
> John Kehayias  writes:
>
>> Bringing back up an old thread, but
>>
>> On Fri, Apr 28, 2023 at 05:55 AM, John Kehayias wrote:
>>
>>> Dear Andreas and fellow Guix-ers,
>>>
>>> On Tue, Apr 25, 2023 at 04:09 PM, Andreas Enge wrote:
>>>
>> [...]
>>>> Each and every package is not yet in shape; please feel free to submit
>>>> patches for your favourite packages that fail to build. In particular:
>>>> - python-yubikey-manager does not build currently; work to correct this
>>>>   is underway.
>>>
>>> I've just submitted <https://issues.guix.gnu.org/63139> which does a bit
>>> more than just fix that package and would be good for a Python feature
>>> branch. I'll send the cover letter to this list for wider visibility,
>>> though the Python team was cc'ed on the series. I suppose I should add
>>> myself to the team after this.
>>>
>>
>> Thanks to Pierre Langlois python-yubikey should now finally be fixed
>> on master, without all the big python build changes. (Poetry is still
>> broken.) This was via <https://issues.guix.gnu.org/63354>. Unfortunately
>> I managed to clobber the author line in the git log after editing and
>> testing locally, sorry about that! (Anything that can be done to fix
>> that?)
>
> Thank you for picking up the series!  I admit I had forgotten I had sent
> it, I don't have too much time for Guix these days I'm afraid.
> Attribution is always nice of course, but mistakes happen, especially
> with git, so I wouln't worry about it in this instance! At least
> personnally I don't mind :-), just happy that the patches went in.
>

Thanks for the patches! I was so caught up in all the other python
stuff I ran into at the time that I never looked to pull out the fixes
for python-yubikey-manager (which I use and had been using from my
local branch).

I appreciate the understanding! We do have your copyright lines in
these additions. Given the overlap and complementary nature of both
patch series (I hadn't done your cleanup but did other things) I
should have just had us both as co-authors clearly from the start.

I'll give myself a slight pass since I think this was the first
non-trivial thing I pushed for someone else (working on a separate
branch locally, rebasing several times to make fixes, etc.). Hopefully
that means the next ones will be smoother and I can get rolling on
helping out with reviews and pushing our many great patches we have
waiting.

Thanks again for the work and understanding and Guix is always happy
to have your contributions!

John

Re: Core-updates merge

[John Kehayias]

Hi Felix,

On Fri, Jul 14, 2023 at 09:52 AM, Felix Lechner wrote:

> Hi John,
>
> On Fri, Jul 14, 2023 at 9:37 AM John Kehayias
>  wrote:
>>
>> Unfortunately
>> I managed to clobber the author line in the git log after editing and
>> testing locally, sorry about that! (Anything that can be done to fix
>> that?)
>
> Giving proper attribution can be essential in group projects. You
> could revert your commit and then re-commit the change with the
> correct author.
>

I agree, which is why I had a minor panic upon seeing the wrong name
in the git log on savannah after pushing (after having checked locally
so much to make sure I got it right, I must have glossed over it in
the final formation).

Revert/re-commit seems how we have to handle it, thanks for that note.

> How to handle it best depends on your dynamics with the injured party.
> I personally never cared enough—especially not in the case of an
> innocent mistake—but I know that some folks feel strongly about being
> made whole.
>

I made sure Pierre was aware in the original patch thread and here,
and see a response now...

> Kind regards
> Felix

Thanks Felix!
John

Re: Core-updates merge

[John Kehayias]

Bringing back up an old thread, but

On Fri, Apr 28, 2023 at 05:55 AM, John Kehayias wrote:

> Dear Andreas and fellow Guix-ers,
>
> On Tue, Apr 25, 2023 at 04:09 PM, Andreas Enge wrote:
>
[...]
>> Each and every package is not yet in shape; please feel free to submit
>> patches for your favourite packages that fail to build. In particular:
>> - python-yubikey-manager does not build currently; work to correct this
>>   is underway.
>
> I've just submitted <https://issues.guix.gnu.org/63139> which does a bit
> more than just fix that package and would be good for a Python feature
> branch. I'll send the cover letter to this list for wider visibility,
> though the Python team was cc'ed on the series. I suppose I should add
> myself to the team after this.
>

Thanks to Pierre Langlois python-yubikey should now finally be fixed
on master, without all the big python build changes. (Poetry is still
broken.) This was via <https://issues.guix.gnu.org/63354>. Unfortunately
I managed to clobber the author line in the git log after editing and
testing locally, sorry about that! (Anything that can be done to fix
that?)

Now to return to the big python series, minus a few patches.

John

Re: Guix's python has pip's user dir in its loadpath

[John Kehayias]

Hi,

On Sat, Jul 01, 2023 at 11:57 AM, Maxim Cournoyer wrote:

> Hi,
>
> Wojtek Kosior  writes:
>
>> The precedence of local, pip-installed Python libraries over Guix ones
>> has already been a source of bugs. And these can be hard to diagnose.
>
>> I imagine an optimal solution would be to configure this behavior on
>> per-package basis. The vast majority of applications does not need to
>> load local libraries. There are just a few exceptions like
>> `python-virtualenv`.
>>
>> Once I did write a package definition that deliberately disabled user
>> site dir package loading. I used code similar to what's below.
>>
>>> (modify-phases %standard-phases
>>>   (add-after 'wrap 'prevent-local-package-interference
>>> (lambda* (#:key outputs #:allow-other-keys)
>>>   (substitute* (string-append (assoc-ref outputs "out")
>>>   "/bin/")
>>> (("^#!/.*$" shabang)
>>>  (string-append shabang
>>> "export PYTHONNOUSERSITE=1\n"))
>
> That is indeed a simple thing we could do to harden Python binaries from
> picking up user pip-installed dependencies potentially causing
> problems.  I would welcome such a patch.
>

Perhaps, but if this is expected (and known) upstream behavior, I'm
wary of deviating from these expectations. This general area does seem
tricky and no simple best answer I guess.

>> Of course, it makes no sense to add such snippet to all definitions.
>> Instead, we could modify python-build-system to allow doing a similar
>> thing based on a flag passed in package's `(arguments)`.
>
> I think it need not be made configurable but just applied
> indiscriminately to the wrap phase used in the python-build-system.

And this is part of the same question then, we should try to be
consistent, yes. I don't see a clear right path, but I haven't thought
much about this area. I think it comes down to a current
issue/limitation/quirk of Python from upstream and packaging for our
distro puts us in between what comes from them and how to take care of
our users.

But we'll be rebuilding the Python world anyway, so now is a chance to
try out some changes like that, though maybe it is a bit much with
what we are trying already. See 

John

Re: Transformations Shell Syntax

[John Kehayias]

Hello,

On Sun, Jul 02, 2023 at 09:54 PM, Ludovic Courtès wrote:

> HI,
>
> John Kehayias  skribis:
>
>> As one who also would like a shorter syntax option, here's a quick
>> thought: what about a short version of what we have for when there
>> is only one package given or it can be applied to all packages/be a
>> positional argument? An example is perhaps best, so what if we could
>> write:
>>
>> guix shell  --with-latest
>>
>> or guix shell  -
>>
>> which is short for
>>
>> guix shell  --with-latest=
>>
>> or even
>>
>> guix shell--with-latest
>>
>> to apply to all packages.
>
> This should be possible.  We should check the option parsers and
> transformation procedures in (guix transformations).
>

Sounds good, seems like a nice little project for someone. I'll pass
for the time being as I catch up on my patches and trying to review
more. (And I want to add some basic multi-arch support to the FHS
container, too.)

>> Or something like
>>
>> guix shell   --with-git-url= 
>> 
>>
>> and so on.
>
> This I’m unsure; one might argue that it’s also ambiguous, I’d always
> wonder what ‘--with-git-url’ applies to.
>

I generally agree; I'm not a fan of too much positional-ness in
arguments as it can get confusing. Though we do have --development for
guix shell already, of course.

Re: guidelines for package names (namespaces?)

[John Kehayias]

Hi Andy,

On Mon, Jul 03, 2023 at 01:55 PM, Andy Tai wrote:

> Hi, in Guix there seems no guidelines for package  names/namespaces
> although there are conventions like Python packages prefixed with
> python-...  (good).  However, this does not cover cases like Gnome
> applications. For example, I submitted the original definition for
> (the GNOME terminal package) terminator, and later I tried to rename
> that gnome-terminator but that was rejected by reviewers.
>

This is a good question and one I wonder about when packaging
sometimes. The general guideline I've seen expressed in Python land at
least (not sure if this is in the manual, but this discussion can go
towards clarifying) is that generally "libraries" get the "python-"
prefix but end user "applications" don't. I believe this is true for
Golang as well, though exceptions abound.

The line is not necessarily clear. As an example, the "autokey"
package is written in Python and I would guess almost always used as
an application as is, but you can script with autokey itself as well.

> Or GNU Guix maintainers may want to consider some type of name space
> support in the package name space because I think name collisions will
> be more and more likely as more packages are submitted for inclusion
> in Guix.

I don't have any insight or knowledge on the internals here but the
general practice of a language prefix for libraries and the like has
so far worked well I think.

John

Re: Branch (and team?) for mesa updates

[John Kehayias]

Hello,

On Wed, Jul 05, 2023 at 08:08 PM, Andreas Enge wrote:

> Am Wed, Jul 05, 2023 at 09:47:06AM -0600 schrieb Katherine Cox-Buday:
>> I disagree with this because it seems like Mesa moves along at a pretty
>> brisk pace and I feel like we'd be constantly recreating the same branch:
>> 23.1.3, 2023-06-22 (14 days)
>> 23.1.2, 2023-06-08 ( 9 days)
>> 23.0.4, 2023-05-30 ( 5 days)
>> 23.1.1, 2023-05-25 (15 days)
>> 23.1.0, 2023-05-10
>
> I doubt we would be able to move at such a brisk pace and update mesa
> every other week! It is a package with more than 4000 dependents, so in
> any case it would be good to regroup with somewhat related changes.
>

Since the Cuirass job remains, I don't think it is really any extra
work or not to recreate the branch as needed, if that is preferred.
I'll defer to whatever people want, we can always change our mind if
anything comes up.

As for the pace, on the main 23.1.x releases we see about 2 weeks
currently. If that is a bit fast I think about once a month is a
reasonable pace, maybe depending if it is a small bugfix release or
what other changes we may need causing other rebuilds to be grouped on
the branch. We can slow down if major changes are introduced, but
seems Mesa is moving along without breaking changes or major changes
in building/support these days.

As you can follow along on  within a
couple of days x86_64 and i686 had good weather (though I can't see
specifically what may have broken yet) while as we'd expect things
like aarch64 are still churning away. The speed of this build job
might determine how fast we can go anyway, but waiting too long will
mean a new minor release to build :)

John

Re: Branch (and team?) for mesa updates

[John Kehayias]

On Thu, Jun 29, 2023 at 12:09 PM, John Kehayias wrote:

[snip]
>
> I'll open a branch merge request issue later today as per new
> procedure for QA. Though I believe that only builds 2 branches, which
> is occupied at the moment. Or can someone set a separate build job
> specifically for mesa-updates, especially if we think it is a good
> idea to have this going forward?
>

Merging request for "mesa-updates" branch created:
<https://issues.guix.gnu.org/64369>

>> Do we want a "Mesa team" or something a bit larger? Not sure what
>> exactly, since "graphics" is perhaps too broad. Happy to help
>> spearhead the Mesa front for Guix (the very package that got me first
>> involved in the patching process).
>>
>
> This is still a good question I think, of how we want to have a
> team(s) to handle things like xorg, wayland, mesa, and related
> packages. They are a bit all over the place in terms of scope and what
> they touch. For now I'd like to go ahead with a regular mesa-updates
> branch since that sees regular releases and is pretty self-contained
> currently.
>
> Thanks!
> John

Re: Branch (and team?) for mesa updates

[John Kehayias]

Hello again,

On Mon, Jun 19, 2023 at 02:10 PM, John Kehayias wrote:

> Hi everyone,
>
> With our move to a branching strategy for patches that require many
> rebuilds, I would like to propose a branch for Mesa updates. Based on
> how Mesa has been developed the last few years, there should be
> frequent (roughly a couple of months or quicker) releases that
> shouldn't be breaking anything or requiring lots of packaging work.
>
> Famous last words, I know, but at least in the last few years the only
> big change I know of was the one we hit on the last core-updates cycle
> with old hardware being dropped. The previous cycle had some build
> changes, perhaps due to missing many intermediate version changes.
> Both were rather self-contained and resolved quickly.
>
> So, I have submitted a patch for the latest stable release at
> <https://issues.guix.gnu.org/64175> I'm aware of one other patch that
> should also go here, <https://issues.guix.gnu.org/64044> are there any
> others?
>

I've created a "mesa-updates" branch with these two commits (the one
updating mesa has been updated to the latest version from last week).

> I've tentatively labeled with "mesa-updates" as a proposed branch
> name. With Mesa's release cycle I propose keeping this as a branch for
> Mesa updates, and I suppose related required changes (say libdrm). If
> everything goes smoothly, we can give the build farm some time to
> build everything, check for any breakages, and then push to master
> with substitutes available. Master can be merged into this branch just
> prior to a patches going to this branch with the expectation merging
> back to master will be soon after and changes are only affecting
> packages that won't be touched on master anyway. I think this should
> be relatively clean and straightforward, a good use of our new
> branching/building strategy.
>
> Thoughts? Can someone set up a build job for this branch and/or let me
> know how to do that? (I would also require access to Cuirass.)
>

I'll open a branch merge request issue later today as per new
procedure for QA. Though I believe that only builds 2 branches, which
is occupied at the moment. Or can someone set a separate build job
specifically for mesa-updates, especially if we think it is a good
idea to have this going forward?

> Do we want a "Mesa team" or something a bit larger? Not sure what
> exactly, since "graphics" is perhaps too broad. Happy to help
> spearhead the Mesa front for Guix (the very package that got me first
> involved in the patching process).
>

This is still a good question I think, of how we want to have a
team(s) to handle things like xorg, wayland, mesa, and related
packages. They are a bit all over the place in terms of scope and what
they touch. For now I'd like to go ahead with a regular mesa-updates
branch since that sees regular releases and is pretty self-contained
currently.

Thanks!
John

Re: modifying (rather than fully replacing) a build phases

[John Kehayias]

Hi Csepp,

On Thu, Jun 29, 2023 at 02:30 AM, Csepp wrote:

>
> I think I've used assoc-ref to find the build phase I wanted and then
> called it from something else.  It's not super pretty, but it works.

Yes, you can get phases from other build-systems, for instance, which
is needed sometimes. Here I meant actually modifying such a phase,
rather than just using/replacing/removing it. Perhaps that is not so
straightforward currently. (And like you said, getting phases from
elsewhere can feel a little clunky, luckily not needed too often from
what I've seen.)

John

Re: modifying (rather than fully replacing) a build phases

[John Kehayias]

My question remains, but I've updated the patch in question to use
#:make-flags (thanks to mirai for the idea), separating out these
parameters from the build command and making this a much easier
change.

Still, we have examples of just copying some phase by hand to make
adjustments, is there anything better we can/should do?

On Tue, Jun 27, 2023 at 12:29 PM, John Kehayias wrote:

> Hi Guixers,
>
> A question that is either relatively simple or else getting into the
> weeds a bit, that I came across in my proposed patch
> <https://issues.guix.gnu.org/64213> The general question is: how can I
> modify a build phase without replacing it completely? More
> specifically (as seen in the proposed patch) there is a build phase
> consisting of just an "apply invoke" call where I want to remove one
> of the arguments given. Is there a simple way to do this rather than
> just manually copying and deleting that string?
>
> I would assume so, but I wasn't sure how to do it and couldn't quite
> grep any examples. Some light exploring in the guix repl shows me that
> package-arguments is a keyword list (not sure the proper terminology
> in Guile) with #:phases a big gexp. Essentially I want to modify that
> to remove a string. I suppose it is a question of manipulating a gexp
> directly. (I'm sure macros can do all this but there doesn't seem to
> be a need for that at this level, right?)
>
> Thanks in advance!
> John

modifying (rather than fully replacing) a build phases

[John Kehayias]

Hi Guixers,

A question that is either relatively simple or else getting into the weeds a 
bit, that I came across in my proposed patch 
 The general question is: how can I modify a 
build phase without replacing it completely? More specifically (as seen in the 
proposed patch) there is a build phase consisting of just an "apply invoke" 
call where I want to remove one of the arguments given. Is there a simple way 
to do this rather than just manually copying and deleting that string?

I would assume so, but I wasn't sure how to do it and couldn't quite grep any 
examples. Some light exploring in the guix repl shows me that package-arguments 
is a keyword list (not sure the proper terminology in Guile) with #:phases a 
big gexp. Essentially I want to modify that to remove a string. I suppose it is 
a question of manipulating a gexp directly. (I'm sure macros can do all this 
but there doesn't seem to be a need for that at this level, right?)

Thanks in advance!
John

FreeType Brotli (WOFF font) support; Godot 4

[John Kehayias]

Hi Guix-devel-ers,

I have a local patch that updates Godot to (the big and exciting) version 4. 
Despite the big changes in Godot, besides some larger changes to its package 
definition, the only external change I came across was in FreeType. Guix's 
FreeType is built without Brotli support, needed for WOFF fonts. Is there any 
reason we don't build with Brotli?

The change needed is simply adding 'brotli' to the 'propagated-inputs' of the 
freetype package. The closure increases from 79.9 MiB to 82.3 MiB.

If this is a reasonable change, I can go ahead and make it, I would say on the 
mesa-updates branch I proposed. A change to freetype means over 8000 packages 
need to be rebuilt (17,800 dependents), according to guix refresh. There is 
significant overlap with what gets rebuilt for a Mesa update, and we could see 
about bumping up harfbuzz too, per the discussion in 
.

It is a lot of rebuilds but very minor changes in package definitions in just a 
few places, hopefully making it easier to address any issues (or backing out 
some changes for another time).

I can go ahead an make a branch with these few changes next week if there are 
no objections to at least see how this will go, pending a build job. Sorry for 
splitting this related discussion to a new thread, but I figure this would be 
good for separate input.

Thanks!
John

PS: In my quick tests Godot 4 works great; probably we want to keep the 
previous version 3 series as it is LTS now.

Re: Using (recursive #t) in (origin git-reference)

[John Kehayias]

On Tue, Jun 20, 2023 at 12:33 PM, Felix Lechner via \"Development of GNU Guix 
and the GNU System distribution.\" wrote:

> Hi unmatched-paren!
>
> On Tue, Jun 20, 2023 at 11:52 AM (  wrote:
>>
>> (If you've already tried building it without the RECURSIVE?, then this
>> issue will go unnoticed, as the derivation paths [...]
>> are *exactly the same*
>
> Okay, that nipped me in the hiney. Now it works!
>
> Thanks for explaining! You are an invaluable as well as instant
> resource for this list.
>

Yes, thanks for the quick response (!

This happens enough I feel like it should be explicitly mentioned as a 
potential "gotcha" in the manual, even if one can gleam it from understanding 
package definitions and hashes, if it isn't already. Or maybe a little "tips" 
section on packaging (or in the cookbook) with something like this and related 
ones like idiosyncrasies/conventions in certain language ecosystems. We all 
accumulate a lot of institutional knowledge as we do and pass it along, but 
better to have it properly referenced.

John

Re: Changes to the branching/commit policy

[John Kehayias]

Hi Chris,


On Sat, Jun 17, 2023 at 10:57 AM, Christopher Baines wrote:
>
> John Kehayias  writes:
>
>> Thanks for these changes! Question on branches (sorry if this was
>> covered in a previous thread, but now that we have new language in the
>> manual I figure this is a good place): do we have a convention on
>> branch names and subject headers for emailing patches for the branch?
>> e.g. does [PATCH  1/3] do anything on the QA end?
>
> On the names of branches, there's some typical names like -team, or
> wip- but really it's up to you.
>
> QA doesn't currently support applying patches to anything other than the
> master branch, but that's something that should probably be addressed at
> some point. I'd encourage you to do whatever you think is useful here,
> then hopefully QA can use that to act appropriately.
>

Sounds good, a simple identifier in the "[PATCH]" subject, like for
core-updates, seems simple enough to get started.

>> Or does the section about branch building for once patches are pushed
>> to a branch on Savannah?
>
> I'm not sure what you're asking here.

I'm confused myself over my wording, but I remember what I was trying
to get at:

Currently QA doesn't build patches if they cause a large number of
rebuilds correct? So for building a branch that requires this, will
that only happen on Cuirass with a new jobspec for the branch? Or will
this be addressed through changes to how QA builds? (Maybe this
answered below actually.)

>
>> Does that mean pushing to a branch should follow the same 1-2 week
>> review allowing QA builds? I guess patch series are always built
>> together on QA but wondering if there is anything else to be aware of
>> or needs mentioning to keep things tidy and clear.
>
> Those durations mentioned in the commit policy [1] are intended to allow
> for human review. For QA, it doesn't need to be time based as it can
> report it's progress. Obviously it does need a bit of time to check
> things, but I prefer to leave it up to people to assess the changes and
> any information provided by QA and decide when it's appropriate to push.
>
> 1: 
> <https://guix.gnu.org/manual/devel/en/html_node/Commit-Access.html#Commit-Policy>
>

A separate issue which I wanted to get at was about pushing changes to
any branch on Savannah. Do we expect those to be at the same state as
anything that would go directly to master, just pending the testing of
builds basically? So, after the usual review period? Or can those be
more WIP and not polished yet? I suppose this is for a team/people
working on a branch to discuss and how it will then be merged to
master.

> On keeping things clear, I think with branches I'm hoping the issue
> tracker can help with this, which is why there's now a strong
> requirement to create a guix-patches issue when you want to merge a
> branch, and use those issues to manage the timing.
>
> For those issues, there's currently a convention of using the following
> title: `Request for merging "" branch` e.g. [2]. That helps QA find
> these issues and act accordingly, but of course if someone comes up with
> a better way of naming these issues, we can just adjust the code in the
> qa-frontpage.
>

I see, that's a nice way to then group up discussion if a branch has a
bunch of separate patch threads. Looks like a good idea!

So, to be concrete, with the mesa patch I just sent,
<https://issues.guix.gnu.org/64175> I can open a merge request issue for
QA to process the branch, once the branch is actually created on
Savannah? I assume with the pretty trivial version change here I could
do that to see how the builds go, but I'll hold off pending the thread
I just started about this branch/team.

> 2: <https://issues.guix.gnu.org/63713>
> 3:
> <https://git.savannah.gnu.org/cgit/guix/qa-frontpage.git/tree/guix-qa-frontpage/branch.scm#n63>
>
> Thanks for your questions :)
>
> Chris
>

And thanks again for all you do here, the Guix tooling has been making
great progress!

John

Branch (and team?) for mesa updates

[John Kehayias]

Hi everyone,

With our move to a branching strategy for patches that require many rebuilds, I 
would like to propose a branch for Mesa updates. Based on how Mesa has been 
developed the last few years, there should be frequent (roughly a couple of 
months or quicker) releases that shouldn't be breaking anything or requiring 
lots of packaging work.

Famous last words, I know, but at least in the last few years the only big 
change I know of was the one we hit on the last core-updates cycle with old 
hardware being dropped. The previous cycle had some build changes, perhaps due 
to missing many intermediate version changes. Both were rather self-contained 
and resolved quickly.

So, I have submitted a patch for the latest stable release at 
 I'm aware of one other patch that should 
also go here,  are there any others?

I've tentatively labeled with "mesa-updates" as a proposed branch name. With 
Mesa's release cycle I propose keeping this as a branch for Mesa updates, and I 
suppose related required changes (say libdrm). If everything goes smoothly, we 
can give the build farm some time to build everything, check for any breakages, 
and then push to master with substitutes available. Master can be merged into 
this branch just prior to a patches going to this branch with the expectation 
merging back to master will be soon after and changes are only affecting 
packages that won't be touched on master anyway. I think this should be 
relatively clean and straightforward, a good use of our new branching/building 
strategy.

Thoughts? Can someone set up a build job for this branch and/or let me know how 
to do that? (I would also require access to Cuirass.)

Do we want a "Mesa team" or something a bit larger? Not sure what exactly, 
since "graphics" is perhaps too broad. Happy to help spearhead the Mesa front 
for Guix (the very package that got me first involved in the patching process).

Thanks!
John

Re: Changes to the branching/commit policy

[John Kehayias]

Hi Chris,

On Mon, Jun 12, 2023 at 09:23 PM, Christopher Baines wrote:
>
> Christopher Baines  writes:
>
>> The changes in #63459 have strayed now in to touching the commit policy
>> [1]. My intent was to simplify the guidance by grouping it better, but I
>> think the significant change here is that the commit policy now
>> references the entire branching strategy, rather than just talking about
>> sending patches for review.
>>
>> 1: 
>> 
>>
>> That new branching strategy makes some "should" requirements on sending
>> patches for review and pushing to topic branches for larger changes. It
>> also makes a "must" requirement on opening guix-patches issues to track
>> and manage merging branches.
>>
>> I'd like to merge these changes next week since they've been up for a
>> few weeks, so do comment if you have any thoughts or if you'd like more
>> time to review them.
>
> I've now merged these changes as
> 0ea096ae23fa81f05ce97e5e61c15647c0a475ec.
>
> You can now see the updated Commit Policy on the website [1] (you might
> need to force a refresh), as well as the new section on managing patches
> and branches [2].
>
> 1: 
> 
> 2: 
> 
>

Thanks for these changes! Question on branches (sorry if this was
covered in a previous thread, but now that we have new language in the
manual I figure this is a good place): do we have a convention on
branch names and subject headers for emailing patches for the branch?
e.g. does [PATCH  1/3] do anything on the QA end? Or does the
section about branch building for once patches are pushed to a branch
on Savannah? Does that mean pushing to a branch should follow the same
1-2 week review allowing QA builds? I guess patch series are always
built together on QA but wondering if there is anything else to be
aware of or needs mentioning to keep things tidy and clear.

Thanks!
John

Re: Transformations Shell Syntax

[John Kehayias]

Hello,

As one who also would like a shorter syntax option, here's a quick thought: 
what about a short version of what we have for when there is only one package 
given or it can be applied to all packages/be a positional argument? An example 
is perhaps best, so what if we could write:

guix shell  --with-latest

or guix shell  -

which is short for

guix shell  --with-latest=

or even

guix shell--with-latest

to apply to all packages.

Or something like

guix shell   --with-git-url= 


and so on.

A positional argument requires a bit more work and signaling/knowledge for the 
user, but maybe just the short hand --with-latest or -x (or whatever letter) 
which errors when more than one package is given is a step in this direction. 
Not sure if these two related suggestions can be combined or not without making 
things too complex.

Anyway, it would be nice to have a short version for, at least for me, the 
common situation of trying to build a single package with a transformation(s) 
for just that one. For instance, that's usually how I check if there's trivial 
version bump or if building from some fork works without modification of the 
package definition beyond the source.

(apologies for the top post and formatting, currently away from a proper 
computer)

John

 Original Message 
On May 26, 2023, 10:50 PM, Ludovic Courtès wrote:

> Hello! "jgart"  skribis: > Uses specified commit hash: > > guix build 
> emacs-ement@8b56efa9387262514daf63151d41c9e111e79567 > > Uses specified 
> commit hash (short): > > guix build emacs-ement@8b56efa > > Uses latest 
> upstream release: > > guix build emacs-ement@latest > > Uses upstream version 
> 0.8.2 if not packaged: > > guix build emacs-ement@0.8.2 > > Uses the latest 
> commit in the wip/find-room branch: > > guix build emacs-ement@wip/find-room 
> I sympathize with the will to get a more compact way to express 
> transformations. Right now, command-line tools parse package specs by calling 
> ‘specification->package+output’. There are no restrictions on version fields: 
> “8b56efa9387262514daf63151d41c9e111e79567” and “latest” are perfectly valid 
> version fields. Thus, if the syntax above was implemented, we’d introduce 
> ambiguity. Consequently, rather than overload “@”, I believe another syntax 
> would need to be found. Thanks, Ludo’.

Re: Order of manifest and overlapping binaries

[John Kehayias]

See also https://issues.guix.gnu.org/58859 and linked issues (some already 
mentioned and most authors on this thread are involved in some/all of these 
issue reports). I think this all points to the need for some fixes and/or 
clarity here for the user.

(apologies for the top post and formatting while I'm away from a proper 
computer)
 Original Message 
On May 17, 2023, 4:23 AM, Attila Lendvai wrote:

>> I could not find documentation on this circumstance or how to resolve. > 
>> Both 'parallel' and 'moreutils' produce a 'bin/parallel' and only one > can 
>> go in the $GUIX_PROFILE. i have proposed a solution for a special case of 
>> this problem: to do alphanumeric sorting, which together with semantic 
>> versioning picks the latest from the collisions of different versions of the 
>> same package. see: https://issues.guix.gnu.org/50878#12 the cleanup commits 
>> reached master by now, but the actual change of the semantics of the union 
>> has not. it needs to be revived/rebased. in its current form it wouldn't 
>> help in your case, but it points to the relevant part of the codebase. HTH, 
>> -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “In a country where 
>> the sole employer is the State, opposition means death by slow starvation. 
>> The old principle: who does not work does not eat, has been replaced by a 
>> new one: who does not obey shall not eat.” — Leon Trotsky

Re: Commit Access: jgart

[John Kehayias]

Welcome and congrats! I've always appreciated your enthusiasm and looking for 
ways to improve the Guix experience.

(apologies for formatting and top posting, email client limitations right now)

 Original Message 
On May 16, 2023, 9:28 AM, jgart wrote:

> Hi Guixers, Thanks for granting me commit access. I'm looking forward to 
> helping out with patch review, with a particular focus on moving patches 
> along for the Python team. I also intend to continue contributing more 
> packages, updates, and improvements. Thanks again for granting me the 
> privilege of commit access and I am looking forward to continuing our 
> collaborations. all best, jgart (Jorge Gomez)

Re: Reverting d477018b57 "gnu: poetry: Update to 1.1.12."

[John Kehayias]

Hi Tanguy,

 Original Message 
On May 24, 2023, 11:48 PM, Tanguy LE CARROUR wrote:
> Hi John,
> Quoting John Kehayias (2023-05-04 17:09:14)
> > I didn't emerge in time for the core-updates merge. There might be a better 
> > way
> > than causing a python world rebuild, but this is my current series
> > which does have Poetry building (might as well do the updates I
> > figure):
> […]
> > I'll give it a try at the week end!

> Days became weeks and… I had no chance (yet) to give it a try! 

> Any updates on your side? I've just seen that Lars answered on #63139 few 
> weeks ago.

No worries! I know all too well how time passes in unexpected ways.

I haven't had a chance to work on this as I'm away for a few weeks from a 
proper computer. I meant to say so on that issue thread but looks like I 
forgot. I'll try that though as the formatting of this message may make clear, 
I'm not using my preferred email client and it is more difficult.

But since days did turn to weeks, I'm nearing the end of my travels and will be 
back in a little over a week so hopefully more progress then.

John

Re: Welcoming Josselin Poiret as a new Guix committer

[John Kehayias]

Welcome Josselin! Another long overdue addition as I've long
appreciated your work and frequent discussions on all things Guix.

Here's to many patches, bug fixes, and pushes!

On Thu, May 11, 2023 at 11:27 AM, Maxim Cournoyer wrote:

> Hi Guix,
>
> I'd like to welcome Josselin among the Guix committers, that is, enabled
> to install the gems waiting to picked up from our guix-patches tracker
> :-).
>
> Welcome and congratulations, Josselin!

Re: Welcome to Simon as a new committer

[John Kehayias]

Yes, congrats and welcome! With all the work you do I had always
assumed you were already a committer and we are lucky to have you step
up into this role!

(On my end I'll be away from a proper computer for 3 weeks but looking
forward to a summer of hacking on Guix when I return.)


On Thu, May 11, 2023 at 08:50 AM, Katherine Cox-Buday wrote:

> On 5/11/23 8:44 AM, Maxim Cournoyer wrote:
>> Hello,
>>
>> I'd like to welcome Simon (aka zimoun) as a new committer.  They have
>> made many contributions along the years, both in code and in community
>> management.  I'm sure they'll make good use of their new
>> responsibility/privilege.
>>
>> Congrats, Simon!
>
> Congratulations, Simon!

Re: Mesa vulkan layer path fix for core-updates

[John Kehayias]

Hello,

On Tue, Apr 25, 2023 at 04:40 PM, Kaelyn wrote:

> Hi,
>
> --- Original Message ---
> On Tuesday, April 25th, 2023 at 2:15 PM, Andreas Enge  wrote:
>
>> Hello Kaelyn,
>>
>> thanks for your research!
>
> You're welcome! :)
>
>> Am Wed, Apr 19, 2023 at 04:07:51PM + schrieb Kaelyn:
>>
>> > *  can be closed when
>> > core-updates is merged, since core-updates contains mesa 22.2.4
>> > * Though not exactly mesa-related,
>> >  can possibly be closed now, and
>> > almost certainly once the core-updates merge is completed. (The
>> > ticket is a number of workarounds the user applied in early Feb to
>> > be able to build their system profile using core-updates, to pick
>> > up Mesa 22 for newer hardware support. I'm not sure if any of the
>> > patches are still relevant.)
>>
>> I just closed these two.
>>
>> > *  looks like an alternate way of
>> > solving the layer path issues that
>> >  also addresses. Additionally, it
>> > adds two new nonstandard VK_*_PATH variables to vulkan-loader,
>> > with at least VK_ILAYER_PATH seeming very similar in functionality
>> > to VK_LAYER_PATH and VK_ADD_LAYER_PATH described at
>> > 
>> > *  would be fixed by
>> > 
>
> I feel these two can be closed once 59453 lands, as then the manifests
> will have the store path to their corresponding shared objects. I also
> think having the full paths in the manifests will lead to fewer
> cross-version/cross-package mixing of objects, compared to setting and
> using environment variables of directories to search.
>

I haven't looked at the details, but I'm guessing these can all be
closed now?

>> > *  might need a modification to
>> > mesa e.g. to add VDPAU_DRIVER_PATH as a native-search-path (one
>> > possible solution; in my home profile I made VDPAU work by setting
>> > "VDPAU_DRIVER_PATH=/run/current-system/profile/lib/vdpau").
>> > *  appears to be the same
>> > VDPAU_DRIVER_PATH issue as .
>
> For the VDPAU drivers, I plan to do a little more digging and some
> experimenting but I suspect defining VDPAU_DRIVER_PATH as a
> native-search-path is the best way forward. I'll send a patch once
> I've tested a change locally without having my profile set
> VDPAU_DRIVER_PATH to /run/current-system/profile/lib/vdpau.
>

I checked that both 48868 and 62313 were fixed in the recent updates
and closed both.

Thanks for the patches!
John

Re: 04/09: gnu: mesa: Update to 23.0.3.

[John Kehayias]

Hi Christopher and Maxim,

On Mon, May 08, 2023 at 02:41 AM, Christopher Baines wrote:

> guix-comm...@gnu.org writes:
>
>> apteryx pushed a commit to branch master
>> in repository guix.
>>
>> commit 0be7838105806819f4586ec9130382a66a22880e
>> Author: Kaelyn Takata 
>> AuthorDate: Thu May 4 20:12:46 2023 +
>>
>> gnu: mesa: Update to 23.0.3.
>>
>> * gnu/packages/gl.scm (mesa): Update to 23.0.3.
>> [source]: Remove obsolete patch and update HTTPS url.
>> [arguments]: Enable the crocus gallium driver.
>> * gnu/packages/patches/mesa-fix-sporadic-test-failures.patch: Delete 
>> file.
>> * gnu/local.mk (dist_patch_DATA): Remove it.
>> ---
>>  gnu/local.mk   |  1 -
>>  gnu/packages/gl.scm| 14 ---
>>  .../patches/mesa-fix-sporadic-test-failures.patch  | 27 
>> --
>>  3 files changed, 5 insertions(+), 37 deletions(-)
>
> → guix refresh -l mesa
> Building the following 1954 packages would ensure 4257 dependent
> packages are rebuilt ...
>
>
> I know there's been some discussion about changing processes regarding
> changes like this that impact lots of packages, but as far as I'm aware,
> the documented process hasn't changed yet. So should this have gone to
> core-updates, and not been directly pushed to master?
>

I should take some responsibility over what happened here as I had
volunteered as a sort of "branch manager" (to use the terminology
later in this thread) but I was a bit slower than I wanted. My plan
was roughly in line with what we are discussing, reviewing the
patches, pushing to a new "mesa-updates" branch, and then asking for
someone to start a build job. We could then have people more easily
test or just to check build for build failures and ensure good
substitute coverage upon merging.

Anyway, I appreciate this coming up for discussion here, so thanks
Christopher. And thanks Maxim for pushing through these changes
anyway, as there were some important fixes for people (some couldn't
use wayland I think due to missing Mesa drivers, video decoding
acceleration didn't work, etc.) and selfishly I'm happy to have the
latest Mesa sooner.

My apologies for not helping lead more on this front after my public
volunteering to do so!

John

Re: Python feature branch

[John Kehayias]

Hi all,

On Mon, May 08, 2023 at 07:28 PM, Lars-Dominik Braun wrote:

> Hi Andreas,
>
>> I wanted to set up automatic building on cuirass for the Python updates
>> branch, but was not sure which one it is:
>> $ git branch -a | grep python
>>   remotes/origin/python-updates
>>   remotes/origin/wip-python-graphviz
>>   remotes/origin/wip-python-mne
>>   remotes/origin/wip-python-pep517
>
> I doubt there is one yet. Per #63139 I was going to set up
> “python-team”, which seems to be the naming consesus right now.
>

Right, I don't think we have a naming convention, but as per some
other threads we want to solidify this new process over the
staging/core-updates monolithic branches. We could add this as part of
the patch naming convention to ease finding patches and using QA for
testing/building. A straightforward "python-team" or "python-updates"
in general sounds good to me.

By the way, I'll comment in #63139, but I'll be away for several weeks
starting at the end of this week. I'll see what progress I can chip
away at before then and leave it in all these capable hands.

Thanks!
John

Re: gcc-toolchain is missing libstdc++.so

[John Kehayias]

Hi Kaelyn,

On Thu, May 04, 2023 at 11:45 PM, Kaelyn wrote:

> --- Original Message ---
> I wasn't sure the best place to share it, so I've attached my "run"
> script for running the binary download of PolyMC in a container. It is
> both a shell script and a guix package manifest, and is the one place
> so far I've worked around the removal of gcc:lib. The main
> program-specific bits are what CMD defaults to and which packages need
> to be included (most of the various shares are to get things like
> hardware 3D, pulseaudio, and dbus working and aren't all always
> needed). It also contains the original manifest commented-out for
> comparison. Hope it can be of help to folks!
>

Thanks, that's a nice little hack! Just something very minor I
noticed, but you don't need to specify glibc directly for the -F (FHS)
option in guix shell, as it will automatically include the (modified)
glibc.

This topic came up again on IRC today and GNUtoo had the correct cli
invocation for getting at gcc:lib. I thought I had tried something
similar but must have missed something, or else didn't notice that
this will only work for guix shell, as guix build doesn't take outputs
in the package list:

--8<---cut here---start->8---
guix shell  -e $'(list (@@ (gnu packages gcc) gcc) "lib")'
--8<---cut here---end--->8---

Thanks to both of you I have some options for workarounds currently,
but based on how this topic keeps coming up I still think we should
have a more straightforward option.

John

Re: gcc-toolchain is missing libstdc++.so

[John Kehayias]

Hi all,

> I have similar use cases of FHS containers to run binaries (primarily
> games). I recently ran into the issue of gcc:lib going away and no
> output from a visible package providing libstdc++. My current
> workaround was to implement a replacement for specifications->manifest
> that could handle packages and '(package "output") pairs in addition
> to strings, so that I could include `(,(@@ (gnu packages gcc) gcc)
> "lib") in place of "gcc:lib". Internally it resolves package strings
> to packages with specification->package, then passes the package and
> optional output specifier to package->manifest-entry. But I digress a
> little...

Nice little hack Kaelyn, would you mind sharing somewhere? I wonder if
this should be something we should have more easily anyway.

On Thu, May 04, 2023 at 12:14 PM, Katherine Cox-Buday wrote:

> On 5/4/23 11:33 AM, Kaelyn wrote:
>
>> Regarding solutions, I would prefer to have libstdc++ in it's own
>> package or output rather than bundled into gcc-toolchain:out; it
>> feels messy and against the grain of isolating programs in
>> containers if I have to make the gcc and g++ compilers available in
>> the container in order to run a program that needs libstdc++.
>
> +1. I recently ran into this as well and went looking for it.
>
> I think a good reason to give libstdc++ its own output is that this
> question continually gets asked.

That sounds reasonable to me as well. I would think the make-libstdc++
procedure would work for this, but as I detailed in my other message,
I'm not sure why it seems to be missing symbols. We would have just
what we need there and could just expose some public package versions
through that or leave it similar to how it is and document (so it is
more of an advanced or edge case scenario and not have more people
going that way when what they really want is the actual gcc-toolchain
package).

John

Re: gcc-toolchain is missing libstdc++.so

[John Kehayias]

Hi again,

On Thu, May 04, 2023 at 11:19 AM, John Kehayias wrote:

> Thanks for opening this and cc'ing; this has come up with some
> frequency on IRC, especially recently. In discussing there today, the
> current reasoning is that usually one will just call g++ which knows
> how to find libstdc++. So, gcc-toolchain does not include gcc:lib as
> part of what it makes available.
>

I tried locally just adding gcc:lib as an input for gcc-toolchain and
that does the trick. And since it is just a union-build, very quick to
try out :)

guix size reports an increase in gcc-toolchain as 0.1 MiB with gcc:lib
included.

> I think what we (and when this comes up, others) are getting at are
> some edge cases or different use cases where one wants to directly get
> at libstdc++. Previously it was more direct to use gcc:lib; of course
> one still can in code and/or cli with the proper call. For example,
> guix build -e "(@@ (gnu packages gcc) gcc)" will download/build/show
> the lib output of the (hidden) gcc package. Though I'm not sure how to
> select just the lib output here.
>
> My use case currently is in the FHS container where a binary wants to
> find some libraries directly. Previously one would include the gcc:lib
> package output in the guix shell call. Now some of those libraries can
> be found elsewhere, like libgccjit, but libstdc++ seems to be the
> trickier one. Open to other suggestions/workarounds, or thoughts on if
> it is worthwhile to include gcc:lib in the gcc-toolchain package (or
> make a gcc-toolchain:lib output?).
>

I tried with my local gcc-toolchain modification and this gets me what
I wanted.

On that note, I forgot to bring up the problem I had with using
make-libstdc++: it does not seem to build the same libstdc++ as
included in the gcc package. The doc string for that procedure notes
that this is meant to be used when using non-gcc toolchains, but we
also have the libstdc++ variable which seems to suggest that
(make-libstdc++ gcc) should be the same library as in gcc.

I'm not sure the difference in looking at the package definitions, but
I don't really know this stuff. Here's an example of the difference I
was finding:

I was running something and it complained that

--8<---cut here---start->8---
 symbol lookup error: : undefined symbol: 
_ZNSt18condition_variableD1Ev, version GLIBCXX_3.4.11
--8<---cut here---end--->8---

Indeed, looking at the libstdc++ I used via (or could have used
libstdc++ here directly since I used the default gcc):

--8<---cut here---start->8---
guix shell -e "(begin (use-modules (gnu packages gcc)) (make-libstdc++ gcc))"
--8<---cut here---end--->8---

I see

--8<---cut here---start->8---
$strings 
/gnu/store/6897bpw5858bdng744ddqw8rrqjb4frr-libstdc++-11.3.0/lib/libstdc++.so | 
grep "_ZNSt18condition_variableD1Ev"
--8<---cut here---end--->8---

while for gcc:lib it is defined

--8<---cut here---start->8---
$ strings 
/gnu/store/l684qgqlrqkbsh8jffp9d8ag6vrpcwgs-gcc-11.3.0-lib/lib/libstdc++.so | 
grep "_ZNSt18condition_variableD1Ev"
_ZNSt18condition_variableD1Ev
--8<---cut here---end--->8---

and using that libstdc++ does not result in that error.

Is make-libstdc++ not meant to be used/mixed with e.g. gcc-toolchain?
Is this expected that it is a different library produced or is this a
bug?

Thanks!
John

Re: gcc-toolchain is missing libstdc++.so

[John Kehayias]

Hi Christopher,

On Thu, May 04, 2023 at 11:05 AM, Christopher Rodriguez wrote:
>
> Sorry for the spam; Resending this without the bugs address, but with
> the issue's address.
>
> Christopher Rodriguez  writes:
>
>>
>> Hello All,
>>
>> I noticed today that libstdc++.so.1 (and some others), which used to be
>> part of gcc:lib, are not included in any of the outputs of the
>> superceding `gcc-toolchain` package.
>>
>> Is there another method for getting these needed shared libraries in a
>> guix system at this point? It's entirely possible I'm missing something.
>>
>> I am CCing guix-devel@gnu.org per podiki[m]'s request.
>>
>> Thanks!

Thanks for opening this and cc'ing; this has come up with some
frequency on IRC, especially recently. In discussing there today, the
current reasoning is that usually one will just call g++ which knows
how to find libstdc++. So, gcc-toolchain does not include gcc:lib as
part of what it makes available.

I think what we (and when this comes up, others) are getting at are
some edge cases or different use cases where one wants to directly get
at libstdc++. Previously it was more direct to use gcc:lib; of course
one still can in code and/or cli with the proper call. For example,
guix build -e "(@@ (gnu packages gcc) gcc)" will download/build/show
the lib output of the (hidden) gcc package. Though I'm not sure how to
select just the lib output here.

My use case currently is in the FHS container where a binary wants to
find some libraries directly. Previously one would include the gcc:lib
package output in the guix shell call. Now some of those libraries can
be found elsewhere, like libgccjit, but libstdc++ seems to be the
trickier one. Open to other suggestions/workarounds, or thoughts on if
it is worthwhile to include gcc:lib in the gcc-toolchain package (or
make a gcc-toolchain:lib output?).

Thanks all!
John

Re: Reverting d477018b57 "gnu: poetry: Update to 1.1.12."

[John Kehayias]

Hi Tanguy,

On Wed, May 03, 2023 at 09:49 AM, Tanguy LE CARROUR wrote:

> Hi Guix,
>
> I noticed yesterday that Poetry was broken:
> .
>
> I might have spotted it earlier if I had spend time testing `core-update`.
> My bad!
>

Yes, I noticed that too but fixing the current version of Poetry sent
me down quite a rabbit hole of dependencies and updates. I didn't
emerge in time for the core-updates merge. There might be a better way
than causing a python world rebuild, but this is my current series
which does have Poetry building (might as well do the updates I
figure): 

The proper polishing and bootstrapping updates is WIP, but that series
will get you Poetry building, after lots of other rebuilding :)

> The problematic commit seems to be d477018b57 "gnu: poetry: Update to 
> 1.1.12.".
>
> What also questions me is the fact that the commit message states that
> it's an upgrade to `1.1.12` when it's the current version and it's
> actually an upgrade to `1.4.2`.
>

Right, probably a typo in the commit message.

> Unfortunately, I have no time to work on this right now. Would it be
> possible to revert the change? Or should I submit a patch to downgrade it?
>
> Cheers,

I haven't tried if a simple revert will build given all the other
changes from core-updates. If that works that would be a good stopgap.
Do you know if that works and is simple enough? Or can you test?

Thanks,
John

Python feature branch

[John Kehayias]

Hello,

Below is the cover letter I sent for a series of patches to fix 
python-yubikey-manager which along the way updates/fixes more. (Apologies for 
the double send to Python team but I wanted the bug number first for this 
message and to include them in any larger discussion.) The series is here: 
<https://issues.guix.gnu.org/63139>

The deepest change is a package that affects pyproject-build-system. I thought 
I would send this here as well for wider visibility on this series as well as 
to garner any ready to go Python changes that can't be pushed to master 
directly.

So, please feel free to help gather any other patches that should go on a 
Python updates feature branch. I would like to keep this for changes that are 
less controversial/quicker changes in order to fix the breakages introduced 
from core-updates, namely poetry and python-yubikey-manager (an important leaf 
for some). However, since this series will rebuild all of 
pyproject-build-system using packages, there is the potential for lots of 
breakage. In that case, we can separate out the newer python-pypa-build package 
just for where it is needed an not update the build-system now.

Thanks everyone!
John

Cover for #63139:

Hi Guix and Python team,

Here is a patch series where the original goal was to fix 
python-yubikey-manager on core-updates and then ballooned to a bit more. This 
should be done in a feature branch for Python.

Mostly this was to fix/update the needed dependencies, though it may be 
possible to do this in a more minimal way just for that package fix. Anyway, I 
tried to generate this series in a way that each patch continues to fix things, 
but due to the complicated dependencies this may not be perfect.

A few notes:

1. Most of the series is pretty trivial, quick fixes/updates, some new packages.

2. What isn't is a few cases of failing tests which weren't immediately obvious 
to me and likely were some network access and/or build environment details. 
Some could be worked around maybe if someone wants to try (e.g. in 
python-virtualenv). I did enable more tests along the way though (like for 
poetry), so on the whole I think this is a step forward.

3. The dependents tend to be maybe 10s, a few in the hundreds, and then about 
3k for python-filelock. Until we get to pyproject-build-system updates:

4. I believe it was poetry that needed a newer python-pypa-build module, which 
then touches all pyproject-build-system (about 6k packages). This isn't 
strictly necessary as we could have a newer and separate package for leafs to 
use rather than in the build system as well, but I figured might as well do it 
sooner rather than later. At least the packages up to python-yubikey-manager 
built with this along with some random others.

5. On that note, I did not complete this change as I wanted some feedback on 
the bootstraping. I've added python-pyproject-hooks which should deprecate 
pep517, but currently it also needs python-pypa-build. I've made the older 
python-pypa-build a -bootstrap package to build this and the newer version of 
itself as well. So I did not deprecate pep517 yet.

   Also, python-wheel was a propagated-input in pep517 which is not needed in 
pyproject-hooks. However, I saw at least some packages will then need that as 
an input to build; so I kept it for pyproject-hooks to ease testing. It should 
be removed and added as an input as needed (no idea if that is just a few or a 
lot of the tree).

Okay, I think those are my notes. We should see what other things are ready to 
be made into this feature branch for Python. One brought to my attention 
recently is <https://issues.guix.gnu.org/63044> though I have not looked at it.

Thanks!
John

John Kehayias (20):
  gnu: Add python-installer.
  gnu: Add python-pyproject-hooks.
  gnu: Add python-rapidfuzz.
  gnu: python-crashtest: Update to 0.4.1.
  gnu: python-cleo: Update to 2.0.1.
  gnu: Add python-deepdiff.
  gnu: python-platformdirs: Update to 3.2.0.
  gnu: python-filelock: Update to 3.12.0.
  gnu: python-distlib: Update to 0.3.6.
  gnu: python-virtualenv: Update to 20.22.0.
  gnu: python-pkginfo: Update to 1.9.6.
  gnu: python-jsonschema: Update to 4.17.3.
  gnu: python-dulwich: Update to 0.21.3.
  gnu: Update python-pypa-build to 1.0.0.
  gnu: poetry: Fix build.
  gnu: Add python-poetry-plugin-export.
  gnu: python-pyscard: Update to 2.0.7.
  gnu: python-fido2: Update to 1.1.1 and enable tests.
  gnu: Add python-makefun.
  gnu: python-yubikey-manager: Update to 5.1.0 and enable tests.

gnu/packages/python-build.scm   |  80 +++-
gnu/packages/python-xyz.scm | 319 +---
gnu/packages/security-token.scm |  61 +++---
3 files changed, 370 insertions(+), 90 deletions(-)


base-commit: aecc6e70587f8412cbbb9b2c13141de4f534518e

Re: Core-updates merge

[John Kehayias]

Dear Andreas and fellow Guix-ers,

On Tue, Apr 25, 2023 at 04:09 PM, Andreas Enge wrote:

> Hello all,
>
> I have just merged core-updates into master and deleted the branch!
> This has been a long adventure, which became particularly intensive
> after the last Guix Days in February. First and foremost many thanks to
> everyone who contributed to the branch, be it by commits, discussions or
> by working on the infrastructure.
>

As others have said, a big thank you to you once again for leading the
charge! Much appreciated!

> Each and every package is not yet in shape; please feel free to submit
> patches for your favourite packages that fail to build. In particular:
> - python-yubikey-manager does not build currently; work to correct this
>   is underway.

I've just submitted <https://issues.guix.gnu.org/63139> which does a bit
more than just fix that package and would be good for a Python feature
branch. I'll send the cover letter to this list for wider visibility,
though the Python team was cc'ed on the series. I suppose I should add
myself to the team after this.

> - R on powerpc does not build; this will also be corrected soon;
> - aarch64 has very few substitutes; I think this is mainly due to the build
>   farm catching up and not so much to packages not building, but this is
>   difficult to know.
> If any of these are essential to you, you may wish to wait a little bit
> longer before a "guix pull", or for the time being pull to a commit just
> before the merge by issuing
>guix pull --commit=472706ae2f9160833951a4e4bcc4c206e03097b0
>
> Every end of a story is the beginning of many new ones. Several areas of
> work have already been identified; I will summarise what came up on the
> mailing list and who expressed interest to work on it - please feel free
> to join if a topic interests you, or launch another initiative if you
> want to work on a different subject!
> - rust-team already has a branch that is almost ready to be built and
>   merged, as a precursor to the team based workflow that we need to invent
>   (Efraim Flashner).
> - R on powerpc64le needs to be built by changes to valgrind and lz4
>   (Simon Tournier, I).
> - Many Python packages need updates, in particular with the aim of building
>   python-yubikey-manager (John Kehayias, Lars-Dominik Braun, Brian Cully).

I'll have to see what other patches we want to add to my series for
the branch and creating/building that as soon as possible. There is
some feedback needed on the deeper changes in my patch series, but
hopefully won't take much to finalize.

> - There is still work to do to bootstrap GHC until the latest version on
>   i686, and to potentially shorten the bootstrap chain (Lars-Dominik Braun).
> - OCaml could be simplified by dropping version 4.07 (Julien Lepiller).
> - After the mesa update is before the mesa update, and it looks like more
>   features can be enabled (Kaelyn Takata, John Kehayias).

When I get a chance next I'll be pulling the relevant patches locally
and building as a first check. And then we'll want this feature branch
to be created and built for wider testing.

> - Too much in Guix depends on too much else, which makes building things
>   needlessly entangled; in particular time zone data should not be referred
>   to by packages, but be loaded at runtime (Leo Famulari).
>
> All the best in your Guix endeavours,
>
> Andreas

Thanks everyone!
John

[PATCH] gnu: guitarix: Update to 0.44.1.

[John Kehayias]

e/gtk-3.0/gdk/gdkconfig.h:8,
>  from
> /gnu/store/3cl87br64f94fr6w45cpwzyn1kv9ma2p-gtk+-3.24.37/include/gtk-3.0/gdk/gdk.h:30,
>  from
> /gnu/store/3cl87br64f94fr6w45cpwzyn1kv9ma2p-gtk+-3.24.37/include/gtk-3.0/gtk/gtk.h:30,
>  from ../src/headers/guitarix.h:35,
>  from ../src/gx_head/gui/gx_main_midi.cpp:25:
> ../src/headers/gx_system.h: In function ‘bool
> gx_system::atomic_compare_and_exchange(volatile int*, int, int)’:
> /gnu/store/nb40pwd37v6i1g4b1fq4l6q4h9px3asr-glib-2.72.3/include/glib-2.0/glib/gatomic.h:163:44:
> error: invalid conversion from ‘volatile void*’ to ‘void*’
> [-fpermissive]
>   163 | __atomic_compare_exchange_n ((atomic), _oldval,
> (newval), FALSE, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST) ? TRUE : FALSE; \
>   |^~
>   ||
>   |volatile void*
> ../src/headers/gx_system.h:115:12: note: in expansion of macro
> ‘g_atomic_int_compare_and_exchange’
>   115 | return g_atomic_int_compare_and_exchange(p, oldv, newv);
>   |^
>
> In file included from
> /gnu/store/nb40pwd37v6i1g4b1fq4l6q4h9px3asr-glib-2.72.3/include/glib-2.0/glib/gthread.h:32,
>  from
> /gnu/store/nb40pwd37v6i1g4b1fq4l6q4h9px3asr-glib-2.72.3/include/glib-2.0/glib/gasyncqueue.h:32,
>  from
> /gnu/store/nb40pwd37v6i1g4b1fq4l6q4h9px3asr-glib-2.72.3/include/glib-2.0/glib.h:32,
>  from
> /gnu/store/3cl87br64f94fr6w45cpwzyn1kv9ma2p-gtk+-3.24.37/include/gtk-3.0/gdk/gdkconfig.h:8,
>  from
> /gnu/store/3cl87br64f94fr6w45cpwzyn1kv9ma2p-gtk+-3.24.37/include/gtk-3.0/gdk/gdk.h:30,
>  from
> /gnu/store/3cl87br64f94fr6w45cpwzyn1kv9ma2p-gtk+-3.24.37/include/gtk-3.0/gtk/gtk.h:30,
>  from ../src/headers/guitarix.h:35,
>  from ../src/headers/avahi_discover.h:26,
>  from ../src/gx_head/gui/avahi_discover.cpp:21:
> ../src/headers/gx_system.h: In function ‘bool
> gx_system::atomic_compare_and_exchange(volatile int*, int, int)’:
> /gnu/store/nb40pwd37v6i1g4b1fq4l6q4h9px3asr-glib-2.72.3/include/glib-2.0/glib/gatomic.h:163:44:
> error: invalid conversion from ‘volatile void*’ to ‘void*’
> [-fpermissive]
>   163 | __atomic_compare_exchange_n ((atomic), _oldval,
> (newval), FALSE, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST) ? TRUE : FALSE; \
>   |^~
>   ||
>   |volatile void*
> ../src/headers/gx_system.h:115:12: note: in expansion of macro
> ‘g_atomic_int_compare_and_exchange’
>   115 | return g_atomic_int_compare_and_exchange(p, oldv, newv);
>   |^
>
> Waf: Leaving directory
> `/tmp/guix-build-guitarix-0.43.1.drv-0/guitarix-0.43.1/build'
> Build failed
>  -> task in 'guitarix' failed with exit status 1 (run with -v to
> display more information)
>  -> task in 'guitarix' failed with exit status 1 (run with -v to
> display more information)
>  -> task in 'guitarix' failed with exit status 1 (run with -v to
> display more information)
>  -> task in 'guitarix' failed with exit status 1 (run with -v to
> display more information)
> error: in phase 'build': uncaught exception:
> %exception #< program: "python" arguments: ("waf"
> "build") exit-status: 1 term-signal: #f stop-signal: #f>
> phase `build' failed after 68.7 seconds
> command "python" "waf" "build" failed with status 1
From c652ee7b7339d287b623692190d66c3f5f3a90ee Mon Sep 17 00:00:00 2001
From: John Kehayias 
Date: Wed, 26 Apr 2023 18:25:35 -0400
Subject: [PATCH] gnu: guitarix: Update to 0.44.1.

* gnu/packages/audio.scm (guitarix): Update to 0.44.1.
[arguments]: Use gexps.
[native-inputs]: Remove labels
---
 gnu/packages/audio.scm | 26 +-
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/gnu/packages/audio.scm b/gnu/packages/audio.scm
index dca5e516a1..c369fe1bd4 100644
--- a/gnu/packages/audio.scm
+++ b/gnu/packages/audio.scm
@@ -2314,7 +2314,7 @@ (define-public freepats-gm
 (define-public guitarix
   (package
 (name "guitarix")
-(version "0.43.1")
+(version "0.44.1")
 (source (origin
  (method url-fetch)
  (uri (string-append
@@ -2322,14 +2322,14 @@ (define-public guitarix
version ".tar.xz"))
  (sha256
   (base32
-   "1bsjlfd7x09p3iiljilyfbns6hpqn9

Re: `mumi send-email' means no more debbugs dance to send multiple patches

[John Kehayias]

Hi Arun,

On Mon, Apr 24, 2023 at 04:12 PM, Arun Isaac wrote:

> Hi all,
>
> mumi, the software powering our debbugs frontend at
> , now also comes with a command-line
> interface. At the moment, the new command-line interface allows
> searching for issues, and sending patches. Most notably, it frees us
> from the debbugs dance required for sending multiple patches!
>

Thanks for this and the detailed instructions!

I just tried it out and it worked smoothly for sending 2 patches to an
existing bug. However, it was sent only directly to the bug address,
not being aware I guess of anyone that had submitted or since been
CC'ed to the thread.

Is there anything we can do here? I learned about the useful "wide
reply" when using debbugs in Emacs and presumably mumi can/does know
about all addresses on the bug. So hopefully this wishlist item won't
be too difficult.

Thanks again, will be happy to use this on the 20 patch series I'm
about to send...

John

Re: python-ledgerblue as input for electrum? (was: Core-updates, the last metres)

[John Kehayias]

Hi,

On Sun, Apr 23, 2023 at 04:32 PM, Vagrant Cascadian wrote:

> On 2023-04-23, Guillaume Le Vaillant wrote:
>> Here are a few leaf packages that don't build because of some
>> failing dependencies:
>>  - blender is blocked by opencolorio
>>  - electrum is blocked by python-ledgerblue
>
> Hrm this is kind of an aisde, but python-ledgerblue is a plugin for
> electrum to support specific hardware, which makes me wonder why it is
> in one of the inputs for electrum at all...
>
> There is at least one other, python-btchip-python; electron-cash has
> similar inputs, python-keepkey and python-trezor.
>
> My understanding is optional plugins in general should not be in inputs
> for packages; people should add the plugins to their profile or
> manifest, etc. for the hardware they plan to use...
>

Right, that's a good point. I had no idea about these packages before
trying to fix this one, but what you point out suggests a
refactoring/restructuring needed.

Alas, python-ecdsa (needed for electrum) is a transient failure I
think, as it built locally.

John

Re: Core-updates, the last metres

[John Kehayias]

Hi Guillaume,

On Sun, Apr 23, 2023 at 08:00 PM, Guillaume Le Vaillant wrote:

> John Kehayias  skribis:
>
>> If things continue looking good, are we planning to see the merge in
>> the next few days? Any other more leaf packages anyone has noticed
>> needs a fix someone should look at?
>
> Hi,
>
> Here are a few leaf packages that don't build because of some
> failing dependencies:
>  - blender is blocked by opencolorio

Not sure what is wrong here (error for a null pointer...) and it is
quite out of date. Looks like it'll need more than a quick fix to
update it, though admittedly I didn't try.

>  - electrum is blocked by python-ledgerblue

python-ledgerblue doesn't have tests. Looking at an old build log it
would run tests, but there was nothing. Something has changed so this
errors now. Anyway, disabled, it built, as did electrum locally.
Pushed as bacee2da69dc3fdc20c384bed479d7596d9234a9.

>  - freecad is blocked by python-pyside-2
>

Didn't look much here yet, looks like part of the Python QT bindings
ecosystem. Maybe someone familiar with this can see a quick fix.

> I'm not sure if I'll be able to look at them before the merge, so if
> someone has the time to try and fix them, please go for it.
>

I picked off the easy one, thanks in advance to whoever gets the
tougher ones done!

Re: Core-updates, the last metres

[John Kehayias]

Dear Andreas and Guix,

On Sun, Apr 23, 2023 at 09:30 AM, Andreas Enge wrote:

> Hello,
>
> yesterday I updated my system to core-updates. Since I am writing this
> message now, you can deduce that it succeeded. Well, there is no reason
> you should care, but it could encourage you to do the same. :)
>

As did I! (I'm on x86_64.) And boldly late at night and everything
went smoothly (caveat/tip below). I did build all my profiles and
system beforehand so my reconfigure was just some new minor builds and
activating.

Big thanks again to you (Andreas) for really pushing this all through.
I hope our move to more feature branches will make such a job less
onerous or maybe even extinct in the near future.

> I used commands like "./pre-inst-env guix package/system ...", but this
> resulted in strange behaviour; I suppose I may have ended up with a
> mixture between old and new guix.
> I think the safest approach is to use "guix pull --commit=..." with
> a commit from core-updates, or probably just
>guix pull --commit=core-updates
> Then I would start by updating the system, followed by the user profiles.
>

I agree here; I reconfigured my system (I switched my channels.scm
over to core-updates or similar for all my channels) but forgot to
update all my user profiles. I think this is what lead to not dropping
back to lightdm after I exited my WM, though the system booted fine. I
updated my profiles and rebooted (had to use some magic keys or I was
too impatient again) and all was good.

So, I also suggest updating everything all together before doing a
restart. I think last time I had some weird stuff as well, likely
because of the deep changes from moving to core-updates.

One noticeable difference was some fonts, perhaps a different
rendering or substitutes for some graphical programs? My terminal
emulator, kitty, also decided to pick some different default font, but
looked fine when I manually specified a font. Perhaps a difference in
some default font picking somewhere in an update? I also did a manual
fc-cache -fv for good measure.

> Please tell us if there is a show-stopper for the merge!
>

None I'm aware of, other than a few others with the same weirdness
when partially upgrading it seems.

> We already received a first report:
> python-yubikey-manager does not build. It should be repaired very shortly
> after the merge (the fix is there, but would cause too many rebuilds now);
> so if you rely on it, maybe delay updating your system, or hold this one
> package back in your profile ("guix package --do-not-upgrade ... -u").
>

I have a patch set almost ready for this. I need to do some minor
polishing and try to get some ordering here. It is one of those cases
where an update needed by one package causes others to need updates
and so on. It isn't too bad, about 20 mostly trivial patches. A few
need 10s to 100s of rebuilds, and at least one (python-filelock) will
need 3000 packages rebuilt. This will get us some overdue updates for
the affected packages.

(If you use a yubikey as a gpg smartcard everything should be fine, as
that's what I'm doing. I only use python-yubikey-manager for one time
codes or card setup. Just be aware before you update, or use another
computer/device for codes if need be.)

Likely we'll want to tack on any other core python packages in need of
updates, so please feel free to let me know of those or send patches.
I'll send a bug number here once I submit the series and we can make a
quick feature branch just after the core-updates merge. We do want to
keep this quick and not let it become another huge endevor, but I'm
happy to incorporate what we can.

> As for the architectures, x86_64 looks good; i686 and powerpc64le look
> okay (or at least not much worse than on master; R does not work on
> powerpc64le, and should also be fixed shortly after the merge).
> For aarch64 it is difficult to say, since the build farm has trouble
> keeping up. We brought back a few machines, but they are churning through
> the backlog.
>
> Andreas

If things continue looking good, are we planning to see the merge in
the next few days? Any other more leaf packages anyone has noticed
needs a fix someone should look at?

Thanks everyone for your great work here!

John

Re: Mesa vulkan layer path fix for core-updates

[John Kehayias]

Hi Kaelyn, Andreas, and Guix!

On Wed, Apr 19, 2023 at 04:07 PM, Kaelyn wrote:

> --- Original Message ---
> On Wednesday, April 19th, 2023 at 3:26 PM, Andreas Enge  
> wrote:
>
>>
>> Hello,
>>
>> thanks for bringing this back to our attention!
>
> You're welcome! :)

Yes, thanks Kaelyn, especially since I believe I said I wanted to help
these patches through for core-updates, before I had to step away for
a bit (all good now!).

>>
>> Am Wed, Apr 19, 2023 at 02:41:57PM + schrieb Kaelyn:
>>
>> Given how many packages depend on mesa and their importance, I think it
>> would be safer to postpone the fix until after the merge; be it in a
>> dedicated mesa feature branch that could quickly be merged afterwards,
>> or better yet regroup other changes in this area. (Searching for open mesa
>> packages on issues.guix.gnu.org returns quite a few matches, this could be
>> a good occasion to sort them out.)
>
> I'm okay with it waiting until after the core-updates merge and going
> into a feature branch, especially if that branch includes updating
> mesa to the latest release (currently 23.0.0, or 22.3.7 if going with
> the latest patch of the previous feature release instead of the x.y.0
> release of the newest series).
>

I was going to suggest the same, a feature branch just after
core-updates is merged, including getting Mesa to the latest version
(I suspect a stable v23 will be here soon). I've used a newer Mesa
locally for some packages and I didn't hit any issues in the packaging
for v23.

I would be happy to team up with you to prepare this branch once it is
time. I see you did a good search below collecting the various related
issues/patches. I can take a look as well.

> Inspired by your comment, I also just did a quick scan of
>  and took a
> peek at some tickets that looked like they may be related to the mesa
> package. From that perspective, six other tickets caught my eye in the
> results:
>
> *  can be closed when core-updates is
> merged, since core-updates contains mesa 22.2.4
>

Great, we'll be able to close it Real Soon Now.

> *  looks like an alternate way of
> solving the layer path issues that 
> also addresses. Additionally, it adds two new nonstandard VK_*_PATH
> variables to vulkan-loader, with at least VK_ILAYER_PATH seeming very
> similar in functionality to VK_LAYER_PATH and VK_ADD_LAYER_PATH
> described at
> 
>

Thanks, will have a look at the differences.

> *  would be fixed by 
> 
>

Great, always good to close some issues :)

> *  might need a modification to mesa
> e.g. to add VDPAU_DRIVER_PATH as a native-search-path (one possible
> solution; in my home profile I made VDPAU work by setting
> "VDPAU_DRIVER_PATH=/run/current-system/profile/lib/vdpau").
>

Right, this one is my report. I haven't had a chance to return to it
to figure out what is best here. Things do work with manual
intervention. It is a little tricky since my guess is we want VDPAU to
work without needing to install e.g. Mesa explicitly. But I don't
remember the details right now. Happy to discuss!

> *  appears to be the same
> VDPAU_DRIVER_PATH issue as .
>

Yes, I think so too, though VLC does have Mesa as an input so the fix
might be easier directly for VLC at least.

> * Though not exactly mesa-related, 
> can possibly be closed now, and almost certainly once the core-updates
> merge is completed. (The ticket is a number of workarounds the user
> applied in early Feb to be able to build their system profile using
> core-updates, to pick up Mesa 22 for newer hardware support. I'm not
> sure if any of the patches are still relevant.)
>

Also my quick reading of it; will double check none of those are still
out of date and we can close with core-updates.

> Cheers,
> Kaelyn

Thanks again Kaelyn! I think this will make for a small (but
non-trivial) set of patches for a feature branch and trying out that
workflow. We can then have the CI build the branch and people can try
their system with the newer Mesa plus fixes for testing. I don't think
there is any hardware support being dropped (unlike with v22) and it
will help for newer hardware, so I don't anticipate issues. But let me
not say that too loudly.

Looking forward to it!

John

Re: Core-updates after the staging merge

[John Kehayias]

On Tue, Apr 18, 2023 at 12:55 AM, John Kehayias wrote:

> Oh, looks like no x86_64 builds currently on Cuirass...hopefully
> temporary?
>

Never mind, builds going through now! Okay, I really must get to bed!

Re: Core-updates after the staging merge

[John Kehayias]

Hi Andreas and Guixers,

Again, thanks Andreas for all your work on core-updates!

On Mon, Apr 17, 2023 at 11:03 AM, Andreas Enge wrote:

> Hello,
>
> just a quick update after a night of building on CI.
> Things look generally quite good on x86_64; some things are being rebuilt
> due to the recent wget update, and that should hopefully also sort out
> i686 rather quickly.
>

Packages I use all seem to be good too, except wine as there is a
samba failure for i686. Pending fix: 

> Am Sun, Apr 16, 2023 at 01:09:02PM +0200 schrieb Andreas Enge:
>> - python: It is mostly green, but with lots of red, including at least
>>   one package that prevents icecat from building. I would call for careful
>>   updates and bug fixes to move on; leaf packages can be updated to the
>>   latest version, but for important intermediate packages please be
>>   careful.
>
> Python probably is the place where work is needed most, the strip in the
> dashboard looks like it has measles. Fixing one package there could bring
> us a long way (or not - we are in plain hand resolution of dependency
> problems there, one update could also hide another one). Sometimes it is
> just a matter of trying an update to see whether a package and its
> dependents build, so even someone who knows nothing about Python, but
> has a web browser pointed at , can do some useful work
> (I am speaking from experience).
>

I took a stab at a few random packages and ended up finding out that
python-urllib3 needed an update due to our updated
python-cryptography. Only saw this through test failures of leaf
packages rather than itself (noted in upstream update that failures
with newer cryptography tend to point people to the wrong packages).
While urllib3 has quite a few dependents (> 500 rebuilds on update)
and this may break more than it fixes, I pushed the latest version. A
few other updates/build fixes for the packages I was trying as well.

In the chain of oauth/google packages I was looking at, I was left at
python-google-auth-1 (older version) as failing tests even adding in
the missing python-mock. At quick glance I'm guessing due to the newer
python-cryptography. I haven't had a chance to see how we want to work
around that, or if the older versions of the few dependents should be
kept. It is only a few packages anyway, so maybe bigger fish to hunt
first.

I'll check in tomorrow on how the urllib update built on Cuirass and
see if I can pick off any other easy python-* fixes.

Oh, looks like no x86_64 builds currently on Cuirass...hopefully
temporary?

John

Re: A Joyous Core-Updates Week-End 

[John Kehayias]

Hi Guix,

On Wed, Apr 12, 2023 at 12:16 AM, Josselin Poiret wrote:

> Hello everyone,
>
> It's that time of the year again!  Merging core-updates!  Do you *want*
> glibc 2.35, gcc 11 as default, mesa 22, python 3.10, and more?!  Here's
> your chance!
>

Time flies! Big thanks to Andreas especially for really shepherding (no pun 
intended) us
to an already good state.

> What is core-updates you ask?  It's the big branch where all changes
> that affect significant parts of the dependency graph are pushed, to
> avoid world rebuilds, and also to test them out.  This branch is then
> merged back into the main one periodically, and of course, this is never
> painless, as upgrading important dependencies like glibc or gcc are
> going to cause issues.  Note that this workflow might change in the near
> future though, see [1].
>
> Some people have been hard at work fixing most of the build issues, and
> we have most of the big changes sorted out now.  However, it's
> impossible for the usual suspects to cover every single package, so we
> would like to ask the wider community for their help with the merge.
> Anyone can help, and in various ways (you don't need to know how to code
> in Guile!).
>
> This week-end (15–16 April), let's get everyone together and work on
> core-updates!  Here's what you can do to help:
>

Thanks for organizing! I know many are in Europe (US here) so I may miss a lot 
of people
due to time zone/schedule, but looking forward to helping out. I've been away 
from Guix a
bit due to some other stuff but hope to put my eyes for review and commit 
access to good
use to help out with everyone's hard work.

>
> 1) Use `guix time-machine` to test out your packages.
>
> If you have a package manifest lying around (you can get one out of your
> profile by using `guix package --export-manifest`), you can see which
> packages are available by doing `guix time-machine --branch=core-updates
> -- weather -m MANIFEST.SCM`!
>
> You can also test the packages themselves by doing `guix time-machine
> --branch=core-updates -- shell YOUR-PACKAGE`, at which point you'll be
> in a shell with the new package available.  You can then try it out to
> see if it works properly!
>

I also want to stress this point, more testing of your favorite package (or 
reconfiguring
your system to core-updates if you are feeling more adventurous) is really 
helpful. So
don't be afraid even if you have no idea what's happening under the hood.

>
> 2) Hack on `core-updates`.
>
> This is better to do beforehand: add a new worktree for core-updates
> (you don't want to prune all of the .go files when switching between
> master and core-updates, don't you?) using `git worktree add
> ../core-updates/ core-updates`.
>
> You can then enter that directory and follow the instructions from the
> manual, at "(guix)Building from Git".  You will have a checkout ready to
> work on core-updates!  You can then try to build your manifest from the
> checkout and see if everything builds, and try fixing the ones that
> don't.
>
>
> 3) Hang out on #guix and on the MLs.
>
> Follow what fellow members of the community are doing and struggling
> with, cheer them on and/or offer them your assistance!  Feel free to
> reply to this if you have any questions about the process, or with your
> attempts, struggles and successes!
>

Question on procedure: are we going to be posting every patch to guix-patches 
and waiting
for QA to build? Or only for not trivial (whatever that means) patches? I guess 
I'm asking
if this will be a sort of sprint weekend and larger changes/cleanup in the 
aftermath? This
depends on context of course, but in light of recent discussions on patch 
pushing, QA, and
teams (that I need to catch up on) I wanted to see what we were doing here.

>
> See you this week-end.
>
> [1] 
> 
>
> Best,

Thanks again, see you all on IRC and git logs this weekend!

John

Re: staging branch merged to master

[John Kehayias]

Hello Maxim,

On Fri, Apr 14, 2023 at 03:29 PM, Maxim Cournoyer wrote:

> Hello,
>
> The staging branch has been merged to master.
>

Nice, thanks!

> Should we remove the branch from Cuirass and Guix, knowing that teams
> is the way going forward?

I also agree with this change, yes to branches going forward.

John

Re: Emacs next variants

[John Kehayias]

Hi all,

On Fri, Mar 10, 2023 at 04:39 PM, Simon Tournier wrote:

> Hi,
>
> On Fri, 10 Mar 2023 at 16:04, Cayetano Santos  wrote:
>
>> Fine with me, even if to me emacs-next means latest from master,
>> including tree-sitter. You decide to use the feature, or not.
>
> Just to be sure we are on the same wavelength. :-)
>
> The package emacs-next is not currently following "master" and instead
> is following the branch Emacs 29 -- the next release of Emacs.  As far
> I know, this branch does not contain the feature Tree-sitter.
> Instead, the feature Tree-sitter is in the branch "master", which will
> be branched later as Emacs 30 and somehow will be the next next
> release of Emacs.
>

During this discussion some changes were made to this inheritance structure in



and



by Andrew Tropin. I've cc'ed him here to make sure he sees this discussion to 
make sure everyone is on the same page. I've seen notes before of pgtk being 
only for Wayland and potentially causing issues on X (this was some time ago, I 
can dig up the emacs threads if anyone wants). Personally I've never had issues 
in the past with pgtk on X, but haven't used it in quite some time. Just a note 
to keep in mind.

John

Re: Merging core-updates?

[John Kehayias]

Hi Katherine et al,

On Mon, Feb 13, 2023 at 09:40 AM, Katherine Cox-Buday wrote:

> Efraim Flashner  writes:
>
>> On Sun, Feb 12, 2023 at 10:05:40AM +0100, Julien Lepiller wrote:
>>> Hi Guix!
>>>
>>> As discussed at Guix Days before Fosdem, we haven't merged core-updates
>>> in a very long time. I'd volunteer to lead this effort, but I don't
>>> know what steps I should follow. Do we have some documentation about
>>> that?
>>
>> I think we normally have a 2 week last-chance window to get all sorts of
>> last minute packages bumped and then we freeze it and try to build
>> "everything".
>
>
> On that note, could we possibly give Mesa one final bump? The latest
> version is 22.3.5, and the version in core-updates is 22.2.4.
>
> I also hope to be able to test, but my schedule tends to run away from
> me.

Yes, I would like to get Mesa up to date, we have the smaller LLVM closure now 
and the pending patches for some Vulkan paths/packages. I would like to help 
review/test/push those but I'm in the middle of a move so if anyone wants to 
beat me to it in the next week or so, please do :) But yes, those are all on my 
radar and certainly a preferred activity to more packing...

John

Re: Merging core-updates?

[John Kehayias]

Hi guix-devel!

On Sun, Feb 12, 2023 at 03:49 PM, Josselin Poiret wrote:

> Hi everyone,
>
> Andreas Enge  writes:
>
>> I volunteer to follow your lead, but also have no clue what is actually
>> expected.
>
> I would also like to give a hand!
>

Count me in as well!

I only did some spot fixes the last round, but at least I have some familiarity 
with what happened. I don't have any particular expertise but I'm happy to help 
coordinate overall, test, and generally do some cat herding.

And now that I have commit access I hope I can really help move things through 
with review and pushing ready patches. Unfortunately had some other stuff come 
up for the last few months since I got access, but I should have more time 
starting in a week or so.

On that front, I think looking through relevant core-updates patches (the Mesa 
ones in particular, for me) is a good first step for review and I'll try 
building on my end to see how far I get.

>> [...]
>> Actually I am wondering whether the first step of killing these untamed
>> non-feature branches would not be to build and merge staging? It is based
>> on master, supposed to contain only medium sized changes, but which I
>> suspect end up being a world-rebuilding cluster of changes.
>
> You're right, for this to smoothly transition into the "feature branch
> workflow" (if that's actually what we want to do) I guess we also need
> to lay out a plan first, and prepare for the post-c.u-merge
> world. Getting rid of staging first could be an easier exercise,
> followed by c-u. I was planning on taking your notes of the Guix days
> and opening a discussion about how we could concretely transition to
> this new workflow, I can maybe do that this evening.
>

I need to catch up on the thread about feature branches and I'm looking forward 
to that as well. It has something that I have long wanted and will gladly help 
in that transition and doing what I can to make that go smoothly.

John

Re: Guix Games Collection

[John Kehayias]

On Wed, Feb 01, 2023 at 09:23 PM, Tobias Platen wrote:

> I had submitted a talk for LibrePlanet called "Gaming on a Talos II -
> how I avoid using Steam". Unfortunately, there were so many high
> quality talks that it was impossible to fit them all in the program.
> So I will do a lightning  talk [1], about my work in progress Guix
> Games Collection, a list of games that are playable on a freedom
> respecting machine such as the Talos II or Thinkpad X200 (with
> Libreboot) on the Guix System. I also plan to make a haunt page with
> screenshots/videos for each game, similar to the Steam storepages.

Cool, thanks for sharing and working on free(dom) games on Guix!

>
> [1]:
> 

Re: FOSDEM’s coming!

[John Kehayias]

Hi Ludo’,

On Thu, Jan 19, 2023 at 04:58 PM, Ludovic Courtès wrote:

> Hello Guix!
>
> I’ve prepared a blog post about FOSDEM and the Guix Days that we could
> publish tomorrow (Friday) or Monday:
>
>   
> <https://git.savannah.gnu.org/cgit/guix/guix-artwork.git/tree/website/drafts/meet-guix-at-fosdem-2023.md>
>

Thanks for doing the work of collecting all this, can't wait to see the talks!

Unfortunately I won't be able to be there in person and the time difference 
will make attending much of it live difficult, but I'll be around besides for 
my own short talk.

> Please take a look and send patches if needed!  If someone can come up
> with some kind of a logo for the Guix Days, that’d be great; otherwise
> we’ll just reuse the one from last year.
>
> Ludo’.

You may want to clarify what "video only" means. I take it that is for remotely 
presented talks, which are recorded beforehand, though the presenter should be 
available virtually during that time for live questions.

Speaking of, just a minor typo I noticed (for my own small part). Here is a 
quick diff for the change:

--8<---cut here---start->8---
index ae39095..bb53765 100644
--- a/website/drafts/meet-guix-at-fosdem-2023.md
+++ b/website/drafts/meet-guix-at-fosdem-2023.md
@@ -30,7 +30,8 @@ 
track](<https://fosdem.org/2023/schedule/track/declarative_and_minimalistic_compu>
- In [“_Using GNU Guix Containers with FHS (Filesystem Hierarchy
  Standard)
  Support_”](<https://fosdem.org/2023/schedule/event/guixfhs/>) (video
- only) will be present the recently-added [`guix shell --container
+ only) John Kehayias will present the recently-added [`guix shell
+ --container
  
--emulate-fhs`](<https://guix.gnu.org/en/blog/2023/the-filesystem-hierarchy-standard-comes-to-guix-containers/>).
  - [“_Declaring just what is

necessary_”](<https://fosdem.org/2023/schedule/event/minimalguixsystemimages/>)
--8<---cut here---end--->8---

See you all soon!
John

Re: Packages grow, no longer fit on a 

[John Kehayias]

Hi all,

On Tue, Jan 17, 2023 at 05:18 PM, Ludovic Courtès wrote:

> Hi,
>
> Efraim Flashner  skribis:
>
>> I've made some progress on LLVM and I think I have a working LLVM that
>> can be used as an input for mesa.
>>
>> (ins)efraim@3900XT ~$ du -sch 
>> /gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/*
>> 3.9M/gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/bin
>> 8.0K/gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/etc
>> 21M /gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/include
>> 67M /gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/lib
>> 16K /gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/share
>> 92M total
>>
>> (define llvm-for-mesa
>>   (package/inherit llvm-11
>
> Yay, great news!  Let’s have that in ‘core-updates’.
>

Yes, very nice!

A note that after some debugging that latest mesa (22.2.3 as of today, to be 
updated on
core-updates) seems to want newer LLVM, namely llvm-15. Fortunately we have 
this LLVM
version thanks to other's hard work on this front.

I don't think there were any errors in building mesa with older LLVM, but on 
current
hardware (unfortunately brings in non-free considerations) this was necessary. 
I believe
this is the summary: 

Props to katco on IRC for going through some long building and debugging to 
track this
down. The end result is that mesa 22.2.x with llvm-15 gave proper support for 
current gen
hardware, both parts (and current kernel) being needed.

So, I'd vote for having llvm-for-mesa to be at the latest LLVM version as well 
as Mesa. As
per the discussion on another thread, this could make sense for a feature 
branch and to
get thorough testing. Seeing as how LLVM seems pretty core to what Mesa does 
these days, I
would feel better having that tested across different hardware and use cases. 
Again, I
think a singular feature branch works well for this and I'm happy to help out 
on that
front. But I'll leave those discussions to the other thread.

>> In addition to what I have below I found that nix has a patch to make
>> llvm-11 (and the others) use the GNUInstallDirs, so we'd be able to move
>> the include directory to another output, saving another 21MB, bringing
>> llvm-for-mesa down to 71MB. Another possibility would be to have
>> llvm-for-mesa use llvm as an input, and then to use some configure-flags
>> to tell llvm-for-mesa to use the includes from llvm (the input).
>
> Good.  We can make these changes incrementally, but having
> ‘llvm-for-mesa’ would already be a noticeable improvement.
>
> Thanks!
>
> Ludo’.

Thanks all, I'm here for smaller closures!
John

Re: Guix driver paths for icecat RDD sandbox

[John Kehayias]

Hi Jelle and guix,

On Sun, Jan 15, 2023 at 04:37 PM, Jelle Licht wrote:

> Hi guix,
>
> I was playing around tyring to get hardware enabled video decoding
> working in icecat and/or firefox in guix, and found out that the fine
> folks working on Nix have already gotten a patch upstreamed that allows
> stuff in /nix/store to be loaded[0]. (Grep around for '/nix/store' in
> our icecat sources to see what I mean).

I think hardware accelerated decoding is more than just nice to have due to the 
massive
savings in CPU usage (and thus battery power for a laptop beyond general 
overhead). So
props for working on this!

As a disclaimer, last I tried icecat was not giving me hardware video decoding, 
but I did
fix it in firefox, at least in my testing. As that is not in Guix (for branding 
and/or
addon interface, is that correct?) I won't reference that in any detail, though 
I know you
(Jelle) are familiar with the details there. I'll explain a little below, for 
others
though. Hardware acceleration does let me watch the highest quality videos I 
could find
with minimal CPU usage.

>
> From what I can see, the RDD whitelist reads through symlinks, so the
> actual target file needs to be whitelisted before the file is loaded in
> the sandbox.
>

Yes, that is my understanding as well. And we should note that the RDD (Remote 
Data
Decoder) sandbox is a newer-ish feature that I'm at least aware of in the 
context of video
accelerated video decoding. This is different from other sandboxes used in 
icecat/firefox.

However, there is also an automatic allowance for anything in LD_LIBRARY_PATH. 
That was
the fix I used, drawing upon Mark Weaver's work in icecat to get everything in 
the runpath
for some drivers (e.g. from mesa). Adding that to LD_LIBRARY_PATH allows the 
RDD sandbox
access to them. And, as always, getting good debugging of library loading in 
sandboxes is
a bit tricky. I can probably dig up what I found and used if that is of use for 
anyone
(there's some Mozilla flags at least).

> Without this (or a similar fix), we'd need a custom package per possible
> value of LIBVA_DRIVERS_PATH, as loadable libraries for hardware
> accelaration do not seem directly configurable via
> 'browser/app/profile/icecat.js' at runtime. I may be wrong here, but
> this seems to also imply that a recompilation of icecat would be
> required as well every time one of these 'inputs' change :/.
>

Agreed, that is a potential downside, but what are the possible libva drivers 
in Guix?
There's mesa and possibly libvdpau-va-gl? Mesa is already needed anyway (and 
rarely
changes, though see discussion about more staging/feature branches). I also 
didn't see a
way to do this at runtime, based on how the RDD sandbox works.

There's also the question on foreign distros, which I don't know about.

> OTOH, it would have some drawbacks:
> - It hardcodes /gnu/store, instead of $MY_MAGIC_STORE_LOCATION
> - It allows loading of pretty much anything in the store by the
> sandboxed process.
>
> The second drawback seems pretty iffy, but the current suggested
> workaround is to disable the sandbox entirely.
>

I also share some concerns on the second solution but I'm not expert. On the 
one hand, we
expect the store to be world readable in general. On the other, sandboxes are 
there for a
reason and increasing a possible attack vector (say, on old version of 
something in the
store or similarly known exploit) is to be avoided. So I don't know.

> So that leaves us with 2 questions:
>
> 1. Do we want apply a patch to whitelist '/gnu/store'?
> 2. If so, would we want to also send this patch upstream firefox? They
> seem open to accepting it.
>

With my caveat that I didn't get this working on my end for icecat (I should 
retry), I
would be more for a limited workaround. But that might not be workable in 
general. I don't
use icecat much, and rarely for videos, so I haven't pursued it further. This 
might be a
good time to try again though.

If we go the second route I think it makes sense to upstream it in light of the 
similar
patch from Nix.

> I've opened an upstream issue for a similar treatment of /gnu/store,
> which may also simplify the 'build-sandbox-whitelist' phase of our
> icecat package[1] if accepted. I'm not entirely sure if that is
> ultimately a good thing yet though.
>
> Happy to hear any thoughts on this subject.
>
> - Jelle
>
> [0]: 
> [1]: 

All for better hardware video support, alas the ever-growing land of sandboxes 
and
containers (somehow I keep getting drawn in!).

John

Re: Golang go-updates feature branch?

[John Kehayias]

Hi Leo! and hi Guixers,

On Sun, Jan 08, 2023 at 02:22 PM, Leo Famulari wrote:

> Hello!
>
> Now that our build farm is running smoothly, I propose we revive the
> practice of feature branches, when appropriate.
>

Heartily agree here. This has come up a few times on #guix and generally with 
support (don't let me speak for everyone though). I think the idea of smaller 
and more frequent feature branches is a great idea: less code changes coming to 
master to review at once (compared to big core-updates merges), substitutes and 
closer to master to be easier for people to pull the branch and test, and some 
areas despite the number of builds would work nicely as a branch than just 
pushed (and lingering) into core-updates for much later.

> The first one that I propose is a go-updates branch, where we update the
> Go compilers. This will affect ~500 packages.
>
> If I understand correctly, Go is a relatively self-contained reverse
> dependency graph within Guix and thus a good candidate for this
> approach. It contains the Go libaries, and then some end-user
> applications, and they don't really affect other packages. So, it should
> be easy to understand when the update is ready to push.
>

I can't comment on the Go ecosystem, but that sounds good to me.

Another one I would personally like to see is Mesa: they move quickly, newer 
versions are needed for recent hardware, and despite the number of rebuilds it 
causes Mesa is careful with removing older support. This is also an area where 
getting people to test by running their desktop system on a newer version would 
be helpful, and I think the feature branch approach makes it much easier (i.e. 
in core-updates there is a ton to test and look for).

Anyway, perhaps that is a separate discussion and part of a larger discussion 
on number of rebuilds and how we classify changes.

> I can set up a jobset on ci.guix.gnu.org if people like the idea.
>
> Leo

So yes, I'm all for this general approach and think it is one that will make 
Guix better for the end-user and on the development side.

Thanks for bringing this up Leo, something I've also been wanting to move 
towards!

John

Re: Drafting a Guix blog post on the FHS container

[John Kehayias]

Hi Jim,

On Wed, Jan 04, 2023 at 06:07 PM, Jim Newsome wrote:

> Thanks, looks good, and the command in your patch also works for me.
>

Great, thanks for testing!

> I agree that passing and exposing XAUTHORITY seems better. Experimentally, 
> sharing the directory
> read-only also works (using `--expose` instead of `--share`) also works, but 
> I'm not familiar enough with
> this mechanism to be confident that'll work for everyone, or whether making 
> it read-only is worth the
> fuss.
>

Ah, you are right, that seems to be just fine for VSCodium and Tor, in my quick 
test. I think I'll change that.

> Btw it turns out that `libevent` and `openssl@1` can be dropped; they're 
> already bundled. All together,
> here's my current "best" version:
>
> ```
> guix shell --container --network --emulate-fhs \
> --preserve='^DISPLAY$' --preserve='^XAUTHORITY$' --expose=$XAUTHORITY \
> alsa-lib bash coreutils dbus-glib file gcc:lib grep gtk+ \
> libcxx pciutils sed \
> -- ./start-tor-browser.desktop -v
> ```

Nice, thanks for that too! I tried eliminating a few random inputs, but they 
were needed. It is difficult sometimes to get a really minimal set, but this 
looks good to me.

John

Re: Drafting a Guix blog post on the FHS container

[John Kehayias]

Hi Jim,

On Fri, Dec 16, 2022 at 05:39 PM, Jim Newsome wrote:

> Sorry for (presumably) breaking threading; I came across this online and
> don't see a way to set my in-reply-to-email header properly.
>
> Anyways just thought I'd mention that I recently learned about this
> feature, and was able to use it to get a downloaded [Tor Browser Bundle]
> running with:
>
>
> ```
> guix shell \
>--container \
>--network \
>--emulate-fhs \
>--preserve='^DISPLAY$'
>--share=/run/user/$(id -u)/gdm \
>openssl@1 \
>libevent \
>pciutils \
>dbus-glib \
>bash \
>libgccjit \
>libcxx \
>gtk+ \
>coreutils \
>grep \
>sed \
>file \
>alsa-lib \
>-- \
>./start-tor-browser.desktop -v
> ```
>
> `--preserve='^DISPLAY$'` and `--share=/run/user/$(id -u)/gdm` are to get
> access to the display. I'm not sure the second parameter is universally
> correct; I reverse-engineered it via roughly `ps aux | grep -- -auth`.
>
> The `-v` parameter to the browser script keeps it from trying to
> background itself, which otherwise causes the container and browser to
> terminate.
>
> It'd ultimately be nice to package the Tor Browser Bundle properly for
> guix, but it's nice to be able to use it this way in the meantime.

Thanks again for this! I slightly modified it for the blog post, which you can 
see in draft form at . I used 'gcc:lib' 
instead of 'libgccjit' as it is smaller, and changed the needed display options 
to be like the previous ones I had. Yours didn't work for me since it looks 
like it relies on sharing something from GDM, which I don't use. But do let me 
know if my version doesn't work for you.

Also gave you credit for this example; if you prefer not to be mentioned by 
name/link to the mailing list for any reason, just let me know.

Oh, and we do have some (older) patches for building the Tor Browser from 
source, but I don't know if they currently work: 
 Your example was great though, something 
very useful!

John

Re: Should Guix support writing CLI Common Lisp scripts? (Think Roswell)

[John Kehayias]

Hi Guixers/Lispers,

On Tue, Dec 27, 2022 at 06:14 PM, jgart wrote:

> Hi Guixers,
>
> Should Guix support writing CLI Common Lisp scripts? (Think Roswell)
>
>> Although Roswell is a unified interface to Common Lisp implementations, it 
>> also
>> encourages writing scripts with it.
>> A "Roswell script" is an implementation-independent script which can be 
>> invoked from a
>> shell command line, launched by > Roswell and run under standard CL 
>> environment.
>
> Just insert "Guix" wherever you see Roswell mentioned in the above quote.
>
>> * A roswell script can be distributed using quicklisp's infrastructure
>
> Just insert "Guix" wherever you see quicklisp is mentioned in the above quote.
>
>> If you're the author of the library, then consider adding the ros file to 
>> the repository
>> and automatically providing a roswell-installable command-line interface to 
>> it.
>
> Same above, insert Guix.
>
> I think we should make it easier for Lispers to write CLI scripts with Guix.
>
> WDYT
>
> 

I'm not sure what you mean if it is something beyond what we can do already 
with 'guix shell.' Do you mean using a particular hashbang as well? I haven't 
done that but my simplistic usage is quick and easy for me.

For example, I like to have some CL scripts I use for file processing that 
lives as a single .lisp file. To run it I just do:

guix shell sbcl sbcl-cl-csv unoconv -- sbcl --load myscript.lisp 
~/Downloads/*.xlsx

where I can include the compiler/interpreters sbcl, needed library, and an 
external tool that is called by the script for pre-processing. Works great, and 
of course instantly after the first caching. This could be combined with a 
manifest, version/channel pinning, making my script a package in a channel, 
guix.scm file, and so on, to make it more reproducible. But for me this is 
already super handy and easy, just one line.

John