RE: Proposal to turn off AOT in clojure-build-system

[Maxime Devos]

>On Thu, Feb 22 2024, Andreas Enge wrote:
>> Am Thu, Feb 22, 2024 at 03:57:41PM +0100 schrieb Maxime Devos:
>>> Yes. It appears you are unfamiliar with (...)
>>> It also appears you are unfamiliar with (...)
>>
>> May I suggest to not make assumptions about what other people are familiar
>> with or not? There is no point in claiming that others are less knowledge-
>> able than you; they may know as much or even more than you
>
>Asking questions can avoid any such conclusions. It's my favorite
>thing. Here, I might have asked: "Do you know about...?"

OK. I don’t think I’ll do that.

Best regards,
Maxime Devos.

RE: Proposal to turn off AOT in clojure-build-system

[Maxime Devos]

>Am Thu, Feb 22, 2024 at 03:57:41PM +0100 schrieb Maxime Devos:
>> Yes. It appears you are unfamiliar with (...)
>> It also appears you are unfamiliar with (...)

>May I suggest to not make assumptions about what other people are familiar
with or not? There is no point in claiming that others are less knowledge-
able than you; they may know as much or even more than you, and still come
to different conclusions. (And even if people were unfamiliar with
something, I would object to this haughty tone and suggest a more pleasant
way of making suggestions.)

This is hypocritical, you are (somewhat implicitly) making assumptions on my 
(un)familiarity with good manners and (somewhat implicitly) claiming that I am 
less knowledges than you on good manners. Furthermore, you aren’t actually 
suggesting a more pleasant to formulate the message of the part you quoted.

Also, I did not simply _assume_ that Steve was unfamiliar with transformation, 
I _concluded_ that Steve was likely unfamiliar by what they didn’t mention in 
their e-mails. Furthermore, the mere “likelihood” is included in the paragraph 
you quoted as part of the word “appears” – I did not claim that Steve is 
unfamiliar, I only claimed that it appeared to be the case, which is not the 
same thing. 

After all, perhaps Steve does know that such transformation exist but 
personally concluded them to not be a proper solution for some reason. In that 
case, now people know there is a disagreement on the role of transformations 
w.r.t. AOT and perhaps the source of the disagreement can be resolved, 
furthering knowledge and bring us closer to deciding on what the proper AOT 
default would be.

You mention that other people might know more, but there is also a flipside to 
this – sometimes people know _less_. In this case, perhaps Steve simply did not 
know about transformations and the proposed use of transformations in 
combination with enabling AOT by default would be agreeable to Steve and as 
such perhaps an answer to the decision whether to enable AOT by default.

As such, simply _assuming_ that Steve knew of transformations would be wrong, 
so I had to mention the option of transformations.

I did not simply claim that Steve is less knowledgable than me, at most it 
could be said that I (implicitly) claimed that Steve is less knowledged than me 
on the relation between transformations and Clojure AOT problems, but even 
then, I included a qualifier “It appears that”, not “It is the case that”.

The beginning “It appears you are unfamiliar with [...]” is simply a perfectly 
cromulent beginning of a sentence, not some assertion of superiority.

>For instance concerning the topic at hand, knowing that users may transform
packages as they wish to me seems to be independent of which default choice
we should make for the distribution.

It is not independent, see my previous e-mail where I explained how the 
existence of transformations turns some problems mentioned w.r.t. enabling AOT 
into non-problems.

If you have a disagreement with that explanation, please actually say what the 
disagreement is instead of only saying that you disagree. The former might 
bring us closer to some collective decision on what the proper default 
behaviour is for Guix, the latter doesn’t, is almost useless and makes it look 
like you didn’t bother to read the ‘(...)’ part.

While I suppose it is technically possible you have read the part ‘(...)’, 
given your response I simply don’t believe you did and hence I consider it fair 
to conclude that you are not familiar with the contents of the (...) part.

Best regards,
Maxime Devos

RE: Proposal to turn off AOT in clojure-build-system

[Maxime Devos]

 
that library has dependencies. Furthermore, likely many of these dependencies 
are dependencies of other applications as well.

Instead of redoing the compilation of N shared dependencies for each M 
applications  (total work: ~N*M), by not participating in these Clojure 
conventions, the total work of compiling dependencies can be reduced to ~N).

Let’s avoid joining the madness that is Go’s build system and Cargo(*).

(*) Rust is fine for compilation, it’s cargo that is the problem (at least, 
Cargo as used by Guix) – antioxidant 
(https://notabug.org/maximed/cargoless-rust-experiments) avoids the sheer 
inefficiency of Cargo.

>> >I believe this means that with AOT code on, any user who installs a 
>> >different version of Clojure from the one that we used to AOT the libraries 
>> >*may* have problems.
>> 
>> Unlike, say, Maven, this situation simply does not happen in Guix, because 
>> we don’t just download binaries and call it a day (except for some 
>> bootstrapping stuff, but that’s not relevant for Clojure AOT), because we 
>> have functioning recompilation of dependents, because of shebang patching, 
>> because binaries that are to be invoked should not rely on the ambient 
>> CLASSPATH / LD_LIBRARY_PATH / etc. and, if, the underlying binaries do rely 
>> on that, they are wrapped (see wrap-program) to set them (or, at least, they 
>> should be, you might find some bugs in this department if you go looking).
>> 
>> Even if they aren’t wrapped, then in that case the dependencies are 
>> propagated-inputs, and you can only have a single version of a propagated 
>> package in the same environment (barring stacking environment shenanigans, 
>> but then you are looking for it and/or you can just report a bug about how 
>> the binaries should be wrapped/classpath should be patched in/...).
> 

>In this paragraph you're assumption is that a Guix user is only using 
>libraries from within Guix. Hopefully, I've shown why this assumption is 
>unlikely above.

You haven’t. You have shown that other situations exist, not that this 
situation is unlikely. To demonstrate likelihood / unlikelyhood, you need some 
statistics (or a very thorough argument on why people wouldn’t just make a 
package for missing dependencies – it’s very simple, especially if you aren’t 
sending them for inclusion.)

>You also mentioned Debian, and @e.hashiman [2] said that Clojure libraries are 
>not AOT'd on Debian, while applications are. From what I can find there are 
>130 packages in Debian with the word Clojure in them [3]. I looked at a 
>selection and it seems true that Debian does not AOT libraries (and I can't 
>find any Clojure 'apps'). For completeness I also checked what Clojars, the 
>main distribution archive for Clojure developers, does:
> [...]

OK ... and? Other people doing things differently doesn’t mean those different 
approaches are better. If Debian mentions somewhere the reason _why_ it doesn’t 
AOT libraries, that reason can be investigated and perhaps carried over to 
Guix, but it’s the ‘why’ that’s important, not the ‘it doesn’t’.

> Does this help clarify why I'm asking to change the default?

Yes. It appears you are unfamiliar with the “transformations” functionality of 
Guix, which allows for easy rewriting of packages to whatever Clojure version 
you want, eliminating compatibility concerns or even turning off AOT completely.

(This requires the relevant versions of Clojure to be packaged in Guix, but 
that shouldn’t be too much of a problem.)

It also appears you are unfamiliar with Guix importers, which easily make 
package definitions (somewhat low quality w.r.t. descriptions, with lack of 
verification of freeness, etc., but usually otherwise fully functional). I 
don’t know if an importer for Clojure exists but it shouldn’t be too difficult 
I think.

Best regards,
Maxime Devos.

RE: Proposal to turn off AOT in clojure-build-system

[Maxime Devos]

((As I said about a month ago, in theory I should have access to a proper 
e-mail program again a week or two in the past.))

>Hi,

>Guix's clojure-build-system turns on AOT compilation by default. I would like 
>to advocate that 'as a distributor' we should *not* ship Clojure code AOT'd, 
>so we should change the default.

>This has been discussed previously. In #56604 r0man noted that AOT compilation 
>should not be on by default [0], Reilly makes the same point in #53765 [1].

>Maxime makes the point that where a compiler is available it should be used 
>[2] and that if it doesn't work it's a bug:

>>  "if a Clojure library misbehaves when AOT-compiled, without additional 
>> context, that seems like a bug in the Clojure library to me (or the 
>> AOT-compilation code).

>The perspective in the Clojure community is quite different from Guix's on a 
>number of fronts. There's not much discussion about offline builds

This (not the has nothing to do with AOT.

> reproducibility

If some libraries have irreproducible binaries, then just don’t AOT those 
libraries (and, if the irreproducibility is in macros (and maybe inlinable 
generated code, if that’s something the AOT does), the relevant dependents of 
those libraries as well).

An AOT problem for a few libraries is not a reason to turn off AOT by default, 
it is a reason to turn off AOT for those few libraries in particular.

> or code coming from Distributions. 

You are contradicting this by the next paragraph, in which you mention the 
distribution “Clojars”. A rather specialised and lacking distribution, but a 
distribution nonetheless.

>The internalised perspective is that you use the build tools to download 
>libraries directly from Clojars (a Maven repo)

That seems quite similar to Guix, where you use the build tools “guix build” / 
“guix install” / ... to download libraries (by default, when available) 
directly from ci.guix.gnu.org (I forgot the terminology, but you could compare 
it to Clojars I guess).

> and developers create a final uberjar for production usage

Except for the delusion of grandeur (AFAIK, in non-Java English, über only 
appears in the word Übermensch) (maybe whoever coined the term didn’t really 
mean it but eergh), this is also familiar, see “guix system vm” for large-scale 
things, and the command for creating relocatable packs (I forgot the exact name 
and command) on a smaller scale.

> Consequently, there is no specific statement saying 'Distributors should not 
> AOT libraries' that I can point to.

In this bit about differences in perspective, I haven’t seen any mention of 
AOT, hence the “Consequently” does not follow. The part that’s missing here is 
that (IIUC) in Clojure, it is somewhat conventional to stuff the compiled 
.class files in a superior Aryan JAR instead – the inferior UnderJARs you get 
from the “guix install clj-whatever” equivalent would only contain non-compiled 
.clj (and data files, whatever).

> But, I would like to draw attention to this thread on Clojureverse as the 
> best source I could find:
>Alex Miller is the main community manager for Clojure, and is a maintainer of 
>the core libraries, so his perspective is key. He notes that, AOT code is tied 
>to *specific versions of Clojure*:
>
>  "AOT'ed code is that it is inherently the product of a particular version of 
> tthe Clojure compiler ... I would recommend NOT AOT compiling libraries" [4]

This reasoning does not follow – yes, it is tied to the Clojure version, so 
what? Guix automatically rebuilds dependents when the dependency (in this case, 
the Clojure compiler) changes.

I guess the maintainer has something like Maven etc. in mind, where many 
maintainers, each with different distribution, Java version, etc. upload 
binaries, but that’s not the case in Guix. (To a lesser extent, this is not the 
case in, say, Debian as well.)

>In the same thread thheller, who is the maintainer of the most popular 
>ClojureScript tooling,  notes you cannot mix AOT and non-AOT libraries [5]:

"you cannot just ship your library AOT compiles as it would also contain 
clojure.core. Clojure AOT current ... can not load clj files from .class files. 
So AOT produces the class files and will fail if one of the dependent classes 
is missing although the .clj file is present"

This argument makes no sense at all. Of course it can’t load CLJ files from 
.class files – Clojure AOT is a compiler, not a decompiler, cf. how GCC can’t 
‘load’ C files from .o / ELF / ..., and why would it in the first place?

And of course you can’t run or compile an application if a part of the binaries 
/ a part of the source code is missing. This is the same as with C & GCC, 
(Guile) Scheme & “guild compile”, etc..  Yet, GCC and Guile somehow have a 
functioning AOT (if we count Guile’s bytecode as AOT, which we can because we 
are counting .class files as “compiled”).

(As a side-remark, I think a typo was made here:  dependent classes -> 
dependency classes (i.e. 

Re: what is the status of the core-updates now

[Maxime Devos]

Op 23-11-2023 om 13:02 schreef Z572:


Maxime Devos  writes:


[[PGP Signed Part:Undecided]]


Op 21-11-2023 om 18:21 schreef Maxime Devos:

  > [PATCH] gnu: ephemeralpg: Fix cross-compilation.
There is already a patch for that:
<https://issues.guix.gnu.org/49327#8>
and (a rebased version of) it effectively has already been applied:
<https://issues.guix.gnu.org/49327#150>

ok, close it.

I guess you could copy it from c-u to master if you don't want to
wait for the c-u merge, but please use the search function at
<https://issues.guix.gnu.org/>:


what is the status of the core-updates now,it on ci.guix.gnu.org only build
'core', is this a configuration error?


I don't know.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: rust-build-system from antioxidant

[Maxime Devos]

Op 12-06-2023 om 15:05 schreef Maxim Cournoyer:

Hi Maxime,

Maxime Devos  writes:


Op 12-06-2023 om 03:17 schreef Maxim Cournoyer:
[...]
Yes.  Overruling is a form of blocking, and blocking by authority
(whether de facto or de jure) is overruling.


There should not be a notion of 'overruling' in our
contribution processes (unless the Guix co-maintainers have to step in
as a last resort) if all participants strive to build consensus, as
mentioned in info '(guix) Commit Access':
 It is expected from all contributors, and even more so from
committers,
 to help build consensus and make decisions based on consensus.  To learn
 what consensus decision making means and understand its finer details,
 you are encouraged to read
 <https://www.seedsforchange.org.uk/consensus>.
I thought I knew what consensus meant myself, but the above link
helped
me to re-frame a few things in a way that is more conducive to building
consensus.




[...]


For example, the thread of the patch you sent at
<https://issues.guix.gnu.org/51427> is a good example of this, pretty
much everyone (except Ludo') agreed that the provided patch is good.


Let's avoid directly criticising ourselves and try to discuss what I
think has more value, which is coming to a better understanding of the
situation and how the perceived deadlock could be undone.
Consensus is not a majority vote; all parties have to walk the extra mile

>to reach a

common ground.


This quote is taken out of context -- nowhere I stated or assumed that 
it was a majority vote -- (n - tiny)/n is higher than 50%, and I also 
demonstrated that some other criteria were met.


Please don't ignore the explanation I wrote below all that, let's avoid 
taking quotes out-of-context, and let's avoid ignoring the concerns I 
pointed out in my previous e-mail.


> [...] try to discuss what I think has more value,

which is coming to a better understanding of the situation, [...]


I did this in my previous e-mail.  You can bring new discussion from 
another POV if you want to, but why are you ignoring my discussion on 
this matter?


> [...] all parties have to walk the extra mile to reach a common
> ground. [...]

This sentence would have more weight if you explained somewhere how this 
wasn't the case in <https://issues.guix.gnu.org/51427>.



I think the object there was from a semantic point of
view: we'd have a 'garbage collection' command (guix gc) which wouldn't
collect any garbage!  It's a valid objection, although its importance in

> the narrow use case presented was not agreed by all parties.

Ludo' claimed that the resulting patch wouldn't collect garbage:


I believe the effect is that ‘guix gc -D /gnu/store/…-disk-image’ would
remove nothing: /gnu/store/.links would still contain a copy of that big
disk image, so as a result, you’ve freed zero bytes.


On its own, this is a valid objection, but it is false -- it does 
collect some garbage -- it deletes the /gnu/store/ITEM.  This is 
implicitly referred to in:


> https://issues.guix.gnu.org/51427#4

[...]  Obviously there needs to be a way of __removing
single items from the store__, [...]


> zimoun wrote on 9 Nov 2021 19:10

[...] Even if the phase is drastically speed up, it would be probably still
too slow when using the option ’-D’ __remove only one __; or just
some. [...]


Yet, Ludo' seems to have missed this:

> [Ludovic Courtès wrote on 17 Nov 2021 11:02]
> [...]> No; like I wrote, it would have the effect of not deleting 
anything:

> [...

So I have written it down explicitly:

> https://issues.guix.gnu.org/51427#25
Also, it _does_ collect garbage -- it collects the /gnu/store/... item, 
it just doesn't collect _all_ the garbage (it doesn't collect the 
individual files in the store item or the things in /gnu/store/.links).


There wasn't any response.


A consensus-based outcome could be to add a new option to guix gc,
e.g. '--invalidate', which would be documented as "invalidate
(de-register from the Guix database) rather actually delete from the
store".  If that's still argued semantically unclear we could go with a
dedicated 'guix invalidate', although that seems overkill to me.


"guix gc -D" is already semantically clear -- it deletes a single 
/gnu/store/... item.  IIUC, the patch at 
<https://issues.guix.gnu.org/51427#1> fixes the bug where it also 
deletes more than that.


(IMO both deleting and not deleting the relevant links in 
/gnu/store/.links would be acceptable behaviors, as long as it doesn't 
do more than that.  Also IMO deleting more links is technically a bug, 
but harmless __as long as it's efficient.)


(IIUC, it also deregisters the store item, but that's needed for 
consistency of DB->file system, and deregistering can be considered a 
removal of DB entries, so not really a problem.)



This is a bit more work than the 1 line change initially suggested, but
I think we can agree tha

Re: rust-build-system from antioxidant

[Maxime Devos]

Op 12-06-2023 om 03:17 schreef Maxim Cournoyer:

Hi Maxime,

Maxime Devos  writes:


Op 02-06-2023 om 20:02 schreef Nicolas Graves:

A few months ago, Maxime Devos worked on a new rust-build-system to
handle a few issues we were experiencing with cargo (see discussions on
antioxidant in guix-devel).
A month ago, we discussed about the possibility of the integration
in
core guix, and the required steps. Maxime and I had a different
approach. Maxime highlighted the possibility to make a smooth transition
but once that would require many gradual changes and deprecation. My
approach was that since we'll have to eventually migrate all packages to
rust-build-system, and since we can freeze all former rust packages in
an archive channel, I would be clearer to make the transition at once.
[...]


Actually, I started out with the non-gradual approach, but that was
overruled by Ludo', IIRC.


Did you perhaps meant to say that it was disagreed with, or at worst
"blocked by"?


Yes.  Overruling is a form of blocking, and blocking by authority 
(whether de facto or de jure) is overruling.



There should not be a notion of 'overruling' in our
contribution processes (unless the Guix co-maintainers have to step in
as a last resort) if all participants strive to build consensus, as
mentioned in info '(guix) Commit Access':

It is expected from all contributors, and even more so from committers,
to help build consensus and make decisions based on consensus.  To learn
what consensus decision making means and understand its finer details,
you are encouraged to read
<https://www.seedsforchange.org.uk/consensus>.

I thought I knew what consensus meant myself, but the above link helped
me to re-frame a few things in a way that is more conducive to building
consensus.


In my experience, Ludovic often doesn't actually practice that, though.
For example, the thread of the patch you sent at 
<https://issues.guix.gnu.org/51427> is a good example of this, pretty 
much everyone (except Ludo') agreed that the provided patch is good.


People pointed out how Ludo' multiple time missed the point of the 
patch.  Yet, Ludo' kept missing the point, e.g. consider:


> [Liliana Marie prikler wrote on 18 jul 2022 19:03]

Am Montag, dem 18.07.2022 um 15:57 +0200 schrieb Ludovic Courtès:
Toggle quote (7 lines)

Hello,

With commit 472a0e82a52a3d5d841e1dfad6b13e26082a5750 (Nov. 2021),
partially fixing <https://issues.guix.gnu.org/24937>, there is
hopefully less pressure to skip the remove-unused-links phase.

Should we close this issue?

As a hard disk user, I'm leaning towards "no".  In fact, I recently
encountered a case where I think I might want to skip it even if not
deleting "specific items".  [...


> [Ludovic]

I agree that we should strive to have good performance on that kind of
hardware too, but I don’t know how to get there.


That's what the patch is for:

> [Liliana]
> [...]

I agree that we should strive to have good performance on that kind
of hardware too, but I don’t know how to get there.

I don't think deleting links will ever be fast on that disk.  But what
I've been saying the whole time is that I don't always need the links
deleted.  I think adding "expert" switches to skip these phases might
actually be enough – after all, if I ever do want to run a full GC, the
information ought to be the same, no?


[A remainder by me that Ludovic is missing the point]
[...] The point isn't to work-around slow "deleting unused links" 
implementation, but rather to avoid inherit slowness of deleting 
everything when deleting a few things suffice [...]

> [...]

Apologies for being elliptic.  My point here, as has been discussed
earlier in this thread, is that we can’t just skip that phase or we’d
simply leave files around without actually deleting them.


As repeatedly explained in different ways previously, we can, in fact, 
just do this and these consequences are unproblematic.  This is again 
explained in different ways in a response by Liliana and me.


Given that a few of the participants that wanted the patch in some form, 
are actually committers yet didn't commit it even though there was 
consensus, and that de facto Ludo' has authority and disagreed with the 
patch, the only conclusion I can draw from this, is that Ludo' 
effectively overruled the consensus with their (de facto) authority.


Whether this was intentional or not, the effect was the same.

(Technically ‘https://www.seedsforchange.org.uk/consensus#block’ allows 
for an unilateral block, but it later writes ‘However it provides a 
safety net for situations where a proposal would seriously hurt the 
group of people in it’, which doesn't apply at all here.)


This is not the antioxidant stuff, but it should serve as an explanation 
on why I do not believe that Ludovic practices consensus.  There are 
also a few other examples/threads that 

Re: rust-build-system from antioxidant

[Maxime Devos]

Op 02-06-2023 om 20:02 schreef Nicolas Graves:


A few months ago, Maxime Devos worked on a new rust-build-system to
handle a few issues we were experiencing with cargo (see discussions on
antioxidant in guix-devel).

A month ago, we discussed about the possibility of the integration in
core guix, and the required steps. Maxime and I had a different
approach. Maxime highlighted the possibility to make a smooth transition
but once that would require many gradual changes and deprecation. My
approach was that since we'll have to eventually migrate all packages to
rust-build-system, and since we can freeze all former rust packages in
an archive channel, I would be clearer to make the transition at once.

> [...]

Actually, I started out with the non-gradual approach, but that was 
overruled by Ludo', IIRC.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [GSoC 23] distributed substitutes, cost of storage

[Maxime Devos]

Op 04-04-2023 om 12:53 schreef Attila Lendvai:

Onderwerp:
Re: [GSoC 23] distributed substitutes, cost of storage
Van:
Attila Lendvai 
Datum:
04-04-2023 12:53

Aan:
Maxime Devos 
CC:
Vijaya Anand , pukkamustard 
, guix-devel@gnu.org




it's another question whether this mirroring should be enabled by default in 
the clients. probably it shouldn't,


It probably should -- if things aren't mirrored, then it's not p2p; you
would lose the main performance benefit of p2p systems.

More cynically, some p2p systems (e.g. GNUnet) have mechanisms to
disincentive freeloaders -- clients that aren't being peers will get
worse downloading speed.


any successful p2p solution must have an incentive system that makes attacks 
expensive (freeloading, DoS'ing, censorship, etc). arguably, the most important 
difference between the various solutions is what this incentive system looks 
like.

from a bird's eye view perspective, there are two fundamental architectures of 
p2p storage networks (that i know of):

  1) ipfs-like, or torrent-like, where the nodes register/publish what
 they have in their local store, and other nodes may request it
 from them

  2) swarm-like, where the nodes are responsible for storing whatever
 content "is" in their "neighborhood". (block hashes and node ids
 are in the same domain, so there's a distance metric between a
 block and a node). put another way: Swarm stores not only the
 metadata in the DHT, but also the data itself.

in 1) there's no need to pay for, and to upload content into the network. a 
node just registers as a source for whatever content it has locally, and then 
serves the incoming requests.

but if you have content that you want to make available in 2) then you need to 
make sure that this content gets to a set of distant nodes that will store it. 
this is very different from 1) from a game theoretic perspective, and can't be 
done without some form of payments/accounting.

in 1) it's simpler for a node to share: just give away your storage and 
bandwidth to the network.

in 2) it's more complicated, because if your node is requesting other nodes to 
do stuff, then you're spending a more complex set of resources than just your 
bandwidth, potentially including some crypto coin payments if the balance goes 
way off.


GNUnet is (1) but also more than that, because of the automatic pushing 
to other nodes.  To my understanding it's not (2), but at the same time 
your comment about (2) applies.


Also, this crypto coin balance problem can be avoided by simply not 
basing your P2P system on money (crypto coins or otherwise); it's a 
problem that those systems invented for theirselves.



but both cases are fundamentally the same: users are spending their resources, 
and i wouldn't expect that installing a linux distro will start spending my 
network bandwidth, or any other resource than my machine's local resources.


Network bandwidth (and storage) _is_ a local resource.

Also, how are you going to keep your distribution up to date or install 
new software without allowing your distribution to spend network 
bandwidth? -- For non-P2P systems, it is already the case that that 
network bandwidth is spent by the local machine, P2P systems just makes 
it more symmetrical and hence fairer.


More to the point, recalling that this is a reply to my statement that 
mirroring should be enabled by default:


>> it's another question whether this mirroring should be enabled by 
default in the clients. probably it shouldn't,

>
> It probably should -- if things aren't mirrored, then it's not p2p; you
> would lose the main performance benefit of p2p systems.
>
> More cynically, some p2p systems (e.g. GNUnet) have mechanisms to
> disincentive freeloaders -- clients that aren't being peers will get
> worse downloading speed.

... and noticing that you are making a distinction between the resources 
of the user and others:


‘users are spending _their_ sources, and i wouldn't expect that [...] 
will start spending _my_  network bandwith, [...], _my_ machine [...]’

(emphasis added)

... it appears that your view is that it's ok to spend resources of 
other people even without trying to reciprocate (*), and that it is 
unreasonable to expect reciprocation by default?


(*) I'm not claiming that not reciprocating is always bad -- it's a 
reasonable thing to not do when on a very limited plan.  Rather, the 
point is that reciprocating by default is reasonable and that in 
reasonable circumstances, not reciprocating is unreasonable.


I mean, given how you are a proponent of crypto, you appear to be a 
capitalist, so I'd think you are familiar with the idea that to use 
resources of other people, you need to compensate them (in money like 
with Swarm or in kind like with P2P systems (*)).


(*) I don't consider Swarm to be a P2P system -- Swarm _by design and 
intentionally_ actively maintains a class distinctio

Re: [GSoC 23] distributed substitutes, cost of storage

[Maxime Devos]

Op 25-03-2023 om 20:00 schreef Attila Lendvai:

welcome on board Anand!



In case a user requests for a substitute and there is a missing
block in the decoding process, a HTTP request for block would sent
to the substitute server and the server will encode the
corresponding block in real time and push it back into the
network. The block will be searched again and retrieved. > something to consider here: whose responsibility should it be that a 
block, that is missing from a p2p network, is (re-)uploaded there? the 
clients? or the current substitute server?


my gut instinct says that it's better if the clients do the (re-)upload of the 
blocks.

in this architecture the substitute server is just another storage mechanism 
along the other storage backends (although with a different reliability 
characteristics), and it's the clients that are doing the 
mirroring/spreading/distribution of the blocks among the various backends. the 
clients of course will/should keep the current substitute servers at the bottom 
of their list of backends in their configuration.

this way the load is distributed, and we don't need to add (too much) extra 
complexity to the substitute server codebase, and the actors are less tightly 
coupled.

it's another question whether this mirroring should be enabled by default in 
the clients. probably it shouldn't,


It probably should -- if things aren't mirrored, then it's not p2p; you 
would lose the main performance benefit of p2p systems.


More cynically, some p2p systems (e.g. GNUnet) have mechanisms to 
disincentive freeloaders -- clients that aren't being peers will get 
worse downloading speed.



and the project infrastructure should be running clients where it is turned on. 
altruistic third parties could also enable this mirroring feature, and donate 
their bandwidth/resources.

there's an issue with this, though:

some p2p storage backends will require some form of payment/credentials to use 
their resources. arguably, all p2p storage networks that will survive into the 
future will have some mechanism to limit the infinite abuse of their resources. 
it is to be researched how these payment mechanisms work on the various p2p 
networks, and whether it is possible that the Guix project pays for the storage 
globally, and then the random clients will have the necessary credentials to 
(re-)upload the missing blocks.


GNUnet has a built-in mechanism for mirroring and for avoiding overuse 
of resources.  From what I recall of the documentation and the GNUnet 
papers:


* The more a peer A requests stuff of peer B,
  the more peer B dislikes peer A.
* Likewise, the more peer A fulfills requests of peer B,
  the more peer B likes peer A.
* Requests by liked peers are prioritized.

(If you squint, I suppose this could be considered a form of payment, 
but no literal currencies are involved, so no need for any financing.)


Mirroring:

* When putting a resource on the network, a few copies
  are stored in the network.  (I assume this discreases the dislike of
  the peer that received the copy by the peer that sent the copy, and
  increases the dislike by the peer that sent the copy by the peer that
  receives the copy.)

* The more popular a resource is, the more replicas are stored in
  the network (I don't recall the mechanism, but IIRC this is an
  automatic process).

* Peers set a quotum on how much bytes they are willing to store;
  when exceeded, they throw out old stuff and low-priority stuff.

(The only way to be 100% sure a resource remains on the network, is to 
have a local copy in the local peer, so you can't really reliably ‘save’ 
something on the network, but you can use it as a CDN to spread the load 
and tolerate occasional downtime.)


Greetings,
Maxime


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: intrinsic vs extrinsic identifier: toward more robustness?

[Maxime Devos]

Op 05-03-2023 om 21:21 schreef Simon Tournier:

Whatever the intrinsic identifier we consider – even ones based on very
weak cryptographic hash function as MD5, or based on non-crytographic
hash function as Pearson hashing, etc. – the integrity check is
currently done by SHA256.


How about using the hash of the integrity check as an intrinsic
identifier, like is done currently?  I mean, we hash it anyway with
sha256 for the integrity check anyway, might as reuse it.


Maybe ask GNUnet folk to address by NAR+SHA256 instead on their
specification. ;-)


Obviously, Guix should replace NAR+SHA256 by GNUnet FS URIs /j.


Kidding aside, your comment rises two points of view:

  1. Guix is fetching data from elsewhere and this elsewhere is not using
 NAR+SHAR256 intrinsic identifier.  Therefore, the question is how to
 adapt the source origin for taking into account this elsewhere?

  2. Replace the NAR+SHA256 integrity checksum by what content-addressed
 systems use as intrinsic identifier.  IMHO, that’s a bad idea for
 two reasons: (a) security, for instance SHA1 as used by SWH is not
 secure and (b) it will be unmanageable in practise.


I was thinking of (1), not (2).

All that’s said, Guix uses extrinsic identifiers for almost all origins,
if not all.  Even for ’git-fetch’ method.


For git-fetch, the value of the 'commit' field is intrinsic (except when
it's a tag instead).


No, it is imprecise.  The exception is *not* label tag as value for the
’commit’ field but the exception is Git commit hash as value.


Are you referring to the fact that currently, the 'commit' field usually 
contains a tag name, and that it containing a commit is the exception?

If so, that doesn't contradict my claim.


This can be solved by placing the actual commit in the 'commit' field of
git-reference, instead of the tag name, then things are completely
unambiguous -- this and its opposite were discussed in ‘On raw strings
in  commit field’ (*), IIRC.


The thread you are referencing [1] is based on misunderstandings.  I
would like to move forward, hence my detailed email. :-)

1: 



Your email is about intrinsic identifiers and more robustness, yet it 
doesn't mention using git commits more anywhere.  As such, I do not 
follow ‘hence my detailed email’ -- it contains detail, but it misses 
some relevant detail that I pointed out in my previous response.


Also, with ‘move forward’, do you mean ‘move forward’, or ‘maintain 
status quo’?  Because given that you are replying to the proposed 
solution (that even avoids problems pointed out in those threads) by 
saying nothing of technical importance and by pointing to some 
contentious things, it really appears the latter to me.



(*) Also maybe that thread about tricking peer review.

I didn't understand the position that commit field should contain the
(indirect, fragile) tag instead of the (direct, robust) commit, but
those differences could be sidestepped by having both a 'tag' field and
a 'commit' field, IIUC.


I would not frame this way.  My view is not to replace something by
something else, instead, is to add something and/or several things.


I was thinking of adding the commit (intrinsic) to the git-reference, 
instead of only having a tag (extrinsic) in the git-reference as is 
mostly done currently.


I also want to mention that, except of a general notion of 'more 
robustness' and a specific command "guix freeze -m manifest.scm" and 
such, you never mentioned what your view was, so I had to guess.



The problem then was to somehow map the NAR hash to the FS identifier.


Yes, that’s the problem. :-) GNUnet FS identifier is one case.  And my
discussion here is: could we augment source origin to be able to deal
with various identifier?



A straightforward solution would be to just replace the https:// by
gnunet:// in the origin (like in https://issues.guix.gnu.org/44199,
except that patch doesn't support fallbacks to other URLs like url-fetch
does).


Somehow, your proposition would be to have a list as URI, right?

  (origin
(method gnunet-fetch)
(uri
 (list
   (string-append "mirror://gnu/hello/hello-" version
".tar.gz")
   
"gnunet://fs/chk/TY48PGS5RVX643NT2B7GDNFCBT4DWG692PF4YNHERR96K6MSFRZ4ZWRPQ4KVKZV29MGRZTWAMY9ETTST4B6VFM47JR2JS5PWBTPVXB0.8A9HRYABJ7HDA7B0"
   "shw:1:dir:9c1eecffa866f7cb9ffdd56c32ad0cecb11fcf2a"
(file-name "gnunet-hello-2.10.tar.gz")
(sha256
 (base32
  "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i")


Yes, though in a proper version of 44199 (which doesn't exist yet) it 
would just be integrated into url-fetch instead of having a separate 
gnunet-fetch.



It is not affordable, neither wanted, to switch from the current
extrinsic identification to a complete intrinsic one.  Although it would
fix many issues. ;-)


How 

Re: intrinsic vs extrinsic identifier: toward more robustness?

[Maxime Devos]

Op 03-03-2023 om 19:07 schreef Simon Tournier:

Hi,

I would like to open a discussion about how we identify the source
origin (fixed output).  It is of vitally importance for being robust on
the long-term (say 3-5 years).  It matters in Reproducible Research
context, but not only.

# First thing first
===

## What is an intrinsic identifier or an extrinsic one?
===

  - extrinsic: use a register to keep the correspondence between the
identifier and the object; say label version as Git tag.

  - intrinsic: intimately bound to the designated object itself; say hash
as Git blob or tree and at some extent commit.

>
> [... some reordering for convenience of replying ...]
>
> Please note that the identification and the integrity is not the same.
> Since intrinsic identifier often uses cryptographic hash functions and
> integrity too, it is often confusing.

To my understanding, there is only one 'real' identifier in Guix: the 
(sha256sum (base32 ...)) (*).  Those other identifiers like the URL in 
url-fetch and git-fetch are just hints on where to find the object -- 
very important hints without which finding the object is much more 
likely to fail, but just hints nonetheless.


While identification and integrity might be different concepts, 
content-based identifiers like (sha256 (base32 ...)) accomplish both at 
the same time.


(*) FWIW, I would like to point out that Guix theoretically supports 
some other hashes as well, though they aren't used for any in-tree packages.



The register must be a trusted authority and it resolves by mapping the
key identifier to the object.  Having the object at hand does not give
any clue about the key identifier.  And collisions are very frequent;
two key identifiers resolve to the same content – hopefully! we call
that mirrors. ;-)


I first thought you where writing about 'extrinsic -> intrinsic (e.g. 
hash-based)' registers, so I was confused by your comment about 
collisions -- to my understanding, no sha256sum collisions are known. 
Going by your comment about mirrors, I think you meant an 'intrinsic -> 
extrinsic' map instead, e.g. 'sha256 -> a bunch of appropriate URLs'.



Intrinsic identifier also relies on a (trusted) map but collisions are
avoided as much as possible.  Somehow it strongly reduces the power of
the authority and it is often more robust.


Who is 'the authority' here, how does the absence of collision reduces 
the power of the authority, and what is your point about reducing the 
power of the authority?  I was thinking of ‘the authority=Guix package 
definition’, but then only the 'more robust' part of your conclusion 
makes sense to me.  Also, as you used 'but' instead of 'and', it appears 
you consider relying on a trusted map to be a bad thing, but that 
appears basic security and patch review to me.



Whatever the intrinsic identifier we consider – even ones based on very
weak cryptographic hash function as MD5, or based on non-crytographic
hash function as Pearson hashing, etc. – the integrity check is
currently done by SHA256.


How about using the hash of the integrity check as an intrinsic 
identifier, like is done currently?  I mean, we hash it anyway with 
sha256 for the integrity check anyway, might as reuse it.



## For example, consider this source origin,
==

 (source (origin
   (method url-fetch)
   (uri (string-append "mirror://gnu/hello/hello-" version
   ".tar.gz"))
   (sha256
(base32
 "086vqwk2wl8zfs47sq2xpjc9k066ilmb8z6dn0q6ymwjzlm196cd"

where ’mirror://gnu’ is resolved by Guix itself.  Or this one,

 (source
  (origin
(method git-fetch)
(uri (git-reference
  (url "https://github.com/FluxML/Zygote.jl;)
  (commit (string-append "v" version
(file-name (git-file-name name version))
(sha256
 (base32 "02bgj6m1j25sm3pa5sgmds706qpxk1qsbm0s2j3rjlrz9xn7glgk"

where Guix clones then checks out at the specification of the field
’commit’.

Here both are extrinsic identifiers.  For the first example, the register
is defined by ’%mirrors’.  For the second example, the register is the
folder ’.git/’.

Intrinsic identifier could be plain hash or hashed serialized data.
Using Guix b8f6ead:

--8<---cut here---start->8---
$ guix hash -S none -H sha256 -f nix-base32 -x $(guix build hello -S)
086vqwk2wl8zfs47sq2xpjc9k066ilmb8z6dn0q6ymwjzlm196cd

$ guix hash -S git -H sha256 -f nix-base32 -x $(guix build hello -S)
11kaw6m19rdj3d55y4cygk6k9zv6sn2iz4gpimx0j99ps87ij29l

$ guix hash -S nar -H sha256 -f nix-base32 -x 
/gnu/store/3dq55rw99wdc4g4wblz7xikc8a2jy7a3-hello-2.12.1.tar.gz
1lvqpbk2k1sb39z8jfxixf7p7v8sj4z6mmpa44nnmff3w1y6h8lh
--8<---cut here---end--->8---

Or some Git-like tree md5 of the 

Re: Merging core-updates?

[Maxime Devos]

On 15-02-2023 19:51, Andreas Enge wrote:

I am trying to build openjdk13 without the patch as follows:
(define-public openjdk13
   (make-openjdk openjdk12 "13.0.13"
 "0pxf4dlig61k0pg7amg4mi919hzam7nzwckry01avgq1wj8ambji"
(source (origin
  (method git-fetch)
  (uri (git-reference
 (url"https://github.com/openjdk/jdk13u/;)
 (commit "jdk-13.0.13-ga")))
  (file-name (git-file-name name version))
  (sha256 #f)
(with the hash to be determined later).

This fails with
Initialized empty Git repository 
in/gnu/store/dyxp7njz9k20mar2h2cp8f2g438vigm2-openjdk-13.0.13-checkout/.git/
fatal: unable to access 'https://github.com/openjdk/jdk13u/': Could not resolve 
host: github.com

Do you know why? I am at a total loss as to what is happening...


You didn't write the hash.  As the hash is unknown, it would be 
irreproducible for the Guix daemon to grant the build process access to 
the network, so the Guix daemon doesn't.


You'll need to enter a hash (possibly a bogus one that the daemon can 
complain about later).


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Reducing useless module imports

[Maxime Devos]

On 03-02-2023 18:11, Julien Lepiller wrote:

gnu packages minetest) imports (guix build-system
minetest) but never uses the minetest-build-system. (gnu packages
abiword) uses (guix build-system gnu) but does not use the
gnu-build-system.



The build-system minetest-build-system does not exist; it's 
minetest-mod-build-system, which happens to be defined in (guix 
build-system build-system).  As such, (gnu packages minetest) actually 
does use (guix build-system minetest).


The idea was that in the future we might also package 'subgames' and 
texture packages, for which a 'minetest-subgame-build-system' and 
'minetest-texture-pack-build-system' build system could be defined.
It seemed a bit pointless to me to put those not-yet-defined build 
systems in separate modules.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: guix lint false positives and RFC patch

[Maxime Devos]

On 28-01-2023 22:07, Vagrant Cascadian wrote:

The other thing I remember being caught up on, which was not a
deal-breaker, per se, was hoping for a way to loop through a bunch of
@SOMETHING things ... I was not happy with:

+(if (>= (string-length (string-replace-substring
+(string-replace-substring synopsis "@acronym" "")
+"@code" ""))
+   80)

And then adding @command, @file, @acronym, etc. ... using increasingly
nested levels string-replace-substring would eventually become difficult
to read and surely there is a better way!


How about some regex:

(define regexp
  (delay (make-regexp "@(code|acronym|file)\\b")))

(define example "stuff @acronym @code @file")
(regexp-substitute/global #f (force regexp) example 'pre "" 'post)

$4 = "stuff   ".


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: UTF-8 progress bar

[Maxime Devos]

On 28-01-2023 14:17, Julien Lepiller wrote:

Hi Guix!

I have a patch waiting (https://issues.guix.gnu.org/59975) that will
change progress bars to use some unicode characters. I think they look
better, but I'm a bit afraid they might not look right on some config,
so I'd like to know if your terminal is able to show these characters:

"▏▎▍▌▋▊▉█▏▕"


It is (mate-terminal).


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Packaging: Need some help replacing a check phase

[Maxime Devos]

On 26-12-2022 17:15, Luis Felipe wrote:

Hi,

I'm packaging a Guile software but the package fails to build when I try to 
replace the check phase, and I can't see what I'm doing wrong.



As mentioned by Kaelyn, guile-build-system doesn't have a check phase to 
replace. The patch series at  adds 
support for tests to guile-build-system; you could apply those patches 
and then replace a phase (and maybe replace 'guile-test-driver' by 
'guile-proba' for autodiscovery).


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Antioxidant (new rust build system) update - 100% builds

[Maxime Devos]

On 03-11-2022 16:16, Ludovic Courtès wrote:

Hi,

[...]
Perhaps we could temporarily support the “old style”, using the run-time
transformation currently in your repo?  That would allow third-party
channels to migrate peacefully, and it would also reduce the likely hood
of breakage during transition in Guix (e.g., if an old-style package is
merged while the new branch is being built.)


That would be possible (some copy-pasting, and replacing
(define-public application/c-library/whatever ...)
with (define-public application/c-library/whatever (vitaminate/auto 
...)), but "guix refresh -l rust-..." won't be fixed until the source 
code changes are done.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Antioxidant (new rust build system) update - 100% builds

[Maxime Devos]

On 02-11-2022 12:20, Ludovic Courtès wrote:

[. ..]
That’s  but it probably needs work if
we want it to work reliably on all the packages.  My understanding is
that we’d need a “flag day” where we’d switch all Rust packages to
Antioxydant in one commit, is that correct?  Any ideas how to achieve
the big migration?


That's the idea I think, yes.  For this big migration, I propose to make 
the "guix style" good enough to do the switch at once (and let 
ci.guix.gnu.org build it etc. before merging).


On how to achieve this: the basic antioxidation (e.g. move 
#:cargo-inputs to inputs) is currently fully automated (*).  Then there 
is stuff like 'add this-input to that-package', which is 
'half-automated', in the sense that there are a bunch of declarative 
alists for that, interpreted automatically but declared manually; those 
could be interpreted by "guix style" as well.


Lastly, there is some more manual things too -- for making packages 
build under antioxidant, I sometimes found it convenient to update the
package (which sometimes also required packaging a dependency).  For 
those additions and updates, I propose to review+merge them (as 
cargo-build-system, not antioxidant-build-system) before the flag day / 
running "guix style".


(*) albeit at runtime, no corresponding source code tranformations (guix 
style) are written yet.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Questions about Cuirass

[Maxime Devos]

On 30-10-2022 13:50, James Hobson wrote:

Sorry for not getting back to you.

Looks promising!

I wish I could release everything under a free license. Baby steps though! I’ve 
managed to release a few things under LGPL since I started! That’s 100% more 
than before!


Sounds good.


But anyway. The biggest hurdle I see is that updating in an air gapped 
environment doesn’t seem supported because guix’s git url is hard coded. Does 
this need to be the case? If not, I might see if I can find an nice way of 
making this more configurable


There is a default, yes, but it's not hardcoded, you can override it. 
Have a look at "Invoking 'guix pull'" in the manual, in particular its 
'--url=URL' argument.



James


> [...]

I prefer no top-posting; my e-mail client, and more generally almost all 
e-mail clients I think, keep a thread of previous messages.  I would 
prefer to not to have to scroll down to read the rest of the message 
only to discover there is none.


Greetings,
Maxime


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Antioxidant (new rust build system) update - 100% builds

[Maxime Devos]

Hi,

100% (rounded up) of the packages build with antioxidant, though a very 
few still fail to build: .


So far, work on antioxidant has been done in a separate channel for 
convenience, but given that almost everything builds now, I think it's a 
good time to start looking into moving it into Guix proper (initially as 
a branch, as there are some remaining TODOs like e.g. 'why are some of 
the binaries made with antioxidant larger than with cargo-build-system + 
fix that').


More concretely, this would mean changing the 'runtime' transformations 
done by 'antioxidant-packages.scm' (in the style of '(guix)Defining 
Package Variants') to source code transformations ("guix style").


IIRC, Ludo' has some "guix style" patches for moving #:cargo-inputs to 
'inputs' and such; those could perhaps be used as a basis.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: bug#58859: profile contents depends on package order

[Maxime Devos]

Looks sort-of but not quite a duplicate of 
 (‘guix shell’ skips profile 
collisions checks) to me.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Questions about Cuirass

[Maxime Devos]

On 20-10-2022 23:19, James Hobson wrote:

Hello!

Currently evaluating guix for embedded systems at work. But I have a few 
questions that I can’t quite work out from the docs. Please feel no obligation 
to answer!

Please note that my guix journey is at its very beginning. I’ve not even had a 
go at packaging!

Question 1
We would need to host the guix substitute server in an airgapped environment. 
The server would contain plain guix packages, our in house packages, and maybe 
patched guix packages. Would that be possible without having to rebuild the 
entire guix package set? We don’t have so many build machines, especially not 
for armv7.


You can tell Cuirass to only build a selection of packages (and their 
dependencies), by using a manifest, then not all of Guix is compiled but 
only what's necessary for your particular purpose.


Also, your Cuirass instance still needs access to the source code of the 
packages somehow, which will need to be somehow be squared with your 
'airgapped environment', though maybe 'copy over the result of guix 
build --sources=transitive" would be acceptable (*).


(*) except that this is after application of snippet; some kind of 
"--sources=raw,transitive" may be needed.



Question 2 [...]


I don't know the answer to this.


Question 3
Our software is sadly proprietary. Is there a way for guix build to selectively 
unpack and patch all non-proprietary sources so that we can provide it to 
anyone who asks? I feel like if this isn’t a thing already, I guess I can write 
it in scheme?


I assume you meant 'patch all non-proprietary' -> 'patch out all 
proprietary', such that at least the free parts can be used?


In that case, this is done already in some package definitions in Guix, 
by a 'snippet' removing parts that are non-free, such that they are not 
built and are not part of "guix build --source". (See: ‘Snippets versus 
Phases’ in the documentation, though it doesn't mention non-free things 
directly).


The Guix user can still access the unpatched source code though, by 
inspecting the package definition and removing the snippet, so it looks 
to me like that option is only good for 'you aren't allowed to modify 
this part of the source code + guix build --source must produce 
something free', not for 'you aren't allowed to see or distribute this' 
situations.


Alternatively, you could avoid all this complexity by making your 
software free.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: crate importer throws

[Maxime Devos]

On 12-10-2022 23:51, Csepp wrote:

And yes, the error message could be clearer,
although I'm not sure where that should be fixed.


How about in the Guix code that uses the semver stuff?


Guile doesn't know
what packages correspond to what modules and it should probably stay
that way.


It doesn't need to know, e.g. Guix could do

(define-module (whatever) #:autoload (bar) (foo))

(define (foo-if-available)
  (catch 'unbound-variable
(lambda () foo)
(lambda _ #false)))

[...]

(define (stuff)
  (define foo (foo-if-available))
  (unless foo
(error "To use \"guix import crate\", you need to install 
guile-semver"))

  [... use foo ...])

Greetings,
Maxime.




OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: crate importer throws

[Maxime Devos]

On 15-10-2022 01:35, Csepp wrote:
of PGP Signed Part]]
[...] 
This works without adding guile:

guix shell --pure guile-semver -- "$(which guix)" import crate the-way
Any idea why?  I didn't add guix to the shell because I wanted it to use
the same guix profile.


"guix shell --pure guile-semver" resets all environment variables, no 
search path is set.


Could you check if

$ guix shell --pure -- "$(which guix)" import crate the-way

(i.e. a pure environment without any packages) works for you?  Maybe 
somehow there is some other guile-semver in your usual environment that 
breaks Guix.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: crate importer throws

[Maxime Devos]

On 12-10-2022 17:50, jgart wrote:

On Wed, 12 Oct 2022 14:24:26 +0200 Maxime Devos  wrote:

That still throws:

guix shell guile-semver -- guix import crate the-way
[...] 
WDYT


I think you need to add 'guile' as well (profiles don't properly compose 
yet w.r.t. search paths):


$ guix shell guile guile-semver -- guix import crate the-way

Can't reproduce locally though ('guix import crate the-way' works for me 
even without 'guix shell guile guile-semver').


$ guix describe
Generatie 1418:45:20 10 okt 2022(huidig)
  guix f0a6aaf
bewaarplaats-URL: https://git.savannah.gnu.org/git/guix.git
tak: master
commit: f0a6aafa22c2e7c7f33786dd7de7083a64401d01

Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: crate importer throws

[Maxime Devos]

On 11-10-2022 23:39, jgart wrote:

```
  λ guix import crate the-way
;;; Failed to autoload string->semver-range in (semver ranges):
;;; no code for module (semver ranges)
Backtrace:
In ice-9/boot-9.scm:
   1752:10  9 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
8 (apply-smob/0 #)
In ice-9/boot-9.scm:
 724:2  7 (call-with-prompt _ _ #)
In ice-9/eval.scm:
 619:8  6 (_ #(#(#)))
In guix/ui.scm:
2263:7  5 (run-guix . _)
   2226:10  4 (run-guix-command _ . _)
In guix/scripts/import.scm:
 92:11  3 (guix-import . _)
In guix/scripts/import/crate.scm:
 95:24  2 (guix-import-crate . _)
In guix/import/crate.scm:
 287:9  1 (crate->guix-package "the-way" #:version _ # _ #:repo _)
260:26  0 (find-crate-version #< name: "the-way" latest-v…> …)

guix/import/crate.scm:260:26: In procedure find-crate-version:
error: string->semver-range: unbound variable
```

WDYT


I think it should ask to run in an environment with "guile-semver" in a 
nice error message.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Small change request to the manual page "Building from Git"

[Maxime Devos]

On 10-10-2022 18:18, Mehmet Tekman wrote:

Sorry for the message duplication, it's the default with my email provider.

If "guix shell -D guix --pure" is included in the new version of the
manual, then I'm more than happy to drop my suggestion.
Thanks for the extra context, and the general tips!


If you do "guix shell -D guix --pure", you are using --pure, and 
consequently "guix git authenticate" and "make authenticate" will fail, 
contrary to what you seem to want.


My proposal was:


I suppose removing --pure (from the manual) might solve this problem.
However, --pure appears to have been added for a reason
(commit 43ec98ef3025f67ff4f66b7da0bcb79a6f088042), so I expect the solution is to rephrase things somehow (maybe something about running "guix git authenticate" outside "guix shell -D guix"). 


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Small change request to the manual page "Building from Git"

[Maxime Devos]

On 10-10-2022 16:18, Mehmet Tekman wrote:

Hello,


Try "guix environment guix --pure guix" or "guix shell guix -D guix" instead.


Yes I understand, but the manual states that:


The following command starts a new shell **where all the dependencies and 
appropriate environment variables are set up to hack on Guix**:
guix environment guix --pure


You are reading an old version of the manual, the latest version 
mentions "guix shell -D guix --pure".



This gives the impression that everything needed for `make
authenticate' to work is included in the above command (and I guess it
would be on the native distro).


> I think a small sentence mentioning the extra Guix dependency for
> non-native users isn't completely unwarranted, or perhaps maybe a hint
> in the linked "invoking guix environment" page?

As I wrote previously, foreign system / Guix System makes no difference 
-- in both cases, --pure changes the PATH, and not doing --pure should 
get you the 'guix' from ~/.guix-profile/bin/guix if Guix is set up

previously.

I suppose removing --pure (from the manual) might solve this problem.
However, --pure appears to have been added for a reason
(commit 43ec98ef3025f67ff4f66b7da0bcb79a6f088042), so I expect the 
solution is to rephrase things somehow (maybe something about running 
"guix git authenticate" outside "guix shell -D guix").

On Sun, 9 Oct 2022 at 11:08, Maxime Devos  wrote:

> [lots of text]

You seemed to have duplicated the previous mail here, but most e-mail 
clients keep the previous mails available, so there's no need.  Not 
top-posting also saves a little time for the reader.


Greetings,
Maxime;.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Small change request to the manual page "Building from Git"

[Maxime Devos]

On 06-10-2022 15:35, Mehmet Tekman wrote:

Hi there,

I'd like to request some small changes be made on this page:
 > https://guix.gnu.org/manual/en/html_node/Building-from-Git.html 



1. Authenticating on a foreign distro

When at the "make authenticate" stage of the build process on a foreign 
distro, this fails because it cannot find guix.
I think it's because the `guix environment guix --pure' command doesn't 
include it, or obscures its path.


As a workaround, I ran `PATH=/usr/local/bin/:$PATH make authenticate'


"guix environment guix" only includes the dependencies of guix, not Guix 
itself.  Try "guix environment guix --pure guix" or "guix shell guix -D 
guix" instead.


"guix environment guix --pure" does indeed 'obscure its path' -- that's 
what --pure is for, if you don't want that, don't include --pure.


Both of these are independent of whether you are on a foreign distro or 
Guix System.



2. Easy fix for failing `make check' tests

I had a few failing tests on my foreign distro relating to setting locales.
Digging around led me to this reddit thread for the solution:

https://old.reddit.com/r/GUIX/comments/jpq1uw/bashminimal507binbash_warning_setlocale_lc_all/


Here they suggest running `sudo guix install glibc-locales` instead of as a 
user.
Maybe this should be mentioned, since I'm not the only one coming to GUIX from 
another distro.


Despite the name on Reddit, the name is Guix, not GUIX.

Also, assuming you have installed the Guix daemon with your foreign 
distro's package manager, this is a bug in the foreign distro's 
packaging, see 
 in case of 
Debian.  You could ask your distro to do a similar fix.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Planning for a release, for real

[Maxime Devos]

On 07-10-2022 11:50, Ludovic Courtès wrote:

Hi,

Maxime Devos  skribis:


I have some unapplied security patches (from before the latest release
(1.3.0) (!)) (more precisely, some patches that prepare for actually
being able to write the security patches, once the preparation patches
are merged, the actual security fixes should be relatively simple); I
think it would be rather bad for known security fixes to not be
applied for a whole release, especially given how long release cycles
are in Guix.


Could you provide links to these?


'These' are the ‘openat’ patches:


If they depend on ‘openat’ in Guile, that won’t be an option
unfortunately since we’re not going to upgrade the ‘guile’ package
during that time frame.


However, upgrading the Guile package is not required, I proposed making 
a separate ‘guile-with-openat’ package (= current Guile + some patches), 
which can then be used by the service code: 
<https://issues.guix.gnu.org/54485> (without world rebuilds).


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: system/package testing?

[Maxime Devos]

On 06-10-2022 20:34, Maxime Devos wrote:

  -- I miss


Nevermind this, just some writing error during composing the message


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: system/package testing?

[Maxime Devos]

On 06-10-2022 17:51, Simon Josefsson via Development of GNU Guix and the 
GNU System distribution. wrote:

Hi

Is there any infrastructure to run a system-wide test suite, or at least
a per-package test suite?


There's "make check-system", for system services (and implicitly, for 
the software behind those services), and "make check", for individual 
packages.



 I'm not thinking of 'make check' that is run
during build, but of something similar to Debian's debci/autopkgtest to
test that an installed package works as expected (which may include
running the same self-checks as 'make check' would, but on the installed
package).


I do not understand the difference -- what's installed, is just what's 
built previously, on which "make check" is run, no?


If there are some additional tests beyond "make check", they can be run 
from the check phase (and if they require the package to be installed, 
the phase can be moved after 'install', as is done sometimes already for 
some Python packages).



When designed for Guix, it could be even better than debci/autopkgtest,
it could be something the Guix user could invoke on an installed system
to verify that all the installed packages passes self-tests


This is done automatically, during the build.  If there is a concern it 
depends on the kernel version or CPU model, "guix build --check" can be 
used to redo the build.


 -- I miss

that functionality from debci/autopkgtest.

I'm considering packaging libgssglue for Guix and do some Kerberos
related work on other related packages, but having confidence of any
changes without proper testing of the resulting installed package is
fragile.


If you have additional tests for libgssglue, you can submit them 
upstream, or if they are somehow inappropriate for upstream, do them in 
a check phase.



I'm sure other more complex parts of Guix could benefit from
similar testing too.  A system-wide testing of the published QEMU image
would be useful, to gain confidence that we don't introduce regressions
into a future release of the QEMU image.


See: "make check-system".


Maybe this already exists, and if so my question/request should be
converted into a request for publicity around the feature and a pointer
to its documentation. :)


Info: (guix)Running the Test Suite

Greetings,
Maxime


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Planning for a release, for real

[Maxime Devos]

On 06-10-2022 16:50, Ludovic Courtès wrote:

Hello Guix!

Will Guix’s 10th year be a release year?  I hope so!

We need to plan and coordinate.  Releases have to be a group effort;
some of the most important work won’t be coding but coordination.
Coordination is key.  I don’t think I should be spearheading that
effort, but I’m happy to be part of it.

Who’s ready to commit time towards that goal for the coming weeks?

Here’s a list of things to do to get there: [...]


I have some unapplied security patches (from before the latest release 
(1.3.0) (!)) (more precisely, some patches that prepare for actually 
being able to write the security patches, once the preparation patches 
are merged, the actual security fixes should be relatively simple); I 
think it would be rather bad for known security fixes to not be applied 
for a whole release, especially given how long release cycles are in Guix.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Guix System For Kids

[Maxime Devos]

On 02-10-2022 21:32, jgart wrote:

What do people think of having an education distro for kids?

Similar to these:

https://itsfoss.com/educational-linux-distros/

Could we use that Guix GUI written in smalltalk to get started?



Going by what I'm seeing on that page, in many cases 'education distro 
for kids' would be useful for adults not pursuing an education as well
-- I think we should focus more on the implied 'convenience' aspect than 
being 100% 'education' and 'for kids'.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: What 'sh' should 'system' use?

[Maxime Devos]

On 26-09-2022 09:04, Philip McGrath wrote:

[...]
(Very occasionally, a program really does want to invoke the shell, such
as when shell expansion is part of an existing API.)

From a different perspective, this is part of why I've recently been
thinking we should find 'sh' dynamically: most programs/environments
don't, and shouldn't, need bash{-minimal,-static}, so it seems wrong to
make it a mandatory dependency of libc.


In another thread, I proposed replacing 'system' by a macro 'system' 
that looks for for a, say, GUIX_BIN_SH preprocessor definition and then 
calls _guix_system(GUIX_BIN_SH,...) or such, and remove the 'system' 
function.


That way, glibc does not use bash-whatever anymore, but we still avoid 
doing things dynamically, avoiding the problems that dynamic finding 
entails.


For packages that use 'system', we would need to then resolve the build 
resulting build failure by passing -DGUIX_BIN_SH (maybe we could have a 
libc-system-function package that overrides the header containing 
'system' and automatically sets GUIX_BIN_SH)?



See (1) (reproducibility) -- also, you would need to modify the daemon for 
that, so there are compatibility concerns, and then we're
 stuck with the /bin/sh special case forever (unless breaking compatibility 
would later be considered acceptable).



I don't think there's a reproducibility problem.


You are proposing 'weak references' -- weak references are automatically 
broken if the thing referred to is GC'ed (the weak reference is weak, so 
it doesn't count as a reference that keeps it from being GC'ed).


That means that the build process depends on whether bash-whatever is in 
the store or not.


Even if not, the compatibility concerns remain, and incompatible daemons 
sound like a form of irreproducibility to me.



Guix already can create
reproducible containers with "/bin/sh" (e.g. 'guix shell coreutils
--container -- ls /bin') and without "/bin/sh" (as in package build
environments).

I haven't investigated whether adding the ability to create "/bin/sh" in
build containers would require modifying the daemon or just sending the
daemon different instructions. However, AIUI, Nix *always* creates
"/bin/sh" in build containers, which makes me further expect that any
change needed to the daemon would be small.

>

To be clear, I'm not proposing that we always create "/bin/sh" in build
containers. At a low level, I'm suggesting that we add the ability to
create "/bin/sh" when desired. I can imagine one possibility for a
high-level interface would be to create "/bin/sh" by default when an
input provides "bin/sh", and it might turn out that we end up wanting
"/bin/sh" in most or all build containers in practice, but I see those
as secondary questions.


Again, I don't see how special-casing /bin/sh even further is desirable.


There are a few dimensions here that I want to try to pick apart.

When you say:


a plain "sh" looked up in the $PATH (like other binaries) and substitute*-ed by 
Guix should suffice >

there are a few different things that might mean. > I think you're probably referring to 
the status quo, where "sh" is
looked up in the 'inputs' or a G-expression equivalent and an absolute
reference to that particular "sh" is embedded into the package being
built. (But, when cross-compiling, that "sh" would not be in the $PATH
in the build environment.)


Yes -- to be clear, the looking up in $PATH is for upstream, in Guix it 
would be patched with the absolute reference to something in 'inputs' 
instead.



There's a different question about $PATH vs. _CS_PATH that I'll address
later.

I see at least two reasons to prefer finding "sh" dynamically at run-time.

First, we have other POSIX-like shells packaged in Guix, such as dash,
zsh, and gash. Currently, to create an environment where one of these
shells is used to run 'system'-like functions (e.g. because dash is
supposed to be faster than bash), you would have to recompile everything
that depends on glibc. (Maybe you could craft a very ugly graft.)


dash, zsh and gash are incompatible, so you can't simply replace things 
-- looking it up dynamically would potentially introduce bugs. 
Additionally, 'sh' might not exist in /bin/sh or $PATH, so possibly it 
couldn't be found dynamically, and possibly the version it finds is 
incompatible (reproducibility).


If dash is faster than bash and sufficiently compatible, you can propose to


Second, sometimes people may want to create environments, images, etc.
without an "sh" available.


You can do this without dynamic finding and its downsides, see e.g. the 
preprocessor thing mentioned in the beginning.



In some sense this is a special case of using
an alternate shell, but the consequences of the status quo are
especially notable. Currently, practically any environment or image Guix
creates will include bash-static, because glibc refers to it.

For an especially ironic example, consider this note from `info
"(guix)Invoking guix pack"`:

Re: git guix checkout automation for contributors

[Maxime Devos]

On 25-09-2022 17:04, Danny Milosavljevic wrote:

Also, this doesn't work:

On Sun, 25 Sep 2022 16:06:06 +0200
Danny Milosavljevic  wrote:

(1) install the guix system
(2) log in as regular user
(3c) git clone --depth=1https://git.savannah.gnu.org/git/guix.git  guix \
&& cd guix \
&& guix shell


This issue had been reported in <https://issues.guix.gnu.org/50960#13>:

Liliana Marie Prikler wrote:

What would happen on the top-level of the Guix source tree or deep
inside the tree of a guile package that deals with manifests, that
aren't necessarily related to Guix?  I think we should try searching
for something less ambiguous first (".guix-shell/manifest" perhaps?)
and maybe also provide further options after manifest.scm (e.g. build-
aux/guix.scm or etc/guix.scm)


This issue was considered unlikely:

Ludovic Courtès wrote:

You mean a directory that contains a file named ‘guix.scm’ or
‘manifest.scm’ but that happens to do something completely unrelated?

We can never rule this out, but I’d say it’s unlikely (these two
conventions are rather well established) and it’s up to the user to pay
attention.

WDYT?


Later shown to be incorrect on the top-level of the Guix source tree 
more explicitly:


Maxime Devos wrote:

Guix itself doesn't follow this convention: the guix source tree has an 
unrelated
"guix.scm" file, that doesn't evaluate to a package.  I'd expect that running
"guix shell" within the guix source tree would be equivalent to
"guix environment guix", but currently this doesn't seem to be the case.


As such, it was proposed to leave out auto-detection of ‘guix.scm’:

Liliana Marie Prikler wrote: > Leaving out the auto-detection of 
‘guix.scm’ and ‘manifest.scm’ for now

is a good idea, given that Guix itself would be an edge-case for that.
[ proposed alternative solution ]


It was agreed that auto-detection fails here:

Ludovic Courtès wrote:

Guix itself doesn't follow this convention: the guix source tree has an 
unrelated
"guix.scm" file, that doesn't evaluate to a package.


Indeed, but that’s because Guix predates the convention.  :-)

Toggle quote (4 lines)

I'd expect that running "guix shell" within the guix source tree would
be equivalent to "guix environment guix", but currently this doesn't
seem to be the case.


Right, it would fail poorly in this particular case.


... and then ignored, AFAICT.

Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: substitute derivation: also substitute grafts?

[Maxime Devos]

On 19-09-2022 18:26, Josselin Poiret wrote:

Hi everyone,

Maxime Devos  writes:

Fallbacks might be necessary (not every store item is constructed from a
package), but it all sounds doable and efficient.  Also the union could
needs to be modified to ignore the .graft-offsets of the union'ed things.


If I understand the whole thing properly, the daemon already scans for
references and could register their location in the database, avoiding
the need for a potentially brittle in-store format as you highlighted,
although that would require a non-trivial change to both the daemon and
the database format.


If you do this, fallbacks are still required for old daemons.

Daemon database changes sound much more brittle to me than the simple
.graft-offsets -- the only brittleness I see is the unioning code, but 
that's easy to address.


OTOH, in the ".graft-offsets" proposal, the build process has to do 
reference scanning (even though the daemon will do so as ell later), 
which adds some inefficiency -- so perhaps performance-wise, the 
additional testing for the fallbacks and brittleness might be worth it.



In any case, I don't think it's necessary to treat packages in a special
way: any derivation can hold references and thus be grafted.


(Corrected a likely typo derivations->references.)

They indeed don't need to be treated specially -- in my proposal, the 
grafting code wouldn't care whether a store item is of a package or 
something else, the only thing it cares about, is the presence of a 
.graft-offsets file.


However (unless we go for daemon changes), this .graft-offsets still 
needs to be made somewhere -- my proposal was to do so this, in case of 
packages, in an additional phase (which would call a 
'make-graft-offsets' procedure '(make-graft-offsets #$output ...)'.


Then, I suppose this could be extended to non-packages as well (e.g. 
modify gexp->derivation and sexp->derivation to do the 
make-graft-offsets automatically).  However, packages should cover 
almost everything (profiles are mostly a bunch of symlinks and thus 
don't benefit from grafting optimisations much IIUC), and then 
gexp->derivation would insert an additional file without it being asked 
for, which would sound rather surprising to me, so I think that covering 
packages would be sufficient.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: What 'sh' should 'system' use?

[Maxime Devos]

On 19-09-2022 02:13, Philip McGrath wrote:

1) If we want to continue to hard-code a specific shell into Glibc,


We do, for reproducibility -- otherwise, the behaviour of the 'system' 
function depends on whatever is the current /bin/sh, and sometimes 
/bin/sh is updated (and on some foreign systems it might not even be the 
bash we are used to).


 I think we

should document the decision (for example, why 'bash-static' vs. 'bash-
minimal'?)


Because cycles -- bash-minimal is linked to a (shared) glibc, which is a 
separate package from bash-minimal, so glibc cannot use bash-minimal, it 
uses bash-static instead which is linked to a (static) glibc (which 
might use a bootstrap bash (not 100% sure), but it's statically linked, 
so no reference to the bootstrap bash remains IIUC).


Also, why?  This is an implementation detail.  Who would the target 
audience be for this documentation?



and recommendations for how packages should use it: '_PATH_BSHELL'
is the best mechanism I've heard of so far, though I wish it were
standardized, and the fact that it can't be portably assumed to be a string
constant could be surprising.


I consider _not_ using it, and using (4) instead, to be best.
If not suitable (for example, because a shell is needed to run an actual 
shell script), then a plain "sh" looked up in the $PATH (like other 
binaries) and substitute*-ed by Guix should suffice.




2) If we want to make 'sh' a weak/dynamic reference, I think we should
strongly consider arranging to make it available at '/bin/sh' when present. I
expect this option would require less patching of other packages*by far*  than
any other approach.


See (1) (reproducibility) -- also, you would need to modify the daemon 
for that, so there are compatibility concerns, and then we're stuck with 
the /bin/sh special case forever (unless breaking compatibility would 
later be considered acceptable).




3) If we want a dynamic 'sh' not located at '/bin/sh', I think we should
implement a function similar to '__bionic_get_shell_path()' and use it for
'_PATH_BSHELL', 'system', etc. That begs the question of how the function
should find 'sh', and I don't have an answer for that.


How about $PATH?


In principle, we could
design a configuration mechanism for 'confstr(_CS_PATH, buf, sizeof(buf))' and
use it to find the shell: that has some appeal, but making the mechanism
extensible enough to support "all of the standard utilities of POSIX.1-2017"
seems like a challenge.
What do you think?


(4) Stop using 'system' in applications -- instead use whatever the 
language's equivalent of Guile's system*, execl ... or Guix' 'invoke'. 
Why?  Because 'system'-like functions requires quoting the command line 
arguments whereas in 'system*'-like functions you could just pass a list 
of command line arguments, and it's easy to get the quoting wrong, 
especially if some of the arguments are generated dynamically.


As a bonus, this could often remove a dependency on bash{-minimal,-static,}.

Maybe we can eventually remove _PATH_BSHELL and 'system' from our glibc 
(and Guile, ...).


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Store Functor and Store Applicative

[Maxime Devos]

On 18-09-2022 16:51, jgart wrote:

Hi Guixers,

Why does Guix implement a monad but not a functor and an applicative?


To my knowledge:

Functor: all monads are automatically functors, if someone would like a 
functor interface, they can implement 'lift' and 'fmap' when needed. 
There just hasn't been a need yet.


Applicative functors: likewise.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Stumpwm Contrib Packages

[Maxime Devos]

On 11-09-2022 17:02, Trev wrote:

Hey Guix,

I am trying to decide whether or not to contribute a refactor of
stumpwm-contrib in gnu/packages/wm.scm. It feels like each contrib
module should be its own package with its own checkout and that it might
be a bad idea to update all of the contrib modules through one common
ancestor.

If you are not familar with stumpwm and stumpwm-contrib, you can see the
source repository here:https://github.com/stumpwm/stumpwm-contrib

The inheritance I am referring to is here:
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/wm.scm#n1942

My reasoning for this is that if breaking changes are introduced to one
module, but wanted updates happen to another, it would be nice to avoid
the breaking changes and get the updates.


If the stumpwm people put lots of components in a single
'stumpwm-contrib', I expect that they take care of making sure all the 
components _within a single version_ remain compatible, and that by 
picking a separate commit for each component in Guix, it is likely to 
encounter incompatibilities (breaking changes).


In the hopefully rare case where we encounter an incompatibility, we can 
still choose to override the checkout for the impacted package.


As such, I recommend keeping the status quo.

Greetings,
Maxime


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Updating minetest to 5.6.0?

[Maxime Devos]

On 17-09-2022 02:13, Jan Wielkiewicz wrote:

Forgot the patches...


Some problems

(1) a single package per patch (minetest-naturalslopeslib can be done in 
a commit before minetest-exile)


(2) no superfluous version prefixes -- remove the "v" from
   (version "v0.3.8") and replace (commit version) by
   (commit (string-append "v" version))

(3) naturalslopeslib is lgpl2.1+, according to the README.md

(4) In minetest-exile, input labels can be avoided by replacing

+(add-after 'install 'install-dependencies
+  (lambda _
+(symlink (string-append
+  #$(this-package-input
+ "minetest-naturalslopeslib")
+ 
"/share/minetest/mods/naturalslopeslib")

+ (string-append
+  #$output
+  "/share/minetest/games/exile/"
+  "mods/naturalslopeslib")))

(5) Going by https://content.minetest.net/packages/Mantar/exile/, more 
specifically its Website link, the home page is 
https://exile.planetofnix.com/wiki/pmwiki.php?n=Main.HomePage


(6) According to the content.minetest.net page, you forgot to mention 
the CC-BY-SA-3.0 license.


(7) Patches need to be sent to guix-patc...@gnu.org, not guix-devel.

(8) I found a directory utilities/ -- IIUC, these bundled mods aren't 
actually used by the game, so they could simply be removed in a snippet 
like minetest-naturalslopeslib


with

   (add-after 'install 'install-dependencies
 (lambda _
   (symlink (search-input-directory inputs 
"share/minetest/mods/naturalslopeslib")

   (string-append
 #$output
 "/share/minetest/games/exile/"
 "mods/naturalslopeslib")))


Some things I can confirm:

(1) the hash for mintest-naturalslopeslib matches
(2) likewise for minetest-exile

Note: I didn't actually try out these patches.

Greetings,
Maxime


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Updating minetest to 5.6.0?

[Maxime Devos]

On 17-09-2022 02:10, Jan Wielkiewicz wrote:
Done. See the patches (I can't really find "git send-email" on Guix for 
some odd reason). The hacks in the Exile package are quite ugly, but I 
don't know a better way, see below.




Try "guix show git", it will mention a 'send-email' output that you can 
install to have access to 'git send-email'.  See ‘(guix)Packages with 
Multiple Outputs’ on how to install such outputs.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Updating minetest to 5.6.0?

[Maxime Devos]

On 16-09-2022 19:57, Jan Wielkiewicz wrote:
Last one question before sending the patches: I'm adding a minetest game 
called Exile and it uses a mod called naturalslopeslib. I packaged both 
of them but Exile expects the lib to be installed in 
"/.guix-profile/share/minetest/games/exile/mods/naturalslopeslib/". 
Should I add the mod as a propagated-input of Exile or should I directly 
copy/link the lib into Exile's tree (if so, how do I go about that)?


How about modifying minetest-exile's phases to add a symlink in
#$output /share/minetest/games/exile/mods/naturalslopeslib
pointing to #$(this-package-input "minetest-naturalslopeslib")?

(I don't know how to do that with copy-build-system though, does that 
build system support phases?  I was thinking of a post-install phase or 
a post-unpack)


(Also I don't know if Minetest allows symlinks inside mods)

That way:

  * it doesn't have to be propagated (so naturalslopeslib only appears
in the (non-game) mod if the user actually asked to install the Guix
package, and not as a consequence of installing something else

  * by using a symlink instead of a copy, a little space, network IO,
disk IO and build time is saved, e.g. when the user installs both
minetest-exile and minetest-naturalslopeslib

I think it would be feasible to write a 'minetest-game-build-system' to 
mostly automatically do such things, but currently such a thing does not 
exist yet.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Updating minetest to 5.6.0?

[Maxime Devos]

On 16-09-2022 00:45, Jan wrote:

The minetest-mod-build-system has some (very basic) tests for testing
that the mods at least load with the new Minetest.

About that, I can never see mods installed with Guix in Minetest. Is the build 
system really working as intended? I'm running the latest Guix. Am I missing 
something? I don't see the mods on the current version nor the latest I 
packaged.


Seems to work over here:

$ guix shell --pure minetest minetest-mesecons
[ in the GUI: ‘Select Mods’ ]
[ GUI: ]
+ mesecons
+ Minetest Game mods

[ in the GUI: select +mesecons ]
[ ...]

Make sure to install Minetest and its mods in the same profile, and to 
log out and in again if it's not done with "guix shell" or "guix 
environment --ad-hoc" but with "guix home" or "guix install", such that 
Guix can set appropriate environment variables (MINETEST_MOD_PATH).



I tried installing a mod by force installing it into 
"share/minetest/mods/modname" using the copy-build-system just like mineclone2 
is installed, but it doesn't work this way.


minetest-mod-build-system does this too, in 'mod-install-plan'

Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Updating minetest to 5.6.0?

[Maxime Devos]

On 15-09-2022 19:59, Jan wrote:

Hello,

I almost effortlessly updated the minetest package to 5.6.0 so I wonder if 
there's a person who is in charge of updating minetest-related stuff or should 
I just send the patch?


No, Guix does collective maintenance, and AFAIK nobody has made a patch 
yet or volunteered to keep track of Minetest packages.  If you have a 
patch, you can simply sent it.



Is there any test procedure for minetest mods and games?


You can compile them:

$ ./pre-inst-env guix build minetest minetest-worldedit minetest-...

The minetest-mod-build-system has some (very basic) tests for testing 
that the mods at least load with the new Minetest.


For some additional testing, you could create a new world (creative 
mode), enable all the mods, place blocks of various mods, testing things 
out.


> Should I also update games and mods?

If you want to, but I would say it's only required if otherwise the mods 
fail to load or are otherwise broken by the update.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: substitute derivation: also substitute grafts?

[Maxime Devos]

On 15-09-2022 19:43, Csepp wrote:

Could we store the offsets of references somewhere at build time?


I now remember that idea, I forgot about that one.

My answer: I don't see why not, maybe by adding a phase to 
%standard-phases (at the very end, to avoid it becoming invalid) that 
saves it in, say, OUTPUT/.graft-offsets?


That would avoid grafting files that don't even have any references, 
where a copy-file (maybe using copy_file_range or such when available)

would suffice.

Fallbacks might be necessary (not every store item is constructed from a 
package), but it all sounds doable and efficient.  Also the union could 
needs to be modified to ignore the .graft-offsets of the union'ed things.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: substitute derivation: also substitute grafts?

[Maxime Devos]

On 15-09-2022 16:46, Csepp wrote:


Ricardo Wurmus  writes:


[...]
Did I say *all items*? Well, … grafts are not included, because graft
derivations are marked as not substitutable.

Can we change that conditionally? I would really like to avoid having
to build grafts on B when they have already been built on A.


I would love this too, because IO can be incredibly slow on HDDs and
large packages.  My netbook would be thankful.



There are some some opportunities for optimizations in the grafting code 
before substituting more -- for example, to avoid seek times, it would 
be possible to rewrite multiple files concurrently (maybe using 
'par-for-each' to process each file in a directory in parallel).


This is already done in (guix build grafts), except that:

 * 'find-files' itself is not parallelised, even though parallelising it
could potentially reduce the time spent seeking (*)
 * it uses (parallel-job-count), which is (IIUC) 1 by default because
   --cores=1 by default.  As grafts are more IO intensive than CPU
   intensive, maybe it would be reasonable to impose a _minimum_ amount
   of parallelism, I don't, know, 2 or 4 or so?

(I'm assuming the main problem here is seek times.)

Also combinable with the proposal of having substitutes for grafts.

Greetings,
Maxime.

(*) IIUC, the time for N parallel seeks should, in theory, ≃ 1 seek, for 
small values of N, because of elevator algorithms.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Needed for IceCat-102: rust-1.59 and rust-cbindgen-0.23

[Maxime Devos]

On 13-09-2022 16:35, John Kehayias wrote:

I think you missed the earlier part where I linked to an upstream bug for building 
some versions of firefox with cbindgen > 0.23.

Oops, yes.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Needed for IceCat-102: rust-1.59 and rust-cbindgen-0.23

[Maxime Devos]

On 13-09-2022 15:50, John Kehayias wrote:

Normally I would just go with the latest cbindgen to have that for future uses, 
but maybe we should just have 0.23 and 0.24? I have patches for both, just 
trying to see what would be cleanest here.


From what I've heard (and experienced in antioxidant), they are 
backwards compatible, so just the latest should be fine, though possibly 
firefox's Cargo.toml (assuming it uses those) needs to be patched to 
accept 0.24 instead of 0.23.


Greetings,
Maxime


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Request: build package with source tarball

[Maxime Devos]

On 12-09-2022 03:04, Ryan Prior wrote:
Hi there! Lately I've been testing distribution tarballs with a workflow 
like this:


 1. update some software in my source directory
 2. create a distribution tarball
 3. untar to a directory like /tmp/mypkg-src
 4. run: |guix build --with-source=mypkg=/tmp/mypkg-src|​


It would be nice to skip step 3 there and just run: |guix build 
--with-source=mypkg.tar.gz|​
Guix should then untar and use the result as the source directory for 
the build.


I believe you can do that already -- "guix build" accepts not only 
directories, but also regular files.


For example, the following builds the 'hello' package, but with the 
source code of 'sed' instead of 'hello' (doesn't make much sense, but 
just an example):


$ guix build hello "--with-source=hello=$(guix build --source sed)"

(no need to unpack tarballs!)


To soup the process up even further, allow |--with-source|​ to take a 
URL of a tarball, in which case Guix fetches and untars.


That is already supported:

$ guix build hello "--with-source=hello=file://$(guix build --source sed)"
(succeeds)

$ guix build hello "--with-source=hello=http://localhost/hello.tar.gz;
(network errors, because I don't actually have a tarball there.)

Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: how does antioxidant work?

[Maxime Devos]

On 10-09-2022 19:38, jgart wrote:

On Sat, 10 Sep 2022 18:17:50 +0200 Maxime Devos  wrote:


Caveat: the ‘examples’ and ‘benchmarks’ functionality is unlikely to be
implemented by me (even though they may be useful for development) as
they don't bring much value to Guix packaging. [insert the usual patches
welcome here]


I'm mostly wondering how we would get the rust community to adopt our rust 
tooling for serious rust development.

Do we expect rust people to use Guix for developing rust projects without cargo?


A next step I had in mind, was to separate some functionality of 
antioxidant-build-system into a Guix-independent 'antioxidant' Guile 
program, that can also be used by other distro's.


Then Rust people could do things like:

$ apt-get install rust-foo ...
$ antioxidant configure
$ antioxidant build
$ antioxidant check
$ antioxidant install

or

$ guix shell -D some-app
$ antioxidant configure
$ antioxidant build
$ antioxidant check
$ antioxidant install

or whatever.  Basically, separate 'package management' from 'building 
software' -- the former needs the latter, and the latter doesn't really 
care how packages are managed as long as it is told where to find the 
dependencies.


I don't think we should expect rust people to use Guix or antioxidant 
though -- it would be nice if rust people found antioxidant and Guix 
useful, but if they use Cargo, that's fine too for us.


(Not recommended though, because of the usual problems with 
single-language package managers and their registries.)



Are there other package managers in the rust community that people are using 
besides cargo?


Maybe bazel: https://github.com/mrboatsman/rust_bazel_hello_world (seems 
to be a mix of a package manager and a build system).


Also maybe crane: https://github.com/ipetkov/crane/blob/
 
I realize cargo has issues. I don't have a comprehensive list of them but I'm curious to find out more.


Maxime, what have you researched with regards to cargo issues or where
should I start studying up on that to get familiar with why antioxidant
is needed?

Is there a thread that introduces the need for a new rust build system and why?


There are:

* https://lists.gnu.org/archive/html/guix-devel/2022-03/msg00269.html
* https://lists.gnu.org/archive/html/guix-devel/2022-08/msg00194.html

but I don't think they are good introductions.

Anyway, for Guix, some of the main issues are:

  * overly picky versions requirements -- the rust-cbindgen-VERSION
crates are (at least, sufficiently) backwards-compatible.
  * dependencies like "rust-windows-..." must be packaged even when not
compiling for Windows.
  * cargo is inefficient -- if you compile FOO which depends on BAR
having the feature "a" enabled, and later compile FOO2 which wants
the feature "b", then IIUC FOO2 is compiled twice, once with "a" and
then with "b".  How about compiling it once instead, with both "a"
and "b".
  * cargo-build-system is inefficient -- if you do
"guix build this-app", and then do "guix build that-app", then the
compiled dependencies of "this-app" cannot be reused for "that-app".

Cargo does not seem to have an option to ... not do that.  Even if
it did, there is the previous issue with 'features'.

(Crane seems to somehow tell Cargo how to not do that, but it still
appears to have the feature problem and it heavily depends on
crates.io or Cargo.lock, not really suitable for distros).

Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: how does antioxidant work?

[Maxime Devos]

On 10-09-2022 16:50, jgart wrote:

On Thu, 08 Sep 2022 12:03:36 +0200 Maxime Devos  wrote:

There are also some other small things with some checks for vendoring,
#:test-options


Cool! Will this build system support rust development with guix or it's
only for packaging end user rust apps?


You can already do things like

$ guix build -f file-containing-a-definition-of-the-rust-library.scm

As how antioxidant treats dependencies is similar to, say, 
python-build-system and gnu-build-system, the previously compiled 
dependencies of the library will be reused.


Unlike C, the configure scripts (‘build.rs’ in Rust), when they exist, 
they do very little work.  For many libraries, the compile time is 
usually short (on the order of, say, a most dozen seconds (*), at least 
on my local computer and not counting tests) (**).


As such, a ‘setup package.scm to use a recursive local-file as origin, 
modify source files, do "guix build -f package.scm"’ should be feasible 
(**).  I wouldn't know how it compares to Cargo though, I never used that.


(*) Note: I haven't actually timed it.  Also, if you adjust the 
optimisation level to something lower and non-LTO, it should in theory 
decrease.

(**) With some exceptions, e.g. rust-proptest takes long to compile.

Caveat: the ‘examples’ and ‘benchmarks’ functionality is unlikely to be 
implemented by me (even though they may be useful for development) as 
they don't bring much value to Guix packaging. [insert the usual patches 
welcome here]


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: nix installed with guix on a foreign distro

[Maxime Devos]

On 10-09-2022 04:02, jgart wrote:

On Sat, 10 Sep 2022 02:45:46 +0200 Maxime Devos  wrote:

Worse, this is not just a service, but a distro.

I mean that I want to run nix, the package manager, not the distro.

Nix suffers from the holy trinity #~problem unlike Guix.

#$https://invidious.slipfox.xyz/watch?v=qjq2wVEpSsA


Nix comes with lots of packages, making it a distro.
Similarly to how you can install guix (distro) on top of a 'foreign 
distro' (e.g. Debian).


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: nix installed with guix on a foreign distro

[Maxime Devos]

On 27-07-2022 00:50, jgart wrote:

What do you think Debian people should be providing to make it easier to run 
the guix installed nix package?

Or, what should I ask the Debian people if I decide to inquire with them?

all best,


Right, you want to install "nix" (a distro) with "guix" (a distro) on 
$FOREIGN_DISTRO (a distro) (let's assume Debian).


My answer is: why not install nix on $FOREIGN_DISTRO directly?  There 
appears to be a nice Debian package already
 (at least, I assume it is 
nice, I've never used nix).


I mean, this is service stuff, not just an application.  In theory I 
suppose that, with lots of work, Guix could support some kind of 
'foreign services', but I think you'll be better off (less effort, 
better integration) with installing the service directly, with 
$FOREIGN_DISTRO when available.


Worse, this is not just a service, but a distro.  You would be putting a 
distro inside a distro inside a distro ... that's too much nesting, at 
least for me.  Sounds interesting to observe from a distance though ...


Now I notice I haven't actually answered your question ... Let's say, 
they shouldn't be providing to make that use case easier -- if they 
really want to, then they can decide to do so, but see the stuff above, 
would the effort be worth it?


Greetings,
Maxime


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Sanitizer of record fields?

[Maxime Devos]

On 08-09-2022 13:35, b...@bokr.com wrote:

Hi Simon, et al

On +2022-09-08 09:59:15 +0200, zimoun wrote:

Hi,

The website is currently failing [1] to build because a typo in some
package declaration.  The error message is not very helpful,

 srfi/srfi-1.scm:241:2: In procedure map:
 In procedure map: Wrong type argument: "https://www.qt.io/;
 building pages in '/tmp/gnu.org/software/guix'...


ISTM this "wrong type argument" is an infuriatingly common
and typically useless error message.

Would it be possible to have a debugging hook where the message is output
which if activated would show the call stack, so one could see where in
the user code it happened?

Activation could I imagine be by guile checking e.g.
"GUILE_THROW_DEBUG_WRONG_ARGUMENT" on invocation, to turn on the hook
in a way that would have zero performance effect if not activated.


Why a hook / environment variable in Guile? Backtraces are printed
by default, I do not see the benefit of disabling them by default and
adding an environment variable to enable them:


antipode@antipode ~$ guile -c '(map 0 1)'
Backtrace:
In ice-9/boot-9.scm:
  1752:10  6 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
   5 (apply-smob/0 #)
In ice-9/boot-9.scm:
    724:2  4 (call-with-prompt ("prompt") # …)
In ice-9/eval.scm:
    619:8  3 (_ #(#(#)))
In ice-9/command-line.scm:
   185:19  2 (_ #)
In unknown file:
   1 (eval (map 0 1) #)
In ice-9/boot-9.scm:
    218:9  0 (map 0 1)

ice-9/boot-9.scm:218:9: In procedure map:
In procedure map: Not a list: 1

Problem is already solved.

It appears that the website code catches exceptions and prints the error 
message,
but forgets to print the backtrace, but that seems to be a choice that 
the website

code made, not Guile. (Likely, an accidental choice, that can be improved.)

Greetings,
Maxime.




OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Sanitizer of record fields?

[Maxime Devos]

On 08-09-2022 13:35, b...@bokr.com wrote:

Can geiser trace stuff? IWBN to have something analogous to bash's
shopt for printing expression sources as they are read and/or executed.
Does something like that exist?


See: 'trace' in the Guile manual.  (This is a Guile feature, not a 
geiser feature.)


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Sanitizer of record fields?

[Maxime Devos]

On 08-09-2022 13:16, zimoun wrote:

Hi,

On Thu, 08 Sep 2022 at 11:32, Maxime Devos  wrote:

On 08-09-2022 09:59, zimoun wrote:

We could add a lint checker.  Is it a “good” idea?

We already have one, 'check-license'.

Yeah, but I was talking about check if the field return the expected
record type.  Not only for the license one.

Then maybe add more checkers, for the other fields as well? I'm not seeing
any downsides.

Because lint is not always applied, a check should be done when running
’make’ or a special target.  Is it a “good” idea?

I suppose it is a possibility, but it adds a few seconds to every 'make':

time ./pre-inst-env guix lint --checkers=license
make  all-recursive  [...]
gnu/packages/qt.scm:1373:13: qtshadertools@6.3.1: invalid license field
gnu/packages/tex.scm:11816:2: texlive-setspace@59745: invalid

It means that “guix lint” is not systematically run. :-)

I agree that the overhead on ’make’ is probably not worth.  Maybe, more
checks could be run by the pre push hook.  WDYT?

I suppose?  Not seeing any immediate problems, but I haven't ever pushed
to the Guix repo or ever used pre-push hooks ...

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: how does antioxidant work?

[Maxime Devos]

On 08-09-2022 03:17, jgart wrote:

Hi Maxime,

how does antioxidant build system for rust work at a high level?

https://notabug.org/maximed/cargoless-rust-experiments

all best,


There are two parts:

 * The actual build system (antioxidant.scm) -- it reads the Cargo.toml
   to see what the dependencies are, looks for the (compiled!)
   dependencies in the 'inputs' and 'native-inputs', runs build.rs,
   compiles it (with 'rustc') and install the compiled result (sources
   aren't installed).  It also runs tests.
 * The cargo -> antioxidant transformation code.  The procedure
   vitaminate/auto* takes a package, and rewrites it to fit with
   antioxidant -- one of these is moving #:cargo-inputs into 'inputs',
   and (recursively) rewriting these inputs. Sometimes phases need to
   be adjusted, sometimes for cargo there was #:skip-build? but
   antioxidant doesn't do that, and it runs tests, so it sometimes
   turns out tests need to be disabled.

   (This will have to later be changed to _source code_
   transformations, maybe with "guix style", to integrate things into
   Guix proper)

There are also some other small things with some checks for vendoring, 
#:test-options


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Sanitizer of record fields?

[Maxime Devos]

On 08-09-2022 09:59, zimoun wrote:

We could add a lint checker.  Is it a “good” idea?

We already have one, 'check-license'.

Because lint is not always applied, a check should be done when running
’make’ or a special target.  Is it a “good” idea?


I suppose it is a possibility, but it adds a few seconds to every 'make':

time ./pre-inst-env guix lint --checkers=license
make  all-recursive  [...]
gnu/packages/qt.scm:1373:13: qtshadertools@6.3.1: invalid license field
gnu/packages/tex.scm:11816:2: texlive-setspace@59745: invalid

real    0m1,492s
user    0m3,331s
sys    0m0,214s

As such, here's an alternative proposal: instead of checking it at 
compile-time
(which is currently impossible), let's check them at runtime, with a 
field sanitizer.


Given that, unless I'm mistaken, build-aux/compile-all.scm loads every Guix
module anyway, and given that 'license' isn't thunked or delayed, 'the 
runtime

check' would also a compile-time check.

(Performance impact on "guix ..." commands would need to be checked.)

Alternatively, some error checking could be added to the website code, 
to indicate
which package and which is wrong.  Or maybe the website code can run the 
'license'

linter first.

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 07-09-2022 14:17, Andreas Enge wrote:

Am Mon, Sep 05, 2022 at 03:03:34PM +0200 schrieb Maxime Devos:

I meant 'snippet' in this subsubsection. Bugfixes seem useful to have in the
result of "guix build --source", and appear to be required to be in there for
the 'corresponding source' thing.

If we distribute the upstream source (as a "binary" substitute) and the
patch (through the Guix sources), my impression is that we do publish the
corresponding source.


Well, yes, but just sharing "guix build --sources=transitive ..." is 
more convenient than having to include a copy of Guix as well. And for 
people unpacking the result of "guix build --source ..." + "guix shell 
-D ...", having the bugfixes (where possible) included in there is 
convenient.


So yes, not strictly required, but I do consider including the fixes in 
the 'source' field (where possible) to be more practical.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH v2] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

On 05-09-2022 15:12, Maxime Devos wrote:


On 05-09-2022 11:47, Ludovic Courtès wrote:

Hi,

Maxime Devos  skribis:


On 02-09-2022 15:12, Ludovic Courtès wrote:

Hello,

Liliana Marie Prikler  skribis:


* doc/contributing.texi ("Snippets versus Phases"): Replaced with...
("Modifying Sources"): ... this.  List more use cases and some 
principles.
It’s been a while; this looks like a nice improvement to me.  It’s 
a bit
long, but perhaps that’s avoidable.  If there are no objections, 
I’d say

go for it.

Have you seen my v2?

I guess I haven’t (I was catching up with email as I returned from
vacation).

Hmm should you send the latest agreed-upon version to 
guix-patc...@gnu.org for clarity?


I'll integrate the agreed-on changes into a v3 and merge some parts of 
Liliana' version and send it ...


See: #57598.

Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 09-08-2022 18:30, Maxime Devos wrote:

  the
result of 'guix build --source' would be unusable on non-Guix systems, and also 
likely unusable on
Guix systems of another architecture.
--8<---cut here---end--->8---


The Oxford comma or lack thereof are acceptable stylistic conventions, 
but not a grammatical error. There are only two conjucts here though, 
so I don't think the Oxford comma applies here. Additionally, the two 
conjucts are about mostly the same thing, but just different 
(similar!) subcases that merely happen to need different 
qualifications (unusable on non-Guix vs. likely unusable on 
different-arch Guix), so I don't think that 'also' brings something here.


Oops, the 'also' was present in the original. I removed 'also', but 
found it to be more readable with 'also', so I'm keeping 'also' anyways.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH v2] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

On 05-09-2022 11:47, Ludovic Courtès wrote:

Hi,

Maxime Devos  skribis:


On 02-09-2022 15:12, Ludovic Courtès wrote:

Hello,

Liliana Marie Prikler  skribis:


* doc/contributing.texi ("Snippets versus Phases"): Replaced with...
("Modifying Sources"): ... this.  List more use cases and some principles.

It’s been a while; this looks like a nice improvement to me.  It’s a bit
long, but perhaps that’s avoidable.  If there are no objections, I’d say
go for it.

Have you seen my v2?

I guess I haven’t (I was catching up with email as I returned from
vacation).

Hmm should you send the latest agreed-upon version to guix-patc...@gnu.org for 
clarity?


I'll integrate the agreed-on changes into a v3 and merge some parts of 
Liliana' version and send it ...


Greetings,
Maxime



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 08-08-2022 23:51, Andreas Enge wrote:

20.4.5.3 Fixing technical issues (compilation errors, test failures, other bugs
...)
Usually, a bug fix comes in the form of a patch copied from upstream or another
distribution. In that case, simply adding the patch to the 'patches' field is
the most convenient and usually does not cause any problems; there is no need
to rewrite it as a snippet or a phase.
If no ready-made patch already exists, then choosing between a patch or a
snippet is a matter of convenience. However, there are two things to keep in
mind:
First, when the fix is not Guix-specific, it is strongly desired to upstream
the fix to avoid the additional maintenance cost to Guix. As upstreams cannot
accept a snippet, writing a patch can be a more efficient use of time.
Secondly, if the fix of a technical issue embeds a store file name, then it has
to be a phase. Otherwise, if a store file name was embedded in the source, the
result of 'guix build --source' would be unusable on non-Guix systems and
likely also unusable on Guix systems of another architecture.

Do you mean "phase" here instead of "snippet"? That is what I usually do...
My practice is to use a patch when the goal is to eventually change the
source code upstream (or it is already changed upstream and we can take a
patch from their git repo, say), and a phase when one cannot expect upstream
to incorporate the changes since they are specific to Guix (like embedding
store paths, as Liliana writes, where apparently a phase is the only option
anyway).


I meant 'snippet' in this subsubsection. Bugfixes seem useful to have in 
the result of "guix build --source", and appear to be required to be in 
there for the 'corresponding source' thing.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Clojure & Maven Resolver packages

[Maxime Devos]

On 03-09-2022 17:38, Roman Scherer wrote:


Hello Guix,

I would like to enable the S3 transporter for the Clojure package. For
this I need version 1.8.2 of the Maven Resolver packages. Right now we
have version 1.6.3 packaged in Guix.


AFAIK, you don't need to.  IIUC, the patch series at 
 adds it.



Should I update the existing packages to 1.8.2 or create new package
variants with a 1.8 prefix for this?

Right now I have the variants, but I was wondering what the best
approach would be.

Thanks, Roman.


I recommend using something else than Amazon S3 -- Amazon is known for a 
number on unethical practices. E.g., after searching for 'Amazon ethical 
practices', I found 
.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH v2] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

On 02-09-2022 15:12, Ludovic Courtès wrote:

Hello,

Liliana Marie Prikler  skribis:


* doc/contributing.texi ("Snippets versus Phases"): Replaced with...
("Modifying Sources"): ... this.  List more use cases and some principles.

It’s been a while; this looks like a nice improvement to me.  It’s a bit
long, but perhaps that’s avoidable.  If there are no objections, I’d say
go for it.


Have you seen my v2? It has a somewhat different structure (the 'Problem 
-> solution’ structure proposed by Julien), Liliana's patch is more 
‘Solution -> relevant problems’.


Liliana's patch works too, but I would like to make sure you haven't 
overlooked my patch.


If Liliana's patch is preferred, we can go with that one, if my patch is 
preserved, I could look at addressing the comments on the wording for a v3.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

HTTP or HTTPS download URLs?

[Maxime Devos]

(guix gnu-maintenance) uses https://nongnu.freemirror.org/nongnu, 
whereas (guix download) uses http://nongnu.freemirror.org/nongnu/. This 
discrepancy causes my fix for  to 
fail for the savannah updater. Can I just switch http -> https, or is 
there a reason for using http instead of https here?


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: 04/04: gnu: Add fwupd.

[Maxime Devos]

On 01-09-2022 08:27, Reza Housseini wrote:


> No, unless you're using a custom update protocol that fwupd does not
> already support.

The user guide[2] likewise states:
> The OEM vendor is in full control over what models are supported
and is
> the only entity that can add support for new hardware.

The vendor agreement[3] further grants no freedom safe for the
second –
the freedom to make unmodified copies.

Cheers

[1] https://fwupd.org/lvfs/docs/vendors

[2] https://fwupd.org/lvfs/docs/users

[3] https://fwupd.org/lvfs/docs/agreement



Isn't fwupd a package manager which could technically be replaced by guix?


IIUC, fwupd not only receives firmware updates, it also sends the 
firmware to the device (installing it). While Guix supports installing 
software, I don't think it supports installing firmware in specific (at 
least, not yet). And to extend Guix with support for installing 
firmware, I expect it would use a tool like fwupd to actually send the 
updated firmware to the hardware.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: usage of basu as requirement for sd-bus

[Maxime Devos]

On 30-08-2022 11:27, muradm wrote:


IIUC, everything using basu also works fine with elogind (*), so the
'status quo' of still using elogind (for old and new) seems harmless
to me (except for size -- basu is smaller).


I don't find the "everything using basu also works fine with elogind"
statement/assumption/guess correct, as per contents of elogind and
basu. See above comment for ifdef thingy. 


From the README.md of basu:


The sd-bus library, extracted from systemd. Agreed on th

Some projects rely on the sd-bus library for DBus support. However not all
systems have systemd or elogind installed. This library provides just 
sd-bus

(and the `busctl` utility).

This does not look like basu adds additional functionality.


My intention is not to have something that is not used. Roughly, if
elogind is not used, why should I have it on my system.
You should have it because the alternative (i.e., sometimes using basu 
and sometimes using elogind) increases disk space usage -- it's all 
internal, unless there's a bug you shouldn't notice it's using elogind 
instead of basu unless you're doing "guix edit" or such.

Basically,

elogind provides: elogind, loginctl, busctl, libelogind (sd-bus, 
sd-login ...) ...

basu provides: busctl, libbasu

If basu is enought for package it should dependen on basu IMHO.

So my reason is not directly-storage-only, but dependency which
impacts storage in some or another way.
We have package outputs, we can separate the libelogind and busctl from 
the rest. elogind is used, just not in its entirety.

Btw, how much storage are we talking about when having some
packages depend on elogind and some on basu? Is it user
storage or build server/substitute storage concern? 

For basu and elogind itself: 0.9 MiB and 4.2 MiB

For basu and elogind in total: 72.9 MiB and 172.8 MiB.

(See: "guix size").

The latter numbers are a bit misleading, as one of the dependencies is 
'shepherd' and 'libgc', which would be installed anyway by other 
software, and because elogind refers to pkg-config while it probably 
shouldn't.


On "Is it user storage or build serve/substitute storage concern": yes. 
There isn't really a "user / substitute storage" distinction, unless you 
count baked nars. But that's just multiplying the storage by approx. 2 
(ignoring deduplication).


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: usage of basu as requirement for sd-bus

[Maxime Devos]

(**) This is just a guess about what your goal was, maybe you had a 
different reason in mind. E.g., basu seems to be more active than 
elogind.


Oops I misread the dates -- the latest commit in basu was before the 
latest commit in elogind.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: usage of basu as requirement for sd-bus

[Maxime Devos]

On 30-08-2022 09:59, muradm wrote:


Hello,

basu is sd-bus library extracted from systemd.

Currently, there are two packages depending on it,
which are mako and grimshot.

In https://debbugs.gnu.org/cgi/bugreport.cgi?bug=56859,
I suggest switching xdg-desktop-portal-wlr to basu.

In very same issue, Maxime asks to discuss switching
_all_ dependents of elogind to basu.

[1] Some elogind dependents, like wireplumber, as per
code depends on sd-login.h also in module-logind.c.
While I have wireplumber-without-elogind locally,
I don't propose switching it basu, because someone
may want module-logind.c to work.

[2] Currently there are 1461 packages depend on elogind.
First, all of them should be analyzed if they do use
sd-bus only, those can be switched to basu. Then
those using more than sd-bus should be analyzed if
elogind is missing would their functionality be hurt.
If these problems are like [1], then IIUC these problems would manifest 
as build errors. Checking for build errors is relatively simple by 
pushing to a separate branch first, evaluating it on ci.guix.gnu.org and 
checking for new build failures.

Because of [1] and [2], I find it not feasible/not
possible to blindly switch _all_ dependents from
elogind to basu.

Do I miss anything else here? 


IIUC, everything using basu also works fine with elogind (*), so the 
'status quo' of still using elogind (for old and new) seems harmless to 
me (except for size -- basu is smaller).


As far as I know, the benefit of 'basu' is using less storage (**).  If 
most dependents are switched from elogind to basu, then this benefit can 
be fulfilled. But if we just do a mix of elogind and basu, then we have 
both elogind and basu in the store, _increasing_ the storage footprint 
instead of lowering, which is the opposite of the goal of lowering 
storage usage.


As such, assuming that lowering the storage footprint was your reason 
for switching to basu, I think we should either try switching _all_ 
packages to basu or keep using elogind and add elogind instead of basu 
to new dependents.


Greetings,
Maxime

(*) This is an unverified guess. If disproved, my reasoning becomes a 
lot weaker.
(**) This is just a guess about what your goal was, maybe you had a 
different reason in mind. E.g., basu seems to be more active than elogind.




OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Guix Plover package issue

[Maxime Devos]

On 28-08-2022 19:44, Matt wrote:

However, this fails on

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
error: python-plover: unbound variable

Including (gnu packages stenography) doesn't resolve it.  Using guix
edit plover, I see that the definition is in
gnu/packages/stenography.scm. Which module should I be using?


You did "guix edit plover", so you found the module to which "plover" 
belongs -- you found "plover", not "python-plover".


To find python-plover, try "guix edit python-plover".

Given that it does not exist, maybe you need 'plover' instead of 
python-plover.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: antioxidant update: librsvg builds, and other things (core-updates)

[Maxime Devos]

On 28-08-2022 00:04, Liliana Marie Prikler wrote:

Am Samstag, dem 27.08.2022 um 22:01 +0200 schrieb Maxime Devos:

On 27-08-2022 21:54, Liliana Marie Prikler wrote:

* Due to how regularised the Rust build system is, it's feasible
to compile tests even when cross-compiling (*), so cross-compiled
could run the cross-compiled tests on the system they are
cross-compiling for after the cross-compilation to verify their
cross-compiled software.

How exactly does this work without emulating the system in
question?

It works by not performing any work except compilation -- Guix'
responsibility would only be to cross-compile and install the tests
(_not_  running them), you are supposed to install the cross-compiled
thing (including tests) on the target system and run the tests on the
target system.

This doesn't strike me as a rust-specific setup, though.  In principle,
you should be able to do the same with a C/C++ program, but most of the
time "make check" implies both building and running the tests.


The difference between Rust and many C/C++ setups here, is that with 
Rust it's trivial to only compile and install the tests without running 
them (currently, antioxidant-build-system compiles+installs and runs the 
tests in separate phases), whereas in case of C/C++, there usually isn't 
a convenient '"make install-the-tests-without-running-them" target, 
rather building and running the tests is combined in a single "make 
check" as you note.


As I've written previously:


Due to how regularised the Rust build system is, it's feasible
to compile tests even when cross-compiling (*)
but that does not appear to be the case for C/C++, as you've noted with 
your comments about "make check" both building and running the tests.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: antioxidant update: librsvg builds, and other things (core-updates)

[Maxime Devos]

On 27-08-2022 21:54, Liliana Marie Prikler wrote:


  * Due to how regularised the Rust build system is, it's feasible to
compile tests even when cross-compiling (*), so cross-compiled could
run the cross-compiled tests on the system they are cross-compiling
for after the cross-compilation to verify their cross-compiled
software.

How exactly does this work without emulating the system in question?


It works by not performing any work except compilation -- Guix' 
responsibility would only be to cross-compile and install the tests 
(_not_ running them), you are supposed to install the cross-compiled 
thing (including tests) on the target system and run the tests on the 
target system.


Greetings,
Maxime



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

antioxidant update: librsvg builds, and other things (core-updates)

[Maxime Devos]

Some updates:

 * core-updates is now targeted instead of master (because the librsvg
   on core-updates has a less complicated build system and hence easier
   to support with antioxidant)
 * librsvg now builds. It's about 3 times larger than the
   cargo-build-system librsvg (at least, the librsvg from master,
   didn't compare against core-updates yet because there was no
   substitute available), so perhaps some compilation flags need to be
   changed.
 * According to ci.guix.gnu.org, antioxidation is at 90%, but some of
   the 'failures' are because of cancelled builds in dependencies, the
   real number should be higher. Maybe 'Restart all builds' would be
   correct that, though I don't know if that wouldn't restart too much
   builds (including builds that succeeded).
 * Running tests is supported, #:tests? and #:parallel-tests? is
   respected, there is an option to skip some tests (using
   #:test-options).  As antioxidant doesn't do #:skip-build? #t, I
   expect more packages to be tested with antioxidant than with
   cargo-build-system.

   There are some test failure, things like "tries accessing the
   network" are simply skipped. Sometimes it is assuming incorrect
   things about the build environment such assuming that stderr is a
   terminal, those are simply skipped. Sometimes test files are removed
   from the tarballs uploaded at crates.io, this appears to be
   intentional, there is a project to remove all such 'bloat',
   disregarding the value of tests -- for those packages we will have
   to switch to git checkouts, but for now I simply disabled tests there.

   I've also found some time bombs (e.g. rust-rustls appears to have a
   certificate expiration problem), there tests are disabled too.

   There were also some genuine test failures, some of them have been
   reported upstream but not all of them yet.
 * antioxidant now has some defenses against bundling -- it detects if
   the feature "bundle", "vendor" or "vendored" is implicitly enabled
   and if so, bails out unless they were explicitly added to the
   package definition.
 * I don't know if I mentioned it previously, but antioxidant now
   supports some 'unstable' rust code even when using 'stable' Rust
   compilers, by setting RUSTC_BOOTSTRAP=1. It's 'not supposed to be
   used', but if it's good enough for the Rust compiler, then it should
   be good enough for other software too, I'd think.

Some questions:

 * Some Rust crates have 'examples' and 'benchmarks' that can be
   compiled and installed. I could teach antioxidant to compile and
   install them, though it appears to provide very little value at cost
   of more compile time and a greater closure size (more binaries, more
   dependencies and wasn't there some bug with grafts whose fix causes
   outputs to be substituted that ended up unused? Though hopefully
   someone could figure out a better fix ...)
 * Due to how regularised the Rust build system is, it's feasible to
   compile tests even when cross-compiling (*), so cross-compiled could
   run the cross-compiled tests on the system they are cross-compiling
   for after the cross-compilation to verify their cross-compiled software.

   Currently, tests are only compiled and installed when #:tests? #true
   which is only the case when compiling natively(*), but with some
   work I could separate building tests from running tests and build
   tests by default when cross-compiling. Downside: tests have to be
   installed in an output, which increases the closure size.

   (Currently tests are installed in an output, but I could change that
   to the working directory).

(*) antioxidant doesn't support cross-compilation yet, but in the past 
I've tried out cross-compiling the standard library and it seemed to 
work, though it wasn't completed.




OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Clarify the license field of the package

[Maxime Devos]

On 22-08-2022 11:02, 宋文武 wrote:


Hello list, I have some questions about the 'license' of a package,
currently defined as:

 The license of the package; a value from ‘(guix licenses)’, or a
 list of such values.

1. It's the license of source files (guix build -S) or built binary
files?


(If 'built binary files', I would include generated or copied 
documentation in the list. And icons, .desktop files, ..., I'm not 
restricting myself to _executable_ binaries here and also not to 
binaries that aren't sources as well.)


Rarely, there is some weirdness where the source code is free 
(VSCodium?) but the official build has a non-free license (VSCode?). At 
least for that example, it doesn't apply to Guix though (because 
VSCodium is not packaged, and because with some rare exceptions we build 
from source).


However, in my experience, in free software they almost always have the 
same license, so the distinction appears meaningless to me with the 
possible exception of build scripts and test files (including, but not 
limited to, test code).


I think it should include the source files, as the license of the source 
is important for people doing 'guix build --source'.



2. When its value is a list of multiple licenses, it's files under
different licenses (eg: lib/*.so under LGPL, while bin/* under GPL),
or files under one license select from choices?

My guess is that the license field is for source files since we can
disable binary substitutes, and list is used for files under different
licenses.

Does my guess is correct?  Thank you!


As answered in a reply to a patch, myself I go for 'files under 
different licenses' -- to me it seems hard to go wrong with 'just 
include all participating licenses' instead of trying to make a selection.


However, keep in mind that sometimes a file is part licensed as, say, 
BSD(*), part as Expat, with modifications under the GPL -- to me it 
appears that for practical purposes you could consider such a thing to 
be 'effectively GPL', but that's not 100% accurate, as it appears 
required to preserve the BSD and Expat license text. (Such things can 
happen when incorporating code from other, differently-licensed, projects).


(*) let's say without the advertising clause or whatever it was (IIRC 
and IIUC the original BSD was incompatible with the GPL?).


If there's some consensus, I think it would be nice to clarify this 
matter in the manual.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [RFC] Use LLVM_BUILD_LLVM_DYLIB instead of BUILD_SHARED_LIBS

[Maxime Devos]

On 20-04-2022 12:56, Zhu Zihao wrote:

We may introduce following problems if we apply this solution.

1. Increase the closure size of LLVM.

By default, if LLVM_BUILD_LLVM_DYLIB is set true, LLVM still tries to
build the static archive. This may increase the closure size of LLVM.
And some package linking with LLVM may use the static archive instead of
linking to the dynamic library.

My opinion: If we're OK with the bigger closure size, that's not a
problem. If not, we may consider disable the static archive generation.
For LLVM components, we can use `LLVM_LINK_LLVM_DYLIB` to ask these
packages to link with the shared version. For package use `llvm-config`
directly or indirectly, we can pass `--link-shared` to it.


IIUC, this causes the static libraries to be built, right?

If so, they could be moved into a separate output.

I would prefer to disable static libraries, those cause problems with 
grafts and size usage. If not possible, I would prefer them to be in a 
separate output, to avoid them being used by accident and to reduce the 
closure size of dependents of LLVM that use the shared version.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: FSDG issues of SCUMMVM-based games

[Maxime Devos]

On 24-08-2022 22:24, zimoun wrote:

My understanding of the Debian argument is:

  1. the licence is BSD-like respecting the Debian Free Software Guidelines

  2. point #3 of DFSG [2] says «The license must allow modifications and
  derived works, and must allow them to be distributed under the same
  terms as the license of the original software.»

  3. considering game data, all people are equals – from original author
  to users – because the tool set for modifying these game data does not
  exist anymore

Therefore, drascula is part of the ’main’ Debian archive, scummvm too.


I remember (3). I find this an interesting argument.

As far as I know, the 4 freedoms and the whole 'free software!' thing 
was not a goal in itself, but rather a means to  a goal, a response to 
'many times some people and organisations have imposed impractical 
restrictions on software, causing problems (example: this situation in 
19NN, that situation in 20??, ...) -- can we identify the problems and 
generalize until we have a set of rules (4 freedoms) that need to be 
respected to avoid the problems?'.


As far as I know, drascula situation is not comparable (see: 'all people 
are equals') to the old problems. Yet, I cannot say it's free software 
(without the tools, it's effectively a binary instead of source code 
until, if ever, the tools are reinvented) (see some of my other 
responses in this thread).


As such, do the 4 freedoms need some refinement to accept drascula, do 
we have to weaken our requirement of 'only free software' for special 
situations like this one, or do we remove drascula, or is there somehow 
a fourth option I'm not thinking of?


Myself, I do not know the answer. However, I cannot help with the first 
option, the second sounds iffy to me (the exception would need to be 
worded really well, or it would be a 'case-by-case' matter which could 
take a long time to decide case-by-case, and in both cases it doesn't 
seem to fit in 'GNU: free software!').


As such, for me, only the third (removing drascula) is practical for me, 
but there are other people here too which could perhaps, say, do the 
first or something ...


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: FSDG issues of SCUMMVM-based games

[Maxime Devos]

On 24-08-2022 22:24, Vagrant Cascadian wrote:

Is it Functional Data:

   https://www.gnu.org/distros/free-system-distribution-guidelines.html

   "For example, some game engines released under the GNU GPL have
   accompanying game information—a fictional world map, game graphics,
   and so on—released under such a verbatim-distribution license. This
   kind of data can be part of a free system distribution, even though
   its license does not qualify as free, because it is non-functional."


SCUMMVM, among other things, interprets the bytecode of the games (see: VM).

A long time ago, I looked at the Debian package of one of the games, and 
there appeared to be only a single 'game' file IIRC, presumably this 
includes the bytecode.


Bytecode is code and hence functional, if 'functional' is interpreted in 
the narrow sense of 'it does a practical job'.


As such, I do not think this falls under 'non-functional data'

---

That paragraph also appears inconsistent to me. The world map, game 
graphics, sounds ... are one of the most important components of the 
game. If someone wants to modify the game, I consider it more likely 
they have to modify the world map and maybe add some graphics and sounds 
than that they have to change the engine. Seems pretty 'functional' to 
me. It also does a practical job: entertaining the user.


As such, it appears to me that if the ‘meh, it's non-functional data’ is 
non-free, then the game is effectively non-free  software does not 
just consist of code, the non-code parts are sometimes as important as 
the code or more important -- they belong together, as a whole.


Myself, being able to code but not good at art, I would rather have a 
non-free game engine with free non-functional data than the free game 
engine with non-free non-functional data that the FSDG refers to, at 
least I would with a sufficient amount of work (*) be able to replace 
the non-free engine, but don't ask me to replace the artwork ...


(*) and some assistance, depending on the size and complexity.

---

Another thing I would like to note is that, even if it were 
non-functional data, according to (guix)Software Freedom everything in 
Guix is free software:



The GNU operating system has been developed so that users can have
freedom in their computing.  GNU is “free software”, meaning that users
have the four essential freedoms
(https://www.gnu.org/philosophy/free-sw.html): to run the program, to
study and change the program in source code form, to redistribute exact
copies, and to distribute modified versions.  Packages found in the GNU
distribution provide only software that conveys these four freedoms.
(I'm interpreting 'GNU operating system' and 'GNU distribition' as 
'Guix' here.)


That paragraph, and the web page referred to there does not make an 
exception for non-functional data -- if it's software, the 4 freedoms 
should apply, this is usually code but the freedoms and the reasons for 
them apply to software in general, not for code in specific.


To me, it appears that the SCUMMVM games cannot be in Guix, because of 
that rule.


It is, however, contradicted by the following paragraph, which is also a 
bit misleading:



   In addition, the GNU distribution follow the free software
distribution guidelines
(https://www.gnu.org/distros/free-system-distribution-guidelines.html).
Among other things, these guidelines reject non-free firmware,
recommendations of non-free software, and discuss ways to deal with
trademarks and patents.
I consider it contradictory in the sense that it adds exceptions to the 
'software must be free' claimed by the the previous paragraph, without 
being explicit that there are exceptions (see: non-functional data). I 
consider it misleading in the sense that the phrasing implies it just 
adds a bit of rules (on top of the 4 freedoms thing) and advise on 
potential legal problems, even though it also carves out a few 
exceptions (see: non-functional data).



You cannot sell the game itself, but you can charge "a reasonable
copying fee" and distribute it commercially... while slightly confusing
and seemingly contradictory at a passing glance, those two clauses alone
do not appear to violate any of the four freedoms to me:

   https://www.gnu.org/philosophy/free-sw.html#four-freedoms
While initially I thought of it as 'a no-selling clause -> 
non-commercial only -> non-free', after your explanation I agree -- it 
does not appear to have the potential problems referred to in 'Free 
software can be commercial' and there is no explicit 'The freedom to 
sell software.'.



I'm not really sure you have the right to "sell" most software in GNU
Guix, but you're free to distribute it and even charge for the
distribution of it, and use it in products that you sell to customers.

Most licenses do not give you ownership of the software; they roughly
give you permission to use, study, modify, and share it under the terms
of that license. If you do not own it, I am not 

Re: guix lint should support overrides

[Maxime Devos]

On 24-08-2022 10:08, zimoun wrote:

Hi Vagrant,

On Tue, 23 Aug 2022 at 15:22, Vagrant Cascadian  wrote:


But, because there is no way to silence a particular inappropriate
suggestion from guix lint, it becomes noise, and each person evaluating
the results of the package in the future then needs to take time to
figure out if guix lint is wrong, or something should be changed.

Do you have some packages as example?  In order to be concrete about the
false-positive and how to programatically fix them.

For instance, do you mean exclude on specific checker for one specific
package?  Or teach one specific checker for one specific package in
order to avoid an error specific to this package running this specific
checker?


Myself (not Vagrant) I was thinking of the gnu-description linter.

IIRC, there was some package where I proposed to modify the description 
a little to be more informative and fit better in Guix, but then the 
gnu-description proposed to use the upstream description. Consequently, 
it was decided to use the original, IMO worse, description.


Unfortunately I cannot find the relevant e-mails anymore.

This was a true positive, not a false positive, but I think it would 
have been useful to silence the linter there anyway.


At least for these kind of cases, I would go for a package property 
(properties '((silence-linters gnu-description))).


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: secure boot

[Maxime Devos]

On 24-08-2022 05:07, Philip McGrath wrote:

I could imagine a process like this:

  1. Build the binary that needs to be signed.
  2. Outside of the Guix build environment, create a detached signature
 for the binary using your secret key.
  3. Add the detached signature to the Guix store, perhaps with 'local-file'.
  4. Use Guix to attach the signature to the built binary.
  5. Use the signed binary in your operating-system configuration.


To implement this, you could have a look at "dynamic dependencies" in 
guix/store.scm and guix/graftsscm.


From the with-build-handler docstring:

Build handlers are useful to announce a build plan with 
'show-what-to-build'
and to implement dry runs (by not invoking CONTINUE) in a way that 
gracefully
deals with \"dynamic dependencies\" such as grafts---derivations that 
depend

on the build output of a previous derivation."


On grafts: the derivation of the grafted version depend on what the 
references of the store item used to be, this can only be decided 
outside the store (kind of similar to this situation).


Greeetings,
Maxime



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: (ice-9 base64)?

[Maxime Devos]

On 19-08-2022 02:20, Aleix Conchillo Flaqué wrote:

So, what do you think would be the way to proceed in order to include 
a base64 implementation in Guile itself?


For example:

1. Add (ice-9 base64) (or (encoding base64)) to Guile and let new 
projects and existing projects to update with conditional module 
loading to support old versions of Guile.
2. Do unbundling in Guix packages both for projects that have not 
updated upstream and for projects in (1). The unbundling would be done 
by pointing to Guix's (or guile-gcrypt) base64 implementation, or is 
there a way they could point to Guile's implementation?


If the canonical location of base64 becomes Guile itself instead of 
guile-gcrypt, then it needs to be pointed at Guile's base64. Likewise, 
Guix' base64 implementation would need to be replaced by Guile's, with a 
fallback.


I don't see why we would point to Guix' implementation, it's missing 
some bug fixes.




Does that make sense or am I still missing something (I'm about to 
catch a cold so my brain is not working quite well this week)? 
Originally, I was thinking only in (1).


Except for the remark about (1), I think so.  I think the following list 
is a bit more clear though:


1. Add (ice-9 base64) to Guile (or another name (encoding base64)).
2. Inform a few upstreams that used to include a copy of base64 that it
   is now part of Guile itself -- those upstreams can then remove their
   local copy and use Guile's base64, and do conditional module loading
   if they cannot increase their minimal Guile version yet.
3. In Guix, we will have to update Guile to a new version that has the
   base64 module and remove the local fallback copies.  And if upstream
   refuses patches to use Guile's base64 (maybe with a fallback), then
   it will need to be patched locally in Guix.

On (2): I don't think it's necessary to contact _all_ the upstreams, 
though to give a good example it would be nice to contact some of them.


(3) is a Guix concern, not really a Guile concern.

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: (ice-9 base64)?

[Maxime Devos]

Then, if I understood correctly, IMO I would say Guile should not 
really care about Guix's bundling/unbundling. That is, adding (ice-9 
base64) (or however we want to call it... maybe (encoding base64) 
following Golang and Guile's (web ) module) should be totally 
independent of Guix. So, if we add (ice-9 base64) to Guile then Guix 
should figure out what to do with it, but it's Guix's concern not Guile's.


It's not some Guix-specific quirk. It's the same for at least Debian. It 
benefits not only Guix itself but all users of the software:



[...] allows [...] to make transverse changes
such as applying security updates for a given software package in a
single place and have them affect the whole system—something that
bundled copies prevent.
... that was written with Guix in mind, but it applies to every 
distribution and everyone.


Besides, your goal appears to be to unbundle the base64 into a single 
location (as a module of Guile), if we do that I think we should go all 
the way -- just adding it to Guile increases bundling instead of 
decreasing bundling, only if the various upstreams are modified to 
unbundle and use the new location then the unbundling is completed.


On 18-08-2022 02:09, Aleix Conchillo Flaqué wrote:
About Guix's unbundling (maybe that's something that should go on 
Guix's mailing list),
I don't see why, there's nothing to write about except "oops some 
packages are bundling base64, let's unbundle those", and for unbundling 
those, it seems more practical to write about that here on guile-devel. 
Also I noticed I sent some messages to guix-devel instead of 
guile-devel, correcting now.
I don't think currently there's any unbundling for base64 modules or 
at least not in a package I maintain guile-jwt (guile-jwt bundles 
base64). And probably there's no unbundling because there's no 
canonical implementation? Even if there was a canonical 
implementation, how would that look like in Guix's guile-jwt package? 
What would the snippet actually do?
Currently, it's not done yet, presumably for that reason and maybe also 
due to nobody having noticed it yet?


How it would look like, for upstreams that refuse to unbundle or are 
unresponsive:


#~(begin
    (delete-file "local/copy/of/base64.scm")
    [also remove it from the Makefile.am]
    (substitute* (find-files "." "\\.scm$")
  (("(\\local base64 module\\)") "(gcrypt base64)")))

For responsive upstreams that do not mind these kind of improvements, 
there is a preference for submitting a patch upstream -- that way, 
everyone benefits, not only Guix.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: (ice-9 base64)?

[Maxime Devos]

On 17-08-2022 18:22, Aleix Conchillo Flaqué wrote:

Hi Maxime!

On Tue, Aug 16, 2022 at 12:04 PM Maxime Devos  
wrote:



On 16-08-2022 19:21, Aleix Conchillo Flaqué wrote:



On Tue, Aug 16, 2022 at 9:59 AM Maxime Devos
 wrote:


On 16-08-2022 18:10, Aleix Conchillo Flaqué wrote:

Hi,

In many projects I've been copying Göran Weinholt's base64
implementation and I've also seen it in other projects,
would it make sense to include it in Guile's standard
library? [...]


If we do this, we should contact the various other projects
to make them use (ice-9 base64).


I think they could switch whenever they want (i.e. whenever this
was added to Guile) or even not switch at all.


Sure, but they can't switch if they don't know about it. And if
they don't know about it and hence don't switch, the proposal
fails at its purpose of unbundling base64. Besides, we need them
to switch (see Guix no-bundling policy and the reasons behind it)
-- if upstream refuses to unbundle, then in our locally modified
version for Guix.


Forgive my ignorance, but what do you mean by unbundling? I'm not 
familiar with Guix at all, well, just conceptually and for trying a 
few commands years ago.


Sometimes the source code of a package contains a copy of a dependency. 
This is called 'bundling'. 'Unbundling' is the act of undoing the 
'bundling', this is often done by cleaning up the source code (with what 
we call a 'snippet' in Guix: (snippet #~(delete-file-recursively 
"googletest"))) and setting some configuration flags 
("-DUSE_SYSTEM_GOOGLETEST=yes" or such).


For example, in Guix we occasionally encounter a bundled "googletest" (a 
test framework).


In this case, we are kind of (un)bundling the base64 module, though it's 
not _exactly_ (un)bundling because, AFAIK, there is canonical upstream 
location for the base64 module to replace things with. Still seems 
pretty close to me.


Upsides of unbundling, as mentioned in '(guix)Submitting Patches':


 Sometimes, packages include copies of the source code of their
 dependencies as a convenience for users.  However, as a
 distribution, we want to make sure that such packages end up using
 the copy we already have in the distribution, if there is one.
 This improves resource usage (the dependency is built and stored
 only once), and allows the distribution to make transverse changes
 such as applying security updates for a given software package in a
 single place and have them affect the whole system—something that
 bundled copies prevent.
Another benefit: reviewing for absence of malware is less work when 
there's only a single copy to review, though I suppose that in this case 
the module is so small the reviewing benefit is minimal.



Whether we simply replace (guix base64) by (gcrypt base64)
depends on how old (gcrypt base64) is compared to the
earliest 'supported' Guix for pull/time-travel, but even if
it is not present in the old gcrypt, we can work-around that
(we have a 'fake-gcrypt-hash' in build-aux/build-self.scm, so
we can easily have a (define gcrypt-base64 [some copy])). Or
simply update the local guile-gcrypt in buid-aux/build-self.scm.

guile-gcrypt base64 is pretty new with the patch above (but no
release after that), I have no idea if Guix has added anything else.


base64 is available in at least 0.3.0, which is packaged in Debian
bullseye (which is considered "stable"), so not too new, though we
might need to change build-aux/build-self.scm if 0.1.0 doesn't
have base64.  Guix appears to have the pre-quoted-patch version,
without changes of its own except for a different module name.


One more time, forgive me, but what is build-aux/build-self.scm?
It's an implementation detail of Guix, it's a file (from the new 
version, not the old) that is loaded by "guix pull" in the old Guix to 
compile the new version of Guix.



OTOH a similar replacement can be done for (ice-9 base64),
but transitioning to (ice-9 base64) would take much longer,
at least until the various distributions are updated to a
Guile that has (ice-9 base64), whereas (gcrypt base64) could
be switched to immediately.

Maybe this could be handled by each project independently.


They wouldn't have to if the base64 module is put in (guile gcrypt).


And the last forgiveness... (guile gcrypt)?


Oops, that should have been guile-gcrypt -- it's a Guile package -- 
"guix show guile-gcrypt" / <https://notabug.org/cwebber/guile-gcrypt>.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: (ice-9 base64)?

[Maxime Devos]

On 16-08-2022 19:21, Aleix Conchillo Flaqué wrote:



On Tue, Aug 16, 2022 at 9:59 AM Maxime Devos  
wrote:



On 16-08-2022 18:10, Aleix Conchillo Flaqué wrote:

Hi,

In many projects I've been copying Göran Weinholt's base64
implementation and I've also seen it in other projects, would it
make sense to include it in Guile's standard library? [...]


If we do this, we should contact the various other projects to
make them use (ice-9 base64).


I think they could switch whenever they want (i.e. whenever this was 
added to Guile) or even not switch at all.


Sure, but they can't switch if they don't know about it. And if they 
don't know about it and hence don't switch, the proposal fails at its 
purpose of unbundling base64. Besides, we need them to switch (see Guix 
no-bundling policy and the reasons behind it) -- if upstream refuses to 
unbundle, then in our locally modified version for Guix.



I think it would be simpler though to consider the base64 in
guile-gcrypt to be 'canonical', it would avoid problems with old
versions of Guile not having the base64 module and newer version
having it, which would prevent using the proposed (ice-9 base64)
in Guile because it would break build-aux/build-self.scm when
pulling or time-machining from old Guix that have an old Guile.


I've been waiting on a guile-gcrypt release for a while now (Ludo, 
Chrisitine... any help here? :-) ).  I ported guile-jwt to use 
guile-gcrypt but I need a release to have latest base64 changes:


https://notabug.org/cwebber/guile-gcrypt/commit/f8934ec94df5868ee8baf1fb0f8ed0f24e7e91eb

Right, it has some fixes that are presumably important.


But you are right that this would cause a backward compatible problem, 
but I guess that would depend on each project. Can we do conditional 
module loading? I've done this in the past with Python... if we are in 
Python 2 load this module, otherwise load this other one. So projects 
could do that.
Yes, using resolve-module with #:ensure #f & module-ref. Or with syntax 
tricks and (version), to decide things at compile-time. Still, if you do 
a conditional module loading, you still need a fallback, and the 
fallback would still be bundling.


Whether we simply replace (guix base64) by (gcrypt base64) depends
on how old (gcrypt base64) is compared to the earliest 'supported'
Guix for pull/time-travel, but even if it is not present in the
old gcrypt, we can work-around that (we have a 'fake-gcrypt-hash'
in build-aux/build-self.scm, so we can easily have a (define
gcrypt-base64 [some copy])).  Or simply update the local
guile-gcrypt in buid-aux/build-self.scm.

guile-gcrypt base64 is pretty new with the patch above (but no release 
after that), I have no idea if Guix has added anything else.


base64 is available in at least 0.3.0, which is packaged in Debian 
bullseye (which is considered "stable"), so not too new, though we might 
need to change build-aux/build-self.scm if 0.1.0 doesn't have base64.  
Guix appears to have the pre-quoted-patch version, without changes of 
its own except for a different module name.



OTOH a similar replacement can be done for (ice-9 base64), but
transitioning to (ice-9 base64) would take much longer, at least
until the various distributions are updated to a Guile that has
(ice-9 base64), whereas (gcrypt base64) could be switched to
immediately.

Maybe this could be handled by each project independently.


They wouldn't have to if the base64 module is put in (guile gcrypt).

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Who owns guix-mirror?

[Maxime Devos]

On 16-08-2022 14:59, Philip Beadling wrote:

Hi all,

http://github.com/guix-mirror is a mirror for the Guix source code.

I was wondering who owns the mirror as there is no public ownership 
declared on github.com?


Is it owned by Guix or is it run by a trusted third party, etc?


It's unknown who runs it, but it's not Guix, see  
.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: bug#55026: potential prebuilt binaries in the Mono package

[Maxime Devos]

On 12-08-2022 22:03, Maxim Cournoyer wrote:

Hi,

zamfofex  writes:


It seems the package for Mono in Guix uses a tarball that contains a
lot of prebuilt DLLs. This doesn’t seem to have been mentioned when
the package was introduced. Could it have been a mistake? Some related
discussion:  and also


If nobody volunteers to do the work necessary to get our mono package to
build without prebuilt binaries in the next 2 weeks, I suggest we remove
it from our collection.  Only a handful package depend on it, most of
them optionally it seems, and the mono we carry is severely outdated
(2016).

What do you think?

Maxim

I hope someone does the work, but if not, I don't see a problem with 
removing it.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [POSTMORTEM] Subkey is not authorized by .guix-authorizations

[Maxime Devos]

On 11-08-2022 16:26, Andrew Tropin wrote:

* What to do after?
- Accept subkey on guix pull if master key is in .guix-authorizations.


As I've now written on 57091, this would cause security problems with 
old or revoked keys.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 10-08-2022 12:33, bl...@reproduciblemedia.com wrote:


None of the edits I made were in criticism. In an academic context a proof 
reader would mark
almost all the same sentences for revision.


As far as I'm aware, I do not have implied the contrary anywhere previously.

I do think they are criticism though -- more precisely, constructive 
criticism. There might be a difference in terminology here, e.g. 
according to Wikipedia and its source, some languages make a distinction 
between critique and criticism


To be clear, I consider your criticism to be a good thing, I just happen 
to disagree on some points.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 10-08-2022 08:10, bl...@reproduciblemedia.com wrote:

August 5, 2022 1:59 PM, "Maxime Devos"  wrote:
Technical grammatical correction: the software that Guix "has" is that in the 
monorepo,
but it "distributes" many packages. Thus:
--8<---cut here---start->8---
* In principle, Guix only distributes free software; when the upstream source 
contains some
non-free software, it should be removed such that ‘guix build --source’ returns the 
"freed"
source code rather than the unmodified upstream source (see: 28.4.1 Software 
Freedom).
--8<---cut here---end--->8---

I consider the difference between referring to external source code by 
including a
(snippet-sanitised) copy or downloading it from an URL + snippet-sanitising to 
be immaterial,
except for space and I/O savings, so I consider "has" to include "distributes".

While "distributes" is more specific, I really meant "has" here -- 
gnu/packages/patches/... and
gnu/packages/*.scm must be free too, even if it is was not a distributed 
package (more concretely,
see the bits about patches failing to remove non-freeness).

[...]

This is simply a grammatical error. "Has" is third person singular. While its 
common to speak
in such a way, it's not proper English, and including minor slang should be 
avoided in technical
writing. Otherwise inconsistency of presentation is guaranteed, causing serious 
overhead for
readers.


"Has" is third person singular, and "Guix" is singular and not "I" or 
"you", so "has" seems appropriate here to me.


I think that matching the singular/plural of the verb and the subject is 
common, standard and proper English and not slang.


Going by previous replies, you seem to know English well, so I think 
there's some miscommunication here.


(Even better would be to replace the rather generic ‘to have’ by 
something more specific, but I'll have to think a bit about what would 
be more specific yet not overly specific.)





* The source of the package needs to correspond to what is actually built 
(i.e., act as the
corresponding source), to fulfill our ethical and legal obligations.

The [i.e.] addendum above is redundant, its better worded as:
--8<---cut here---start->8---
* The source of a package must correspond to what is actually built (i.e., 
there must be
an explicit relation between source code and the result of its build for all 
builds),
to fulfill our ethical and legal obligations.
--8<---cut here---end--->8---

You write that the addendum is redundant, but then change the addendum by 
replacing a word in the
addendum by a possible definition. I'm not following how that reduces 
redundancy, and it also
appears to be contrary to the lack of verbosity that Andreas would like.

Redundancy in language is information expressed more than once. Including 
redundant clauses
is bad grammar[1]

You wrote:

The source of the package needs to correspond to what is actually built (i.e., 
act as the
corresponding source)

You simply said the same thing twice. It is by definition, a redundant clause.


then change the addendum by replacing a word in the
addendum by a possible definition.

Elaboration doesnt necessarily add redunancy, it is useful for clarifying 
statements. I was
trying to infer your intention in adding the clause, to offer an example of how 
it could be
more clearly stated.

[1] https://en.wikipedia.org/wiki/Redundancy_(linguistics)


The purpose of 'i.e.' constructs is to state the same thing differently, 
to clarify matters (by elaboration or by redundancy). I don't see how 
redundancy is bad, though it can easily be overdone.


I think that explicitly mentioning the term 'corresponding source' 
instead of only the more implicit 'source of X corresponds to Y', 
because the former 'corresponding source' has a very specific meaning 
(*) in free software, whereas the latter construct is more ambiguous.


Compare with your definition 'there must be an explicit relation ...’, 
which loses a lot of nuance.


(*) E.g., the GPL has a long and detailed definition: ‘The 
"Corresponding Source" for a work in object code form means all the 
source code needed to generate, install, [lots and lots of text]’.


I can look into inserting a footnote linking to the GPL or copyleft.org 
or such, to make clear "corresponding source" is a term on its own and 
not just some description, for people that don't already know about 
"corresponding source".





To make things more concrete and to resolve conflicts between the principles, a 
few cases have been
worked out:

To a newcomer (the target audience), the above may lead to confusion as to what 
wasn't already
concrete in the above descriptions, or 

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 09-08-2022 20:58, david larsson wrote:

On 2022-08-05 15:59, Maxime Devos wrote:

[..]



20.4.5.3 Fixing technical issues (compilation errors, test failures,
other bugs ...)

Usually, a bug fix comes in the form of a patch copied from upstream
or another distribution. In that case, simply adding the patch to the
'patches' field is the most convenient and usually does not cause any
problems; there is no need to rewrite it as a snippet or a phase.

If no ready-made patch already exists, then choosing between a patch
or a snippet is a matter of convenience. However, there are two things
to keep in mind:

First, when the fix is not Guix-specific, it is strongly desired to
upstream the fix to avoid the additional maintenance cost to Guix. As
upstreams cannot accept a snippet, writing a patch can be a more
efficient use of time. Secondly, if the fix of a technical issue
embeds a store file name, then it has to be a phase. Otherwise, if a
store file name was embedded in the source, the result of 'guix build
--source' would be unusable on non-Guix systems and likely also
unusable on Guix systems of another architecture.


There may be other reasons to add patches: [...]


Agreed (*), but I don't think that subsection claims those are the only 
reasons for patches -- that section is only about fixing technical 
issues, not adding new features, as implied by the name of the section.


I can look at adding a new subsection 'Adding new functionality' for a v3.

Liliana's documentation contains some information not in my v2, I intend 
to look into integrating that information as well.




1. Functionality, that is not yet accepted upstream, because 
maintainer(s) do not have enough time to review all pull requests, or 
are simply slow to review. "if no response within X time from 
upstream, then guix may include your patch" might be a good policy here.


(*) We sometimes do such things already. Example: 
<https://issues.guix.gnu.org/49828> (nowadays upstreamed). I don't think 
this thread is a good place for deciding on the exact rules though -- 
deciding on an appropriate value of X seems difficult, I would like to 
separate that from the current documentation patch.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

On 09-08-2022 21:08, Liliana Marie Prikler wrote:

To be clear, do you mean you:
  * think it's not better, maybe even worse
  * think it's not_much_  better (but still_slightly_  better)
  * are undecided
  * or something else
?
Also, "guix build -S" returns the source code (after snippet / patch,
if any), not its derivation. For the latter: "guix build -S -d"

FWIW I don't think mentioning patch-and-repack is too helpful here
either.  Also, I'd like to use consistent wording at least within this
section, so here "source" means "upstream source" whereas "source
derivation" is a shorthand for the stuff Guix builds.  Yes, the
derivation is not the same thing as the output, but I again fail to see
how being overly precise is helpful.  That being said, I'm open to
suggestions.


I am not reading an answer to my question.

I don't think I've mentioned patch-and-repack (at least not by name, 
which you seem to be referring to?).


I would not recommend "source = upstream source", as the more general 
meaning is used in (guix)Introduction and 
 and elsewhere, otherwise 
terminology would become inconsistent, which can lead to misinterpretations.


I don't think there's such a thing as 'overly precise'.

My suggestion is the same as your suggestion:


You could s/source derivation/the result of
@code{guix build -S}/, but I don't think that's much better.

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

On 09-08-2022 21:08, Liliana Marie Prikler wrote:

On 06-08-2022 08:55, Liliana Marie Prikler wrote:



+If your package has a bug that takes multiple lines to fix,

I don't think this is true for replacing all instances of "foo"
by
"/gnu/store/.../bin/foo" in a file.

Should it?

I don't think so. Directly substituting all the instances instead of
first writing a patch that does "foo" -> "@foo@" or such seems
simpler to me.  This might be a bit too nit-picky though, maybe it's
clear from context that this is not the kind of fix meant by that
line.

I'm struggling to see the issue here.  For starters, it doesn't take
multiple lines to embed a store path, the change usually happens on a
single line.  Of course, translating this into a substitute*, you
expand this single line into multiple ones, but that's not what is
meant here.


I was thinking of files that contain multiple instances of "foo" 
(usually on multiple lines) to be replaced by "/gnu/store/.../bin/foo", 
not files were a "foo
bar" was broken over multiple lines and it needs to be replaced by 
"/gnu/store/.../bin/foo

bar".

At least for the former, I don't think an intermediate "@foo@" is useful 
(with some exceptions, when a pattern match would catch too much).


The latter could in principle happen, but it doesn't seem to happen in 
practice.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

Am Dienstag, dem 09.08.2022 um 18:45 +0200 schrieb Maxime Devos:

On 06-08-2022 08:55, Liliana Marie Prikler wrote:


+If your package has a bug that takes multiple lines to fix,

I don't think this is true for replacing all instances of "foo" by
"/gnu/store/.../bin/foo" in a file.

Should it?
I don't think so. Directly substituting all the instances instead of 
first writing a patch that does "foo" -> "@foo@" or such seems simpler 
to me.  This might be a bit too nit-picky though, maybe it's clear from 
context that this is not the kind of fix meant by that line.

+ Furthermore, as with patches, modifying the snippets causes two
derivations to be built.

This is true, but I don't think reviewers and package authors have to
worry about that.

It does make a difference to the author when debugging their package.
Starting with a phase and then moving it to a snippet can save good
time.
Hm, maybe, I guess I often work on 'small' packages where it doesn't 
matter much.


On 09-08-2022 19:05, Liliana Marie Prikler wrote:

+Such changes include, but are not limited to, fixes of the build
+script(s) or embeddings of store paths (e.g. [...])


[...]

Is that how to English comma?  Sorry, I'm not a native speaker so I get
somewhat weirded out by the when to skip/not to skip rules.

Neither am I. English doesn't seem to do "rules" much. I do think, 
however, that adding a comma after "to" makes things a bit simpler to 
read here, and it doesn't appear to be ungrammatical -- at least, in 
licenses "but is/are not limited to" is often used that way.



Derivations are a rather low-level concept, could they be avoided in
the origin and phases documentation?
I don't quite see how.  You could s/source derivation/the result of
@code{guix build -S}/, but I don't think that's much better.


To be clear, do you mean you:

 * think it's not better, maybe even worse
 * think it's not _much_ better (but still _slightly_ better)
 * are undecided
 * or something else

?

Also, "guix build -S" returns the source code (after snippet / patch, if 
any), not its derivation. For the latter: "guix build -S -d"



+Build phases are limited in that they do not modify the source
+derivation.  Thus, they are inadequate for changes that are to be
+reflected in the source code.  On the other hand, they only cause
a
+single rebuild and are thus slightly easier to debug than phases
and
+snippets.

See Andreas' comment on phase->snippet.

Also, do I understand correctly that the argument here is that
'single rebuild -> less compilation time -> easier to debug'?

Easier to debug for the package author currently fiddling with the
phase/snippet.  Not really a statement in any direction otherwise.
I don't see how "slightly easier to debug than phases" follows from 
"they cause only a single rebuild". My guess was that the intermediate 
step was lower compilation time, but apparently this was not the 
argument. As such, I'm not following.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

On 06-08-2022 08:55, Liliana Marie Prikler wrote:


+If your package has a bug that takes multiple lines to fix,
I don't think this is true for replacing all instances of "foo" by 
"/gnu/store/.../bin/foo" in a file.

  or a fix
+has already been accepted upstream, patches are the preferred way of
+eliminating said bug
+Refer to the @code{origin} record documentation
+(particularly the fields @code{snippet} and @code{modules}), for more
+information (@pxref{origin Reference}).
+


The "Refer to the ... documentation for more information" occurred in 
the old version of (guix)Snippets versus Phases. However, back then, I 
did not find more information on how to decide between snippets, patches 
and phases, and neither do I now.


Maybe:

+Refer to the @code{origin} record documentation
+(@pxref{origin Reference}) (particularly the fields @code{snippet} and 
@code{modules})
+for more information on how to use snippets

, to avoid a reader's assumption that that section contains information 
on deciding between snippets, phases and patches.



+ Furthermore, as with patches, modifying the snippets causes two derivations 
to be built.


This is true, but I don't think reviewers and package authors have to 
worry about that.



Such changes include, but are not limited to fixes of the
+build script(s) or embeddings of store paths (e.g. replacement of
+@file{/bin/sh} with @code{(search-input-file inputs "bin/sh")}).

Include what? I think you need to close the subsentence here:


+Such changes include, but are not limited to, fixes of the build
+script(s) or embeddings of store paths (e.g. [...])



[...]


+Build phases are limited in that they do not modify the source
+derivation.  Thus, they are inadequate for changes that are to be
+reflected in the source code.  On the other hand, they only cause a
+single rebuild and are thus slightly easier to debug than phases and
+snippets.
Derivations are a rather low-level concept, could they be avoided in the 
origin and phases documentation?



+Build phases are limited in that they do not modify the source
+derivation.  Thus, they are inadequate for changes that are to be
+reflected in the source code.  On the other hand, they only cause a
+single rebuild and are thus slightly easier to debug than phases and
+snippets.

See Andreas' comment on phase->snippet.

Also, do I understand correctly that the argument here is that 'single 
rebuild -> less compilation time -> easier to debug'?


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 05-08-2022 18:59, bl...@reproduciblemedia.com wrote:

Hi Maxime,

Adding some basic grammatical corrections below:
I have read several proposed corrections, but most of them don't appear 
to be grammatical corrections but rather stylistic or about 
(non-grammar) clarity or contents.

August 5, 2022 1:59 PM, "Maxime Devos"  wrote:


Technical grammatical correction: the software that Guix "has" is that in the 
monorepo,
but it "distributes" many packages. Thus:
--8<---cut here---start->8---
* In principle, Guix only distributes free software; when the upstream source 
contains some
non-free software, it should be removed such that ‘guix build --source’ returns the 
"freed"
source code rather than the unmodified upstream source (see: 28.4.1 Software 
Freedom).
--8<---cut here---end--->8---

I consider the difference between referring to external source code by 
including a (snippet-sanitised) copy or downloading it from an URL + 
snippet-sanitising to be immaterial, except for space and I/O savings, 
so I consider "has" to include "distributes".


While "distributes" is more specific, I really meant "has" here -- 
gnu/packages/patches/... and gnu/packages/*.scm must be free too, even 
if it is was not a distributed package (more concretely, see the bits 
about patches failing to remove non-freeness).



[...]
* The source of the package needs to correspond to what is actually built 
(i.e., act as the
corresponding source), to fulfill our ethical and legal obligations.

The [i.e.] addendum above is redundant, its better worded as:
--8<---cut here---start->8---
* The source of a package must correspond to what is actually built (i.e., 
there must be
an explicit relation between source code and the result of its build for all 
builds),
to fulfill our ethical and legal obligations.
--8<---cut here---end--->8---


You write that the addendum is redundant, but then change the addendum 
by replacing a word in the addendum by a possible definition. I'm not 
following how that reduces redundancy, and it also appears to be 
contrary to the lack of verbosity that Andreas would like.



* It is convenient for the source derived from an origin to build on any system 
that the upstream
package supports.
* The source needs to actually work, not only on your Guix system but also for 
other systems; this
requires some care for substitutions involving store items and other 
architecture-specific changes.

* Sometimes, there is more than one way to do it. Let's go for the simplest 
one. Sometimes, which
tool is the simplest, is subjective, that's fine too.

I think would be more clearly worded as:
--8<---cut here---start->8---
* When presented with a variety of strategies for defining a package, choose 
whichever is simplest.
Sometimes this is subjective, which is also fine. What matters is that you 
prefer techniques that
are common within the community (i.e. patterns that appear throughout 
gnu/packages/...) and
are thus clearly legible for reviewers.
--8<---cut here---end--->8---


To be clear, this is to replace the third point (Sometimes, there's more 
than one way to do it, etc.), without removing the previous two?


I would not use combinations like 'presented with a variety of 
strategies' however, I don't think that leads to clarity. Also, the 
preferences of 'you' are not all that important here, it's what they 
choose for that's important even if it is not their preference. (Though 
the two coinciding would be ideal of course!) Maybe:


When there is more than one way to do something, choose whichever method is the 
simplest.
Sometimes this is subjective, which is also fine. What matters is that you use 
techniques that
are common within the community (i.e. patterns that appear throughout 
gnu/packages/...) and
are thus clearly legible for reviewers.


To make things more concrete and to resolve conflicts between the principles, a 
few cases have been
worked out:

To a newcomer (the target audience), the above may lead to confusion as to what 
wasn't already
concrete in the above descriptions, or what principles above come into 
conflict. There is a mild,
latent assumption that they are familiar with the Guix workflow, which should 
be avoided. Thus I
suggest:
--8<---cut here---start->8---
For the purpose of clarifying preferred practices and reducing friction in the 
review process
introduced by subjective variation, a few guidelines have been worked out:
--8<---cut here---end--->8---


I don't see how a fancy wording amounting to essentially the same thing 
would reduce confusion or avoid

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 08-08-2022 23:51, Andreas Enge wrote:

Hello,

Am Fri, Aug 05, 2022 at 03:59:14PM +0200 schrieb Maxime Devos:

Here's a v2. I've changed the structure to something close to what Julien
proposed, it looks a lot better now to me!

thanks, it does! I still find it a bit too verbose compared to Liliana's
suggestion, which I would prefer as a starting point of the discussion.


WDYM with 'still' here? The v2 patch I sent preceded Liliana's patch.

I'm not seeing a 'too verbose'-ity or a difference in verbosity myself

---

Something I liked about Julien's proposed structure is:


[...] derive rules for specific cases, based on these principles:

How do I remove non-free software? -> snippet because …

How do I remove bundled libraries? -> snippet or phase because …

How do I fix a build issue? -> patch or snippet if this affects 
building from source, can also be a phase if the result of --sources 
can still build


A test issue?

…

This leaves some cases up to interpretation, but that's probably not 
so different from "it's not an absolute rule". It's also much clearer 
and quicker to figure out in which case you are. If not documented as 
a case, you can fall back to the general principles.
-- i.e., if I want to do $FOO, I could quickly find out how to do it.  
In the v2 I sent, this was reflected in the subsections, each subsection 
is a 'howto $FOO'. Whereas the patch Liliana sent is kind of the inverse 
-- each @item corresponds to a 'what can the method $FOO be used for'. 
Similarly, in the v1 I sent, I followed a similar structure (an item for 
patches, an item for snippets, an item for phases).


As such, the v2 I sent seems a better basis to me.

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Translating news on weblate?

[Maxime Devos]

On 07-08-2022 12:56, pelzflorian (Florian Pelz) wrote:

I was thinking we could have our news file translated at weblate, which
would help having more people translate it. Attached is a script that[…]

Translating etc/news.scm via Weblate adds delay but reaches more
translators.  Now delay is not so important when reading old news from
--list-generations or when one hasn’t upgraded in a while, but I would
prefer if those who make news, continue to request translations on some
mailing list (I forgot the name of the list alias; where is it?).  Even
though I was too slow to react to such mail in the past, I think I can
improve.


We could have both, I think?  After a new news entry, directly update 
the .po in the Guix repo, then after the patch is sent to guix-patches@ 
other people can open their favourite .po editor and add translations 
for their language and send that. No need to wait for Weblate, though 
late contributors can still translate with Weblate.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: FSDG issues of SCUMMVM-based games

[Maxime Devos]

On 06-08-2022 06:59, Liliana Marie Prikler wrote:


2. The "sources" consist of binaries that are installed as-is.


The credits of ScummVM state that the source code is available for 
Drascula[0]


[0] https://docs.scummvm.org/en/latest/help/credits.html: Emilio de Paz 
Aragón from Alcachofa Soft for sharing the source code of Drascula: The 
Vampire Strikes Back with us and his generosity with freewaring the game.


Maybe that only covered the engine and not the game bytecode though ...

I don't think bytecode falls under the 'non-functional data' exception 
of the FSDG, because code (the sprites, backgrounds and sound are 
another matter).


As such, I agree they should not be distributed (unless the actual 
sources are found, but even then there is (1.)), as per the first 
sentence of ‘(guix)GNU Distribution’.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature