date:20230913

Hi,

Simon Tournier  writes:

> Hi,
>
> On Wed, 13 Sep 2023 at 21:14, Liliana Marie Prikler 
>  wrote:
>
>> I do wonder how the ChangeId would work in practice.  Since it's not
>> really assigned by the committer, it would have to be generated "on the
>> fly" and attached to the mail in between, which could result in all
>> kinds of nasty behaviour like unstable Ids or duplicated ones.  Also,
>> if we can automate this for ChangeIds, we could also automate this for
>> patch-sets – the last patch in the series just gets the Closes: tag
>> added by mumi.
>
> I think it would work using some pre-commit hook.  When one commits
> their change, this commit is run and it can pre-fill the commit
> message.  Well, that’s how I have understood the thread.

Yes; exactly like how it's done in Gerrit, if you've ever used that
(we'd reuse their hook).  It'd be enabled out-of-the-box so it'd be
transparent to users.

>> Furthermore, I'm not convinced that it would ease the issue of
>> forgotten bugs as you can't really apply them to the past.  So the
>> practical use is limited to the case where you intentionally cherry-
>> pick this or that commit from a series.  How we want to deal with that
>> case could be a discussion in its own right, and maybe ChangeIds really
>> trump the explicit tags proposed by Giovanni or myself here.  Whether
>> that justifies the cognitive overhead of juggling them around on every
>> submission remains to be shown or disproven.

I like the 'Closes: ' trailer idea; it's simple.  However, it'd need to
be something added locally, either the user typing it out (unlikely for
most contributors) or via some mumi wizardry (it's unlikely that all
users will use mumi), which means its usage (and value) would depend on
how motivated individuals are to learn these new tricks.

On the other hands, having Change-Ids added by a pre-commit hook
automatically would means the user doesn't need to do anything special
other than using git, and we could still infer useful information at any
time (in a server hook, or as a batch process).

For this reason, I think we could have both (why not?  Change-Ids by
themselves provide some value already -- traceability between our git
history and guix-patches).

-- 
Thanks,
Maxim

Re: How can we decrease the cognitive overhead for contributors?

2023-09-13 Thread Ekaitz Zarraga

Hi,

On Wednesday, September 13th, 2023 at 3:42 PM, Maxim Cournoyer 
 wrote:

> 
> There's no lock-in. You can use any tool you want. Most people hacking
> on Guix do so with Emacs and Geiser because these are currently the best
> tools (that I know of) to do the job; these are the tools many of us
> know and can easily recommend. If Visual Code (or editor X) was
> packaged in Guix and had great support for working with Guile, we could
> also mention it in our manual or in the cookbook.
> 
> Notice I use recommend rather than mandate; these are just
> recommendations that try to be helpful. If it's not helpful to you, you
> are free to select your own tool box and share how it works (via patches
> to the contributing section or a blog post for example).

Of course, but it's limiting because there's no other recommendation than that.
Everything is built around it with no sensibility to any other approach.

Many times I've been answered how to solve my problems using emacs and a big
part of this conversation evolved in that direction, proposing technological
solutions to human problems in emacs, and only for emacs, ignoring the fact
that many guix users are not emacs users and also that technology is not the
root issue of this.

If you have a hammer everything look like nails I guess, and we are
programmers, there's nothing shameful about that, but if we really want to be
welcoming we have to be more than just programmers. Maybe that's something not
all of us should do, but some. I don't know.

Further than that. I'll try to summarize my views on this because I think we
diverged a lot in that direction and it would be a shame to miss this
opportunity to be more welcoming.

## Boring stuff coming, you have been warned

The technology itself is not a real deal most of the times. Sometimes it can
be: should I recompile the project? How do I configure the development
environment? Why my setup has to be a little bit worse if I'm not working with
emacs? How do I contribute my patch with the email provider of my choice? Can I
just paste the patch in the body of the email? Maybe attach it? Which one is
better? One patch per email? Maybe I'm bothering people asking the same thing
over and over again?

Most of those are a matter of getting used to some things that might be hard at
the beginning. I'm sure Katherine, who started this thread is not really
limited in a technical level, as I am not. We might be limited in the amount of
time needed to get used to these things and we don't want to bother
maintainers and commiters with stupid errors we feel like we are committing
over and over again because we don't have a solid reference for some of these
things. Examples of missing pieces of information:

> If you can't use `git send-mail` for a reason you can do this and that and
> that instead.

> If you are not an emacs user you can try this and that. (*probably this is
> something we, non-emacs users, should write*)

Example of a good way to do it, (maybe it's missing a reference in the docs):

https://guix.gnu.org/en/blog/2021/the-big-change/

Political/management decisions often more complex to deal with. Changelog style
is, in my opinion, useless, but it's not hard to follow if there are good
guides on how to do it well. There are other changes in the way packages are
described that are hard to follow too. Just copying what other people did is
not enough, probably because we don't feel like we can control what we are
doing. It's some kind of a trial and error process that always ends up in the
hands of the commiter or maintainer that receives our patches.

My take is: *check what other packages/commits do* is not a real answer. The
same way reading some English is not enough to write English yourself
comfortably. We need guidance or clearly written stuff that is *easy* to find.
Most of these questions are answered in the mailing list, the same way I think
Liliana very well explained how Changelog commits have to be done, but it's not
written in Guix itself (there's a link in the manual) by Guix and *for* Guix.

I would prefer to use other kind of tooling. Sourcehut is interesting as it
provides a way to keep the same email based approach we use, but with some
extra goodies, but I'm also OK with what we currently have. So that's not a big
deal for me at this point. I can say I'm starting to get used.

If we had a *very clear* guide, that would be way better for contributors. Not
just about Guix, but about all those other things that happen around. And I
don't mean a copy/paste tuto, but something were decisions are explained with
the goals they have.

(As a note, the GNU Standard for Changelog commit messages has some examples
that are way more verbose than the ones we have in Guix. IMHO those have a lot
of sense, the commit messages we do in Guix most of the times don't.)

The final point I would like to insist on is the contribution experience. I
don't really know how the issue system works as a whole. We have

Re: to PR or not to PR, is /that/ the question?

Hi Giovanni,

Thank you for your detailed message.  Sorry, I have not read all the
details because I have been lost.  What do you want to explain?

Well, let me explain my understanding using my own words. :-)

When one reduces to the core of the workflows, the main difference
between “email workflow” and “PR workflow” is the way the submitter
sends the code to the reviewer.

 1. email workflow: the diff are sent by email.
 2. PR workflow: the diff are sent by the VCS itself.

Then, we can put a lot of glue around these workflows.  It is possible
to work in a centralized fashion or in a distributed fashion.  It is
possible to use a proprietary service for delegating some parts using a
button via a web-interface.  It is possible to have a service based on
free software that automatize some parts of the workflow.  Etc

By glue, it reads a range from mailing list to Github, or from Gitlab to
Sourcehut, or from personal custom scripts to tools as B4, public-inbox,
Gerrit, patchwork, etc.

Obviously, the PR workflow also depends on the features of the
underlining VCS when the email workflow not.  Bitbucket provided a PR
workflow for Mercurial.  And a PR workflow would be hard to implement
using Subversion or worse using CVS.

Note that the email workflow #1 might be centralized.  That’s what we
are doing when using one mailing list; there is only one unique location
for exchanging all pieces of code.  Or it might be decentralized, that’s
how the contributors send their contribution to some linux kernel
maintainers.

Note that the PR workflow #2 might be decentralized.  That’s more or
less how Linus Torvald’s “lieutenants” are doing for asking a merge in
the Linus’s mainline linux kernel.  Or it might centralized and
web-forges provide many examples. :-)

The requirement for #1 is just being able to send emails.  The
requirement for #2 is to have a public Git repository.

Both workflows have pros and cons.  And then these pros and cons also
depend on how the workflow is implemented.  And none is not much better
than the other.

The slight difference between the two is the state of the contribution
from the reviewer perspective:

. About the email workflow #1, the patches might not apply for many
reasons but because they are sent by email, the notification “hey give a
look to thing” and the “diff for thing“ are synchronized.

. About the PR workflow #2, the patches belong to a Git repository
so they are already applied.  However, between the notification “hey
give a look to thing” and the moment when the reviewer pulls the thing,
the Git repository might disappear, be replaced, history rewritten, etc.

Well, that’s my understanding of the difference between #1 and #2.  A
bit simplified but it captures my understanding of the main, I
guess. :-)

Cheers,
simon

Re: [workflow] Automatically close bug report when a patch is committed

Hi,

On Wed, 13 Sep 2023 at 21:14, Liliana Marie Prikler  
wrote:

> I do wonder how the ChangeId would work in practice.  Since it's not
> really assigned by the committer, it would have to be generated "on the
> fly" and attached to the mail in between, which could result in all
> kinds of nasty behaviour like unstable Ids or duplicated ones.  Also,
> if we can automate this for ChangeIds, we could also automate this for
> patch-sets – the last patch in the series just gets the Closes: tag
> added by mumi.

I think it would work using some pre-commit hook.  When one commits
their change, this commit is run and it can pre-fill the commit
message.  Well, that’s how I have understood the thread.


> Furthermore, I'm not convinced that it would ease the issue of
> forgotten bugs as you can't really apply them to the past.  So the
> practical use is limited to the case where you intentionally cherry-
> pick this or that commit from a series.  How we want to deal with that
> case could be a discussion in its own right, and maybe ChangeIds really
> trump the explicit tags proposed by Giovanni or myself here.  Whether
> that justifies the cognitive overhead of juggling them around on every
> submission remains to be shown or disproven.

I agree.  I am not convinced by the benefits and I already see some
troubles.

Cheers,
simon

Cuirass actors

2023-09-13 Thread Ludovic Courtès

Hello Guix!

My (useful? misguided? questionable?) quest around Cuirass has led me
closer to some of the goals mentioned in my previous message¹ and in the
‘TODO’ file².

The ‘wip-actors’ branch³, which I plan to merge soon, splits activities
of the ‘cuirass register’ process among several actors. Quoth ‘base.scm’:

- The "channel updater" is responsible for updating Git checkouts for
channels. There's a single instance of this actor; it limits
concurrent Git updates.

- The "evaluator" spawns evaluations of jobsets for the given channel
instances, again limiting the number of concurrent evaluations.

- The "builder" spawns derivation builds. There are currently two
implementations: the local builder sends build requests to the local
'guix-daemon' process, while the remote build delegates builds to
'cuirass remote-server'.

- Each jobset as an associated "monitor"; it requests channel updates,
evaluations, and builds to the actors above. It also receives requests
such as evaluation triggers that can come, for example, from the
/jobset/NAME/hook/evaluate HTTP endpoint.

- The "jobset" registry is a directory that maps jobset names to their
monitor.

In addition, ‘cuirass register’ implements a “bridge”: it listens for
connections on a Unix-domain socket, which allows ‘cuirass web’ to send
it commands.

This is used for instance to implement the /jobset/NAME/hook/evaluate
HTTP endpoint, which lets users trigger an evaluation of the given
jobset.

With this actor split, one could implement another “builder” backend,
for instance one that talks to a Build Coordinator process. It’s also
obviously close to the programming model encouraged by Goblins, which
should make eventual migration easier (and when that happens, if Goblins
provides an inter-process transport, we’ll no longer need the custom
“bridge” and we’ll be able to move actors from one process to another
much more easily.)

Feedback welcome!

Ludo’.

¹ https://lists.gnu.org/archive/html/guix-devel/2023-07/msg00096.html
²
https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/tree/TODO?id=9b227abd29b15e7e25c54a71c524e7b26252a270
³ https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/log/?h=wip-actors

Re: [workflow] Automatically close bug report when a patch is committed

2023-09-13 Thread Liliana Marie Prikler

Am Mittwoch, dem 13.09.2023 um 11:27 -0400 schrieb Maxim Cournoyer:
> For just closing cross-referenced bugs, I agree.  For closing
> forgotten, already merged issues on guix-patches we'd need the
> Change-Id and a tool able to map Change-Ids -> issue number (mumi is
> in the best place to do so).
> 
> It's been a hard discussion to follow, but I think we're coming to
> some understanding that we are discussing two different schemes that
> could be both implemented to provide different benefits, right?
I do wonder how the ChangeId would work in practice.  Since it's not
really assigned by the committer, it would have to be generated "on the
fly" and attached to the mail in between, which could result in all
kinds of nasty behaviour like unstable Ids or duplicated ones.  Also,
if we can automate this for ChangeIds, we could also automate this for
patch-sets – the last patch in the series just gets the Closes: tag
added by mumi.  

Furthermore, I'm not convinced that it would ease the issue of
forgotten bugs as you can't really apply them to the past.  So the
practical use is limited to the case where you intentionally cherry-
pick this or that commit from a series.  How we want to deal with that
case could be a discussion in its own right, and maybe ChangeIds really
trump the explicit tags proposed by Giovanni or myself here.  Whether
that justifies the cognitive overhead of juggling them around on every
submission remains to be shown or disproven.

Beyond the scope of the discussion so far, it also doesn't help us with
duplicate or superseded patches (e.g. two series on the mailing list
propose a similar change, because one of them has already been
forgotten).  Again, the explicit close tags would allow this case to be
handled in an interpretable fashion.  In both cases, we do however also
introduce the potential for incorrect tagging, which then needs to be
resolved manually (more or less a non-issue, as it's the status quo).

Cheers

Re: Implementing the guix-dameon in Guile

2023-09-13 Thread Vagrant Cascadian

On 2023-09-13, Christopher Baines wrote:
> I think this has been talked about for a while [1], but I want to make it
> happen. Currently the guix-daemon is still similar to the nix-daemon
> that it was forked from, and is implemented in C++. I think that a Guile
> implementation of the guix-daemon will simplify Guix and better support
> hacking on and around the daemon to add new features and move Guix
> forward.
>
> 1: 
> https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/ROADMAP.org#n71
...
> Let me know if you've got any comments or questions!

Sounds great!

My only real concern, as someone maintaining guix packages in Debian, is
to make sure that we do not break compatibility with being able to use
an older daemon, as Debian stable/bookworm is still at guix 1.4.x and it
would be nice to not have to force people to manually upgrade the daemon
(e.g. and even if a newer version lands in a future Debian stable
release, in general it will stuck using that version for some years as
well).

I have noticed occasional issues with the Debian packages of guix having
compatibility issues when newer versions of guile-git/libgit2,
guile-ssh/libssh2, etc. get introduced, and wonder if the same would
hold true of a daemon?

In Guix, by design you wouldn't really notice these sorts of problems as
it is always generally built with the current version, but Debian does
rely on ABI compatibility for package upgrades... I might be able to
keep better track of these types of issues in Debian, although various
guile-* modules that depend on C libraries seem to avoid the normal
detection mechanisms to trigger rebuilds in Debian.

That is a bit of a tangent, but it reminded me about that issue...

live well,
  vagrant

signature.asc
Description: PGP signature

Re: Swineherd: Guix System container manager

2023-09-13 Thread Ricardo Wurmus



> Can you share any ways you're currently using this?

We had been using this at work as part of a much bigger training
platform to launch customizable (but still reproducible) interactive
environments for workshops and courses.

An example is pre-configured RStudio installations with shared course
materials.  With guix.install (provided by r-guix-install) participants
could install any Guix packages from within their running R session.  We
also provided a web interface to customize the environment ahead of
time, and to save/restore the container root fs.

> Any plans/interest in hooking it into `guix deploy`?

Plans no, but interest yes!

-- 
Ricardo

Re: Swineherd: Guix System container manager

Hi Ricardo,

On Wed, 13 Sep 2023 at 11:06, Ricardo Wurmus  wrote:

[...]

>
> The Swineherd was designed to be used with Shepherd on foreign distros,
> so it does not assume to be running on top of Guix System (for better or
> worse).

Oh!  It is very promising.  Really cool!

> Of course the Swineherd is also available as a Guix package called
> “swineherd”.

Hum, I did (or adding guile):

guix time-machine -q -- shell swineherd

Then I do not know what to do.  What are the basic steps for testing it?

Cheers,
simon

Re: Implementing the guix-dameon in Guile

2023-09-13 Thread Development of GNU Guix and the GNU System distribution.

Hi!

Felix Lechner via "Development of GNU Guix and the GNU System
distribution."  writes:

> Hi Chris,
>
> On Wed, Sep 13, 2023 at 9:29 AM Christopher Baines  wrote:
>>
>> Rewrites are risky because you only get the value right at the end,
>> therefore the priority is to get a minimal but viable implementation in
>> Guile that can be switched to
>
> Sounds exciting! I would like to help with testing.

Indeed, congrats for securing the NLnet founding (and thanks to them!).

> The transformation toward a Guile daemon is a point of consistency and
> pride for the project and therefore unlikely to be second-guessed or
> reverted. My recommendation is to replace the daemon gradually—working
> from (apply system* (command-line) downward—and mainline your
> incremental changes as quickly as possible.

I had some musing about the daemon recently; I was thinking libguile
could be added to our old C++ daemon, which could then replace its
functions piece-wise with Scheme implemented ones?

At the end, we'd have almost everything in Scheme, at which point we
could rewrite the main loop to Scheme and say farewell to the old
daemon.

Would that approach makes any sense?  I know of at least one C++ project
integrating libguile, and that's the jami-daemon, so it could perhaps
provide some clues as to how to proceed to do so.

Thanks for the great initiative!

-- 
Thanks,
Maxim

Re: Implementing the guix-dameon in Guile

Hi Chris,

On Wed, Sep 13, 2023 at 9:29 AM Christopher Baines  wrote:
>
> Rewrites are risky because you only get the value right at the end,
> therefore the priority is to get a minimal but viable implementation in
> Guile that can be switched to

Sounds exciting! I would like to help with testing.

The transformation toward a Guile daemon is a point of consistency and
pride for the project and therefore unlikely to be second-guessed or
reverted. My recommendation is to replace the daemon gradually—working
from (apply system* (command-line) downward—and mainline your
incremental changes as quickly as possible.

Thanks for doing this!

Kind regards,
Felix

Implementing the guix-dameon in Guile

2023-09-13 Thread Christopher Baines

Hey!

I think this has been talked about for a while [1], but I want to make it
happen. Currently the guix-daemon is still similar to the nix-daemon
that it was forked from, and is implemented in C++. I think that a Guile
implementation of the guix-daemon will simplify Guix and better support
hacking on and around the daemon to add new features and move Guix
forward.

1: 
https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/ROADMAP.org#n71

My plan is to focus on this over the next year. I left my previous day
job quite a few months ago now to take a bit of a break, that's the main
reason I've been able to spend more time trying to push forward some of
the QA stuff. With some monetary support from NLNet [2], I'm planning to
continue this break and focus for the next year on getting a Guile
implementation of the guix-daemon written and adopted.

2: https://nlnet.nl/project/GuixDaemon-Guile/

Rewrites are risky because you only get the value right at the end,
therefore the priority is to get a minimal but viable implementation in
Guile that can be switched to, and not to get distracted on adding or
improving functionality unnecessarily. That is better done once the new
implementation has been adopted.

While I think there's a substantial amount of work to do, progress
towards a Guile guix-daemon has already been made. There was a least one
GSoC project which did make progress, and there's Guile implementations
of some of the functionality in Guix already.

Still though, I'd like to hear what people think about which direction
the implementation should go, and what features they'd like to see. Even
if those are not essential to make the Guile implementation viable, it
still might inform the direction to take.

The Guile rewrite of the guix-dameon was on my mind over 3 years ago
when I was thinking about the build coordinator [3]. As part of writing
the build coordinator, I got experienced using SQLite and the Guile
bindings and that'll come in very useful. The build coordinator also
uses fibers, and I'll probably look to use fibers as well in the
guix-daemon. I think the work Ludo has done on the shepherd has made
this possible.

3: https://lists.gnu.org/archive/html/guix-devel/2020-04/msg00323.html

Let me know if you've got any comments or questions!

Thanks,

Chris


signature.asc
Description: PGP signature

Re: Swineherd: Guix System container manager

2023-09-13 Thread Thompson, David

Hey, nice job Ricardo! This looks awesome!

On Wed, Sep 13, 2023 at 11:29 AM Katherine Cox-Buday
 wrote:
>
> On 9/13/23 3:06 AM, Ricardo Wurmus wrote:
> > Hi there,
> >
> > you know the Shepherd: it is an elegant service manager looking after a
> > herd of daemons.  Since it can be extended with Guile, I decided to do
> > just that to add an extra skill to the Shepherd, turning it into the
> > Swineherd.
> >
> > The Swineherd is a manager of Guix System containers.  It is implemented
> > as a Shepherd service called “swineherd” that lets you manage Guix
> > System containers as Shepherd services.  These system container services
> > implement a few common features to bring up bridge networking, execute
> > commands inside the containers, peek at files, and get simple usage
> > stats.
> >
> > Swineherd also provides an optional HTTP API server to launch, manage,
> > and destroy containers through an HTTP client.  This makes the Swineherd
> > useful for building web applications that launch Guix System containers
> > on demand.  It comes with an Info manual describing the assumptions, the
> > required setup, as well as the implemented commands and the HTTP API.
> >
> > The Swineherd was designed to be used with Shepherd on foreign distros,
> > so it does not assume to be running on top of Guix System (for better or
> > worse).
> >
> > You can find the code here:
> >
> >  https://github.com/BIMSBbioinfo/swineherd
> >
> > Of course the Swineherd is also available as a Guix package called
> > “swineherd”.
> >
> > If this tickles your interests I’d be very happy to hear your comments
> > and incorporate patches to extend it beyond my original requirements.
>
> This is really cool stuff! I'd love to see Guix continue to eat the
> dev-ops space, which, in my opinion, loves to stack complexity on top of
> complexity.

+1 to that. I did devops professionally for about 6 years and it
really is a mess.

> Can you share any ways you're currently using this?
>
> Any plans/interest in hooking it into `guix deploy`?

Hey now that's a cool idea!

Speaking of devops, I believe with swineherd it is now possible to
implement something I've wanted for many years: the Guix equivalent of
docker-compose.  It would be like an enhanced guix shell that also
starts up services needed for development like PostgreSQL or Redis or
whatever.

- Dave

Re: [workflow] Automatically close bug report when a patch is committed

Hi,

Vagrant Cascadian  writes:

> On 2023-09-09, Maxim Cournoyer wrote:
>> Vagrant Cascadian  writes:
 Did you see my message about integrating a commit-hook similar to what
 Gerrit uses?  It produces unique ID such as:

 --8<---cut here---start->8---
 Change-Id: I9b86781869d80eda347659f0c009b8dfe09bdfd0
 --8<---cut here---end--->8---
> ...
>>> That seems like it would only work if the patch was identical, as
>>> opposed to a slightly rebased patch on top of newer patches on master?
>>>
>>> How can you correlate Change-Id to a patch in the tracker?
>>
>> The Change-Id stays the same unless you manually edit it out of your
>> commit message when amending / rebasing, so the commit hash may change
>> while the Change-Id stays the same.  So you can rebase your feature
>> branch on master and share a v2, whose existing commits will have the
>> same Change-Ids (newly added commits would get their own Change-Id
>> trailer).
>
> Ok, that makes some sense.
>
> Although the majority of bugs in the cleanup work I did were actually
> filed by someone else entirely... so the Change-Id will not help with
> those. Not a reason not to implement it, but not consistent with some of
> the triggers of this thread. :)

I doubt the Change-Id idea would help much closing *bugs* on the
bug-guix tracker, but I'd expect it to be useful to close already merged
(but forgotten on the guix-patches tracker) *patches*.

The Change-Ids would be added automatically without the user having to
install anything, so from the time it's deployed we'd have a means to
identify which patch series were all merged.

-- 
Thanks,
Maxim

Re: How can we decrease the cognitive overhead for contributors?

Hi Fannys,

Fannys  writes:

> Ekaitz Zarraga  writes:
>
>>> > This is what I mean when I say many times emacs is kind of mandatory,
>>> > and
>>> > this thread is kind of a demonstration of what I meant because the main
>>> > discussion evolved to: you can use this or that in emacs to ease the
>>> > dev
>>> > experience.
>>> 
>>> 
>>> One of the benefits of my being able to attend Guix Days was seeing
>>> peoples' workflows and stacks in person.
>>> 
>>> As such, one of my conclusions having (already) committed to Guix was
>>> that I needed to master Emacs prior to Guile
>>> (Im highly flow orientated).
>>
>> But again, even if this is a great option for you, it might be a really bad
>> option for some other people. Everybody does not have the time to spend
>> learning emacs, or other specific tool. It's ok if the workflow suggests that
>> but it's not great if we have no other alternative.
>>
>> It's not accessible and imposes a barrier in some people.
>
> Yeah agreed. And we should be consious of that.
> Ironically by mandating Emacs and Email we force people to use specific
> tools while at the same time even though the same people will complain(!) 
> against vendor lock-in
> like github.

There's no lock-in.  You can use any tool you want.  Most people hacking
on Guix do so with Emacs and Geiser because these are currently the best
tools (that I know of) to do the job; these are the tools many of us
know and can easily recommend.  If Visual Code (or editor X) was
packaged in Guix and had great support for working with Guile, we could
also mention it in our manual or in the cookbook.

Notice I use recommend rather than mandate; these are just
recommendations that try to be helpful.  If it's not helpful to you, you
are free to select your own tool box and share how it works (via patches
to the contributing section or a blog post for example).

-- 
Thanks,
Maxim

to PR or not to PR, is /that/ the question?

2023-09-13 Thread Giovanni Biscuolo

Hi Simon,

please forgive me if I continue drifting in this beautiful sea...

This message is /very/ long but it's just because I'm trying to do my
best to provide a deep analisys of the **big** problems with eventually
adopting a «web based PR model» (please see below for a definition) for
contributions to Guix.

Please skip this if you don't have enough time to drift like I'm doing!

Also, please consider I don't have much experience on how a «web based
PR model» works and /doesn't work/ in practice... but I'm studying :-)

Simon Tournier  writes:

[...]

> This branch of the long thread starts with:
>
> Re: How can we decrease the cognitive overhead for contributors?
> Attila Lendvai 
> Fri, 25 Aug 2023 08:07:53 +
> 
> id:JRUs8LVm3AtAh0MnHjE5rnhB4sNET0vWTOO2N3w2KfvKoM3CALRNwHTmwJ0Y9Bg3ZDWCs8j-1bMCo9aRiaoac8TAkuXAvrWgSwt_8RcwhQA=@lendvai.name
> 
> https://yhetil.org/guix/JRUs8LVm3AtAh0MnHjE5rnhB4sNET0vWTOO2N3w2KfvKoM3CALRNwHTmwJ0Y9Bg3ZDWCs8j-1bMCo9aRiaoac8TAkuXAvrWgSwt_8RcwhQA=@lendvai.name
> https://lists.gnu.org/archive/html/guix-devel/2023-08
>
> Attila speaks about the PR model.

Actually he is speaking about «the web based PR model».

I don't want to sound pedantic, but [1]:

--8<---cut here---start->8---

It's worth noting that the term pull request is not universally used:
GitLab calls them merge requests for example. Furthermore I regard the
terms pull request and merge request to be poorly named, as the terms
can be conflated with terminology used by your version control tool
(e.g. git pull or git merge).  And the implementations of a pull or
merge request may not even perform a pull or a merge (you can also
rebase a pull/merge request, but nobody is calling them rebase
requests). A modern day pull request is so much more than a version
control tool operation or even a simple request to pull or merge a
branch: it is a nexus to track the integration of a proposed change
before during and after that change is integrated.

--8<---cut here---end--->8---

Please let me underline: PR it's *a* nexus, one of the different one
available with a DVCS.  With git, for example, we also have "git
request-pull", with it's email based workflow.

--8<---cut here---start->8---

But alas. Because GitHub coined the term and is the most popular
collaboration platform implementing this functionality, I'll refer to
this general workflow as implemented on GitHub, GitLab, Bitbucket, and
others as pull requests for the remainder of this post.

[...] At its core, the pull request is fundamentally a nice UI and
feature layer built around the common Git feature branch workflow.

--8<---cut here---end--->8---

As I tried to tell elsewhere, all so called "pull request" features
incorporated in various "web based forges" are just /user interfaces/
built around the Git feature branch workflow.

...and guess what: Guix _is_ implementing a Git feature branch workflow
[2]!  Guix is also starting to use a clever "request merging" workflow,
integrated with qa.guix.gnu.org [3], see
https://issues.guix.gnu.org/65846 as an example.

Guix is just not implementing a feature branch workflow to incorporate
_commits_ coming from the contributors, and I still think this is wise.

Furthermore:

--8<---cut here---start->8---

[...] I think it is time for industry to scrutinize the pull request
model and to evolve it into something better.

I know what you are thinking: you are thinking that pull requests work
great and that they are popular because they are a superior model
compared to what came before. These statements - aside from some nuance
- are true. But if you live in the version control space (like I do) or
are paid to deliver tools and workflows to developers to improve
productivity and code/product quality (which I am), the deficiencies in
the pull request workflow and implementation of that workflow among
vendors like GitHub, GitLab, Bitbucket, etc are obvious and begging to
be overhauled if not replaced wholesale.

[...] In other words, the way I see the world is that a specific
vendor's pull request implementation is just that: an implementation
detail. And like all implementation details, they should be frequently
scrutinized and changed, if necessary.

--8<---cut here---end--->8---

The rest of the article is worth reading since it is a professional
analisys of the problems with "Pull Requests" /implementations/ and
workflow, backed by some (sort of?) research:

--8<---cut here---start->8---

[...] the current implementation of pull requests actively discourages
the many smaller changes workflow. And since smaller changes result in
higher quality and faster reviews, today's implementations of pull
requests are undermining

Re: Swineherd: Guix System container manager

2023-09-13 Thread Katherine Cox-Buday


On 9/13/23 3:06 AM, Ricardo Wurmus wrote:

Hi there,

you know the Shepherd: it is an elegant service manager looking after a
herd of daemons.  Since it can be extended with Guile, I decided to do
just that to add an extra skill to the Shepherd, turning it into the
Swineherd.

The Swineherd is a manager of Guix System containers.  It is implemented
as a Shepherd service called “swineherd” that lets you manage Guix
System containers as Shepherd services.  These system container services
implement a few common features to bring up bridge networking, execute
commands inside the containers, peek at files, and get simple usage
stats.

Swineherd also provides an optional HTTP API server to launch, manage,
and destroy containers through an HTTP client.  This makes the Swineherd
useful for building web applications that launch Guix System containers
on demand.  It comes with an Info manual describing the assumptions, the
required setup, as well as the implemented commands and the HTTP API.

The Swineherd was designed to be used with Shepherd on foreign distros,
so it does not assume to be running on top of Guix System (for better or
worse).

You can find the code here:

 https://github.com/BIMSBbioinfo/swineherd

Of course the Swineherd is also available as a Guix package called
“swineherd”.

If this tickles your interests I’d be very happy to hear your comments
and incorporate patches to extend it beyond my original requirements.


This is really cool stuff! I'd love to see Guix continue to eat the 
dev-ops space, which, in my opinion, loves to stack complexity on top of 
complexity.


Can you share any ways you're currently using this?

Any plans/interest in hooking it into `guix deploy`?

Re: [workflow] Automatically close bug report when a patch is committed

Hi Liliana,

Liliana Marie Prikler  writes:

> Hi Maxim,
>
> Am Montag, dem 11.09.2023 um 16:41 -0400 schrieb Maxim Cournoyer:
>> [...]
>> Perhaps both approach[es] could be combined.  I still see value in a
>> general scheme to automate closing applied series that linger on in
>> Debbugs.
>> 
>> [0] 
>> https://lists.gnu.org/archive/html/guix-devel/2023-09/msg00138.html
>> 
>> Change-Ids would also add the benefit that any commit found in Git
>> could easily be traced back to their submission on the guix-patches
>> or guix trackers or vice-versa (git log --grep='Change-Id='), as
>> noted by Giovanni.
> The thing is, we're discussing the same basic workflow (which you lay
> out below), just different kinds of metadata that we'd have to attach
> to our commits.  IIUC ChangeIds need to actually be carried around by
> the committers as they e.g. rewrite patches (rebasing, squashing, what
> have you), and they're basically opaque hashes so I don't see the
> benefit to the reader.  (I think you might be arguing that the benefit
> is uniqueness, but I'm not sure if I ought to buy that.)

Correct; the Change-Ids must be preserved when rebasing ((*if* they were
already published -- otherwise it doesn't matter)), and for the human
reader they're mostly noise.  Since it can be automated though, from the
day it's added all the commits would have one and they would become a
valuable unique key to cross-reference between what's in master and what
was reviewed in guix-patches, for example.

> Meanwhile "Fixes: [whatever notation]" also needs to carried around,
> sure, but at the same time provides semantics by pointing to a (known)
> bug report.  Now again, I personally prefer cool URLs here, but that's
> a bike we can shed however we want.

That's nice, and as I wrote in my previous reply to Giovanni, I think
both schemes have their place.

>> The process could go like this:
>> 
>> 1. commits of a series pushed to master
>> 2. Savannah sends datagram to a remote machine to trigger the
>> post-commit job, with the newly pushed commits 'Change-Id' values (a
>> list of them).
>> 3. The remote machine runs something like 'mumi close-issues [change-
>> id-1 change-id-2 ...]'
> Yeah, I think we basically agree on the 1 and 2 here, but I don't think
> we have to really implement 3.  IMHO we could do something simpler for
> all parties by just carrying the bug number around (in whichever form),
> which we do for some of our pre-ChangeLog explanations already.

For just closing cross-referenced bugs, I agree.  For closing forgotten,
already merged issues on guix-patches we'd need the Change-Id and a tool
able to map Change-Ids -> issue number (mumi is in the best place to do
so).

It's been a hard discussion to follow, but I think we're coming to some
understanding that we are discussing two different schemes that could be
both implemented to provide different benefits, right?

-- 
Thanks,
Maxim

Re: [workflow] Automatically close bug report when a patch is committed

Hi Giovanni,

Giovanni Biscuolo  writes:

[...]

 Liliana Marie Prikler  writes:
>
 > WDYT about the following
 >   Applies: [patch] 
 >   Closes: [patch] 
 >   Resolves: [patch] 
 >   Done: [patch] 

[...]

> Anyway, I agree with Liliana that having more keyworks will help humans
> better capture (and remember) the implied semantics (that we should
> definitely document, anyway); for this reason my proposal is to accept
> all this _lowercased_ keywords (all followed by a ":" with NO spaces in
> between): fix, fixes, close, closes, resolve, resolves, do, done.

OK, I now get the point; we're not discussing synonyms but various
actions both 'Closes and 'Fixes' would be implement via the
n-d...@debbugs.gnu.org special email sent to the control server, but
they convey a different idea to us humans.  I agree.

[...]

>> If we choose this simple scheme where the top commit of a series can be
>> annotated with Debbugs control commands, I'd opt for:
>>
>> --8<---cut here---start->8---
>> Applies: #bug-number
>> --8<---cut here---end--->8---
>
> Uh I think I get it: you mean we could use a keyword in the commit
> message to allow the committer to effectively link a commit to a
> #bug-number, right?

Yes!  I think my thought train went that way while Liliana and yours
were more focused on a post push action to close *fixed* issues, right?
What I described here was a general process by which we could close
*patches* series that were forgotten in an 'open' state.

> This is something we sould consider, but it's another topic: how to
> effectively link commits to #bug-num (I guess we already talked about
> this in some other thread)
>
>> I'm not sure what [patch] or namespace add (is it for a fancy URL?), so
>> I'd drop them.
>
> I'll try to recap, sorry for the repetitions!
>
> Namespace has been assumed as part of the proposed URI to try address
> Vagrant's concerns [1]:
>
>
>So... I maintain the guix package in Debian, and want to make sure
>that whatever bug-closing indicator guix upstream uses, does not end up
>triggering when I push new upstream versions to salsa.debian.org ... and
>start incorrectly marking incorrect bug numbers on bugs.debian.org that
>were meant for debbugs.gnu.org.

I don't understand how this risk could be triggered; we're strictly
talking about commits made in the Guix repository, not in one of
Debian's?  Why/how would a Guix commit message trigger a Debian Debbugs
server action?  Maybe if Vagrant put something like:

Fixes:  that could cause problems?  But then
the URL is different, so we could filter out these, so I don't see the
problem, if we use URLs.  And if we accept just 'Fixes: #N', it
should be documented that these are strictly intended to refer to Guix
issues, not that of other projects.  I don't think we'll be able to make
this error proof, so we should instead aim to make it convenient for the
main use cases, which is to refer to Guix issues.  Does that make sense?

> Fixes: [optional bug description] 
>
> (here, URI is :#)
>
> ...but then you, Maxim, suggested [3] this form:
>
> Fixes: bug#65738 (java-ts-mode tests)

Note that we can stick with the  URL and
achieve the same result with some extra config (see: bug#65883).

 If so, that's adding rather than reducing friction, and I'm not sure
 it'd gain much traction.  The way I see it, it needs to happen
 automatically.
>>> I mean, the way I imagine is that you type this as part of your message
>>> and then debbugs would do the work of closing the bug.  In short, "git
>>> push" saves you the work of writing a mail because there's a hook for
>>> it.
>
> I guess all of us are looking for this very same thing: a server side
> web hook that automatically closes bugs (via email) when committers
> pushing "instructs" it to do so.
>
> The automatic email message will be sent to our "bug control and
> manipulation server" [5], with this header:
>
>
> From: GNU Guix git hook 
> Reply-To:  <>
> To: cont...@debbugs.gnu.org
>
>
> and this body:
>
>
> package guix
> close  []
> quit
>
>
> The "Reply-To:" (I still have to test it) will receive a notification
> from the control server with the results of the commands, including
> errors if any.
>
> Then, the documentation for the close command [5] states:

Note that 'close' is deprecated in favor of 'done', which does send a
reply.

> A notification is sent to the user who reported the bug, but (in
> contrast to mailing bugnumber-done) the text of the mail which caused
> the bug to be closed is not included in that notification.
>
> If you supply a fixed-version, the bug tracking system will note that
> the bug was fixed in that version of the package.
>
> Last but not least, the very fact that "GNU Guix git hook" have closed
> the bug report is tracked and showed in the bug report history, as any
> other action made via email using the Debbugs control

Re: How can we decrease the cognitive overhead for contributors?



Simon Tournier  writes:

> Hi Katherine,
>
> Thank you for your extensive analysis.  I concur.
>
> On Wed, 30 Aug 2023 at 10:11, Katherine Cox-Buday  
> wrote:
>
>> 3. We should reify a way for Guix, the project, to measure, and track 
>> progress,
>>     against long-term goals. Particularly when they're social and not 
>> strictly
>>     technical.
>
> That is the most difficult part, IMHO.  Well, what are the long-term
> goals? :-)
>
> I am almost sure we will get various answers depending on people.  Let
> say the long-term goals of the Guix project are: Liberating, Dependable
> and Hackable.  Then how do you give concrete quantities that we can
> measure or track?
>
> And it is always difficult, if not impossible, to measure or track some
> goals that are not technical but social.  For example, how do you
> measure being welcoming or being a safe place for all?
>
> Do not take me wrong, I strongly think we must think again and again on
> that point for improving.  It’s just easier to tackle technical bug. :-)
>
>
>>    11. Try and get each commit message close to correct and commit.
>
>> I view steps 1-10 as pretty standard "development" steps common to most
>> projects, although 11 compounds the effort in 10.
>
> Maybe I am doing incorrectly but I have never thought much about that.
>
> For that point #11, from my point of view, it is as with any other
> project.  I start by running “git log --grep=” for getting inspiration.
>
> Well, as a rule of thumb, I am doing something like:
>
> top-module: submodule: one line summary.
>
> Fixes .
> Reported by Jane Doe 
>
> * path/to/file.scm (variable):[sub-part]: Description of the change.
> [otherpart]: Other description.
> * path/to/other/file.scm (other-variable): Description.
>
>
> In case of doubt, I am just running “git log --grep=” looking for
> similar thing, as said. :-)
>
>
>>    12. Run `guix lint`
>>    13. Run `guix style` (this is still in the manual although I have since
>>    learned this is not actually advisable).
>>    14. Review the changes these tools have made, and fix things.
>>    15. Run `guix build --rounds=2 ` to check for idempotency.
>>    16. Run `make` to ensure you didn't break anything elsewhere in Guix.
>>    17. Run `guix size` to ensure the closure isn't becoming bloated.
>>    18. Build all the packages that depend on this package.
>>    19. Run `guix pull --url=/path/to/your/checkout 
>> --profile=/tmp/guix.master` to
>>    ensure you didn't break Guix in a different way.
>
>> In other projects I've worked with, steps 12-19 are commonly done in a CI
>> pipeline, and courteous people will try to save CI resources by running 
>> these
>> steps locally first using some kind of environment identical to what CI runs
>> (sometimes a container is used for this. I think Guix has better options!).
>> Sometimes this is not feasible due to asymmetric resources. But having the
>> option to let CI manage this process is very nice.
>
> For instance, I am not seeing “make check”. ;-)  And that omission makes
> very clear the cognitive overhead we are speaking about!
>
> Here I see two annoyances:
>
>  1. The number of subcommands and steps.
>  2. Each subcommand has a list of options to digest.
>
> Well, CI is helpful here, for sure.  However, it would be helpful to
> have a script similar as etc/teams.scm or etc/committer.scm that would
> help to run all these steps.
>
> It does not mean that all these steps need to be run before each
> submission.  However having a tool would help irregular contributors or
> newcomers; it would decrease the cognitive overhead i.e., that overhead
> would be pushed to some script and it would reinforce confidence.
>
> Now someone™ needs to implement this script. ;-)
>
>
>>    20. Run `git format-patch -1 --cover-letter [--reroll-count]`
>>    21. Run `./pre-inst-env ./etc/teams.scm cc-members ` to get 
>> the CC flags for Git
>>    22. Remember that if you're sending multiple patches, an email first 
>> has to be
>>    sent to `guix-patc...@gnu.org` to get an ID, and then...
>>    23. Run `git send-email --to guix-patches  `
>
> Well, my grey hair are biasing my opinion. ;-)  From my point of view,
> the most annoying is #22.
>
> Vagrant suggested [1] to send patches as attachment.  I am not convinced
> it will be better.  Well, it will for submitting but will not for
> following series.  For instance, let consider:
>
> [bug#65010] [PATCH 0/8] Misc Python build system improvements
> Lars-Dominik Braun 
> Wed, 02 Aug 2023 12:37:57 +0200
> id:cover.1690972374.git.l...@6xq.net
> https://issues.guix.gnu.org//65010
> https://issues.guix.gnu.org/msgid/cover.1690972374.git.l...@6xq.net
> https://yhetil.org/guix/cover.1690972374.git.l...@6xq.net
>
> then, one will do reply and probably comment one or more patches over
> the 8.  Then, it is harder for another person to follow.  For example, I
> would have to open the message in order

Re: How can we decrease the cognitive overhead for contributors?



I dont think we need to compare things here. Of course we should be able
to make lives easier for reviewers and contributors. There is no need
here to compare them. Remember that making lives easier for contributors
will make lives easier for reviewers too after all :) Because more
correct pathces and more people wanting to be involved.
Hence me saying its not a comparison or focusing on one or the
other. But more of the same thing.

As I mentioned in another point in the thread we already have
"non-standard" stuff so that argument doesn't really hold water
here. Non standard stuff being emacs, mumi and a bunch of other stuff
that make submissions tolerable. Or emacs-debbugs.

Also yes "visual" flow helps some people. Nothing wrong with that and we
should encourage that. (Encrourage that by fully supporting it). I mean i am 
trying to use the cli as less as
possible since its a horrible interface.

And yes the gnu commit messages could be improved. Its not like they are
set in stone anyway.

MSavoritias

Giovanni Biscuolo  writes:

> [[PGP Signed Part:Undecided]]
> Hello Katherine,
>
> thank you for having summarized (part of) this thread in a list of
> actionable tasks
>
> now Someone™ have the chance to decrease the cognitive overhead for
> contributors by _increasing_ her cognitive overhead to sort out and
> complete each task
>
> as a general comment, it seems to me that you put very much attention to
> the process of contributing v1 of a patch but you underestimate the
> cognitive overhead of the collective patch reviewing process and
> committing to Guix proper process
>
> AFAIU, observing what is happening in Guix since 2019, what is actually
> critical for Guix is _not_ the cognitive overhead needed to send a patch
> to guix-devel, but what comes after and _around_.
>
> last but not least, to be fair we should see at what other distribution
> (not single software projects) are doing for their contributing process:
> I know a little about Debian and in my experience it's far easier to
> contribute to Guix than to Debian (but I know little, I emphasize)
>
> Katherine Cox-Buday  writes:
>
>> Summary of my conclusions:
>>
>> 1. We should use sourcehut or continue to improve mumi
>
> Please forgive me if I insist, but the one and _only_ benefit of using
> SourceHut is the web-UI /helper/ to prepare an email message to send,
> it's "just" a web-UI version of the "git format-patch" CLI; the rest of
> the "patch management workflow" is email **and** CLI (git am) based;
> it's documented.
>
> Furthermore, users that are comfortable with the SourceHut web UI are
> free to use that as their personal working repo, there is no need for
> Guix to use a SourceHut remote as the official one.
>
>>     - QA status should be visable from a patch's page
>
> On mumi web interface, in each issue page related to a patch, there is a
> "badge" linking to the QA status for that patch, right below the issue
> title; i.e.:
>
> https://issues.guix.gnu.org/65694
>
> have a link to https://qa.guix.gnu.org/issue/65694
>
> QA (and relates services, like data.qa) is a great project that could
> greatly improve current situation when completed!
>
>>     - It should be possible to interact with the issue through the
>>  page
>
> I don't exactly understand: what do you mean with "interact"?
>
> ...and what page?  https://issues.guix.gnu.org/ or
> https://qa.guix.gnu.org/issue/65694 (or any other issue)
>
>> 2. We should create scripts/sub-commands to lift contribution activities 
>> into
>>     higher-order concepts:
>>     - Prepare a new submission
>>     - Run pre-checks on a submission
>>     - Submit a patch
>>     - Status of patch
>
> AFAIU you already use some of this "lifting" scripts od commands: can
> you please send patches so thay could possibly be included in Guix
> proper or in some section of the Cookbook?
>
> [...]
>
>> On 8/28/23 4:17 AM, Simon Tournier wrote:
>
> [...]
>
>>  > In order to be pragmatical and list actionable items, could you
>>  > specifically list what you consider as a toil or cognitive overhead?
>>  > Maybe you could share your script helping you.
>>
>> Yes, great point! Let's try to distill all this conversation down into the
>> salient points and see if we can't agree on some actionable items.
>>
>> Here's my understanding of the process to contribute a patch:
>>
>>    1. Check out main, and run `./bootstrap`, then `./configure 
>> --localstatedir=/var --sysconfdir=/etc`
>>    2. Run `make`
>>    3. You need to determine whether the change can be targeted against 
>> main or
>>   needs to target a feature branch, so you go read about that.
>
> [...]
>
>> In other projects I've worked with, steps 12-19 are commonly done in a CI
>> pipeline, and courteous people will try to save CI resources by running 
>> these
>> steps locally first using some kind of environment identical to what CI runs
>> (sometimes a container is used for this. I think Guix has better options!).
>> Sometimes this is

Re: How can we decrease the cognitive overhead for contributors?



Simon Tournier  writes:

> Hi,
>
> On Tue, 29 Aug 2023 at 12:53, MSavoritias  wrote:
>
>> Do you know if there are any plans to write a scheme bug/patching
>> system? Because looking a bit into it, it doesn't seem like its that
>> actively developed so maybe we would be better served by one in scheme.
>> Or Sourcehut of course as somebody wrote in another email.
>> Since not much would change with sr.ht anyways.
>
> The only work I know is named tissue by Arun Isaac.  Arun also
> contributes to Mumi (one web front-end of the venerable Debbugs instance
> of the GNU project).
>
> https://tissue.systemreboot.net/
> https://archive.fosdem.org/2023/schedule/event/tissue/
>
> Well, I am not convinced by the argument « we would be better served by
> one in scheme ».  For instance, Cuirass or Build Coordinator or even
> Mumi, all are implemented in Scheme and they are receiving very few
> contributions.
>
> https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git
> https://git.savannah.gnu.org/cgit/guix/build-coordinator.git/
> https://git.savannah.gnu.org/cgit/guix/mumi.git/
>
>
> Cheers,
> simon

Its more about:
If debbugs has issues and we dont fully control it and can make those
improvements, why not something in scheme instead of sr.ht?
But i could go either way.

MSavoritias

Re: performance issue with TeX Live

2023-09-13 Thread Emmanuel Beffara

De Olivier Dion le 12/09/2023 à 19:11:
> On Tue, 12 Sep 2023, Emmanuel Beffara  wrote:
> > Hello Guix devel,
> >
> > I am facing a severe performance issue with TeX Live: compilation of any
> > document is an order of magnitude slower with a Guix installed system as
> > compared to a manual installation. Is anyone confronted to this phenomenon, 
> > or
> > is there a way to fix this ?
> 
> I had a similar issue not long ago.  I think recent patches fix this and
> now it is smooth as before.

Apparently no. I did a pull and reconfigure and the behaviour is the same as I
described.

$ guix describe
Generation 53   Sep 12 2023 20:49:03(current)
  guix 4cb2e08
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 4cb2e08b8eb1acc824b72ec17b483d7d85b5af68

Running my compilations in a pure environment with

guix shell --pure texlive-scheme-medium texlive-collection-...

leads to the same behaviour, and the same with --container. Not sure how to
investigate this further.

-- 
Emmanuel

Re: performance issue with TeX Live

2023-09-13 Thread Nicolas Goaziou

Hello,

Emmanuel Beffara  writes:

> I am facing a severe performance issue with TeX Live: compilation of any
> document is an order of magnitude slower with a Guix installed system as
> compared to a manual installation. Is anyone confronted to this phenomenon, or
> is there a way to fix this ?

I also experienced a noticeable slowdown; I don't know how to fix it yet.

> I tried doing `strace` on the pdflatex calls to investigate further and it
> appears that the behaviours of the two versions are largely different.
> Counting the number of system calls of each kind gives the following, for the
> most frequent calls:
>
> - minimal document, Guix version:
>
>112860 newfstatat
> 10491 getdents64
>  5247 openat
>  5246 close
>  4397 access
>  3141 read
>
> - minimal document, manual version:
>
>  2772 read
>90 openat
>72 access
>64 close
>64 newfstatat
>60 getdents64
>
> - slideshow, Guix version:
>
>   2831722 getdents64
>   1538560 newfstatat
>   1498287 access
>   1415296 openat
>   1415295 close
>  4283 read
>
> - slideshow, Guix version:
>
>  3913 read
>  1288 getdents64
>  1136 access
>   960 openat
>   925 close
>   920 newfstatat
>
> So apparently no file hash is used in the Guix version and a large part of the
> `texmf-dist` folder is browsed, probably several times.

A ls-R file is generated during profile creation (see
`texlive-font-maps' function in "guix/profiles.scm") but it seems it is
not read.

It may be interesting to compare location and contents of the ls-R files
in both installations.

Regards,
-- 
Nicolas Goaziou

Re: How can we decrease the cognitive overhead for contributors?



Ekaitz Zarraga  writes:

>> > This is what I mean when I say many times emacs is kind of mandatory,
>> > and
>> > this thread is kind of a demonstration of what I meant because the main
>> > discussion evolved to: you can use this or that in emacs to ease the
>> > dev
>> > experience.
>> 
>> 
>> One of the benefits of my being able to attend Guix Days was seeing
>> peoples' workflows and stacks in person.
>> 
>> As such, one of my conclusions having (already) committed to Guix was
>> that I needed to master Emacs prior to Guile
>> (Im highly flow orientated).
>
> But again, even if this is a great option for you, it might be a really bad
> option for some other people. Everybody does not have the time to spend
> learning emacs, or other specific tool. It's ok if the workflow suggests that
> but it's not great if we have no other alternative.
>
> It's not accessible and imposes a barrier in some people.

Yeah agreed. And we should be consious of that.
Ironically by mandating Emacs and Email we force people to use specific
tools while at the same time even though the same people will complain(!) 
against vendor lock-in
like github.

Re: How can we decrease the cognitive overhead for contributors?

2023-09-13 Thread Fannys



Ekaitz Zarraga  writes:

>> > This is what I mean when I say many times emacs is kind of mandatory,
>> > and
>> > this thread is kind of a demonstration of what I meant because the main
>> > discussion evolved to: you can use this or that in emacs to ease the
>> > dev
>> > experience.
>> 
>> 
>> One of the benefits of my being able to attend Guix Days was seeing
>> peoples' workflows and stacks in person.
>> 
>> As such, one of my conclusions having (already) committed to Guix was
>> that I needed to master Emacs prior to Guile
>> (Im highly flow orientated).
>
> But again, even if this is a great option for you, it might be a really bad
> option for some other people. Everybody does not have the time to spend
> learning emacs, or other specific tool. It's ok if the workflow suggests that
> but it's not great if we have no other alternative.
>
> It's not accessible and imposes a barrier in some people.

Yeah agreed. And we should be consious of that.
Ironically by mandating Emacs and Email we force people to use specific
tools while at the same time even though the same people will complain(!) 
against vendor lock-in
like github.

Commenting bug reports via mumi web interface (was: How can we decrease the cognitive overhead for contributors?)

2023-09-13 Thread Giovanni Biscuolo

Hi Ricardo,

Ricardo Wurmus  writes:

> Giovanni Biscuolo  writes:
>
>> AFAIU mumi does not (still?) have ad authentication/authorization,
>> right?
>>
>> If so how do you plan to deal with users posting SPAM or similar
>> unappropriate content?
>
> It only sends email on behalf of commenters, so we’re using the same
> email mechanism to deal with spam.

Please forgive me if I'm not reading the source code for the relevant
mumi function, it would be easier for me to see it in action to
understand how the comment feature works.

I mean: I guess commenters are anonymous (?) and the mumi server will
send the email via authenticated SMTP (I hope) as user "mumi server" (or
something similar) on behalf of the commenter, right?

If so, the email is sent with the SPF and DKIM headers of the mumi
server configured mail server and that information is not useful to
eventually catch commenter email spoofing.

If I'm not missing something, then, anyone could send a comment as
"g...@xelera.eu" containing unappropriate content, right?

I know that the GNU mailing lists mail server surely have an antispam
service, but it cannot use DMARC (SPF and/or DKIM) to filter email
spoofing attempts and all it can do is to assign a "spamminess" score to
messages, that seldom is able to effectively spot "unappropriate"
content, right?

Given all this, does this mean that anyone could send an offensive
comment as "g...@xelera.eu" using the mumi commentig form?

...or are all the mailing lists moderated?

I feel I really miss something important in this picture, sorry for not
understanding what!

As an /antipattern/ example of a bug reporting system using a web
interface also for comments, I point out the one used by git-annex
(ikiwiki): https://git-annex.branchable.com/bugs/

When you try to "Add a comment", e.g. in:
https://git-annex.branchable.com/bugs/fsck_does_not_detect_corruption_on_yt_vids/

You are presented an authentication form supporting 3 auth methods:
registered user, email [1] and OpenID.

I still think that they sould just allow me to send an email to report
and comment bugs.

Thanks! Gio'

[1] The server sends you an unique URL you can use to log in and expires
in one day... why not just send me (forward) the complete message I want
to comment with the right Reply-to field pre-compiled, so I can edit my
comment with my lovely MUA instead of that /awful/ web interface?!?

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

signature.asc
Description: PGP signature

Re: [workflow] Automatically close bug report when a patch is committed

2023-09-13 Thread Giovanni Biscuolo

Hi Liliana and Maxim,

Liliana Marie Prikler  writes:

[...]

> The thing is, we're discussing the same basic workflow

No, we are discussing two (complementary) workflows:

1. the one I suggested to add a "footer-metadata-field" named
Fix/Fixes/Close/Closes/whatever that will allow people pushing to the
Guix official repo to _add_ the information that the (already installed,
to be enhanced) server side git post-receive hook should also close one
or more bug reports; that "metadata-footer" should be "manually" added
to the commit message before pushing, the commit must be _amended_ (git
commit --amend).

2. the one suggested by Maxim (bringed by Gerrit) to _automatically_ add
a "footer-metadata-field" named 'Change-Id' that will allow "a
machinery" (IMO it should be the currently installed hook, enhanced) to
_automaticcally_ close bug reports when all 'Change-Id's contained in a
bug report have been pushed to the official Guix repo.

This is my understanding of what we are discussing here: did I miss
something?

> (which you lay out below), just different kinds of metadata that we'd
> have to attach to our commits.

Thay are different because they serve different needs.

> IIUC ChangeIds need to actually be carried around by the committers as
> they e.g. rewrite patches (rebasing, squashing, what have you)

Since 'Change-Id' is automaticcaly generated by a _local_ git hook upon
committing and left unchanged if already present, the only precaution
the committer should apply is to preserve it when rebasing in case the
person needs to send a new version of the patch. 

> and they're basically opaque hashes so I don't see the benefit to the
> reader.

The benefit are not for the reader but for "the machinery" to be able to
compute when a patch set is completely pushed to the Guix official repo,
this also means that the related bug repo (related to the patch set) can
be happily automatically closed.  No?

> (I think you might be arguing that the benefit is uniqueness, but I'm
> not sure if I ought to buy that.)

The benefit is that 'Change-Id' is autogererated as unique, kept between
rebases (with some pracaution by the _local_ committer) thus is useful
to compute the completion of each patch contained in a bug repo (of
class [PATCH]).

Obviously all of this should be clearly documented, so everyone will
understand how it works.

[...]

>> In case it couldn't close an issue, it could send a notification to
>> the submitter: "hey, I've seen some commits of series  landing to
>> master, but not all of the commits appears to have been pushed,
>> please check"
> I'm not sure how common this case is

Don't know the cardinality, but I guess is a /very/ useful usecase for
people who have committ access to the official Guix repo... at least for
Maxim :-)

Anyway, I think the main use case for this _second_ way of automatically
closing related bugs (the one based upon 'Change-Id') when all [PATCH]es
are pushed is very useful for all people with commit access to the Guix
repo, a sort of "push and (almost) forget".

[...]

Ciao, Gio'.

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

signature.asc
Description: PGP signature

Re: How can we decrease the cognitive overhead for contributors?

Re,

On Wed, 13 Sep 2023 at 09:57, Simon Tournier  wrote:

> Somehow, the line is drawn using a two-axis plot of “easy vs boring”
> when instead we should draw a two-axis plot of “complexity vs capture
> the values”.

I meant, the thread is considering a one-axis plot using a subjective
parameter from “easy“ to “boring” and splits this line into two
subparts.  Instead, we should consider a two-axis plot using the
parameters “complexity” and “capture the value”.  Somehow. :-)

Cheers,
simon

Swineherd: Guix System container manager

2023-09-13 Thread Ricardo Wurmus

Hi there,

you know the Shepherd: it is an elegant service manager looking after a
herd of daemons.  Since it can be extended with Guile, I decided to do
just that to add an extra skill to the Shepherd, turning it into the
Swineherd.

The Swineherd is a manager of Guix System containers.  It is implemented
as a Shepherd service called “swineherd” that lets you manage Guix
System containers as Shepherd services.  These system container services
implement a few common features to bring up bridge networking, execute
commands inside the containers, peek at files, and get simple usage
stats.

Swineherd also provides an optional HTTP API server to launch, manage,
and destroy containers through an HTTP client.  This makes the Swineherd
useful for building web applications that launch Guix System containers
on demand.  It comes with an Info manual describing the assumptions, the
required setup, as well as the implemented commands and the HTTP API.

The Swineherd was designed to be used with Shepherd on foreign distros,
so it does not assume to be running on top of Guix System (for better or
worse).

You can find the code here:

https://github.com/BIMSBbioinfo/swineherd

Of course the Swineherd is also available as a Guix package called
“swineherd”.

If this tickles your interests I’d be very happy to hear your comments
and incorporate patches to extend it beyond my original requirements.

-- 
Ricardo

Re: Mumi CLI client (was: How can we decrease the cognitive overhead for contributors?)

2023-09-13 Thread Ricardo Wurmus

Giovanni Biscuolo  writes:

> AFAIU mumi does not (still?) have ad authentication/authorization,
> right?
>
> If so how do you plan to deal with users posting SPAM or similar
> unappropriate content?

It only sends email on behalf of commenters, so we’re using the same
email mechanism to deal with spam.

-- 
Ricardo

Re: How can we decrease the cognitive overhead for contributors?