Re: [workflow] Automatically close bug report when a patch is committed

[Liliana Marie Prikler]

Am Donnerstag, dem 14.09.2023 um 15:51 -0700 schrieb Vagrant Cascadian:
> On 2023-09-10, Liliana Marie Prikler wrote:
> > Am Donnerstag, dem 07.09.2023 um 09:12 -0700 schrieb Vagrant
> > Cascadian:
> > > I am much more comfortable with the "Fixes" convention of:
> > > 
> > >   Fixes: https://issues.guix.gnu.org/NNN
> > I like the idea, but we should also consider the bugs.gnu.org
> > address
> > here as well as the convention of putting it into angular
> > brackets.  In
> > fact, I might even prefer it if the convention was
> >   Fixes: Bug description 
> > where bug description is a (possibly empty) name for the bug such
> > as "Emacs hangs when I press a key" or something.
> > 
> > 
> > As for when to send it, remember that we already send a bunch of
> > mails to guix-comm...@gnu.org as our commit hook?  I think it
> > shouldn't be too hard to search for the fixes line and send it to
> > debbugs control.
> 
> Well, the complication gets to be ... which branch did it land in? in
> master, it's fairly obvious... you can just mark it as
> done/closed/etc. I guess with other branches it makes sense to mark
> it with the "pending" or maybe some more specific usertag
> "pending-in-BRANCH"?
I don't think such a distinction is needed in most cases.  In fact, if
it's about regular bugs, then a graft should likely hit master in case
that an actual update is needed on another branch.  Other than that,
it'd be silly to mark bugs specifically for e.g. "emacs-team" as still
pending on the account of them not having hit master yet.

Cheers

Re: Implementing the guix-dameon in Guile

[Mekeor Melire]

2023-09-13 16:36 m...@cbaines.net:

I think this has been talked about for a while [1], but I want 
to make it happen. Currently the guix-daemon is still similar to 
the nix-daemon that it was forked from, and is implemented in 
C++. I think that a Guile implementation of the guix-daemon will 
simplify Guix and better support hacking on and around the 
daemon to add new features and move Guix forward.


Yippie! I'm really excited about this; and convinced that it'll greatly impact 
and improve the Guix project.

I'd like to hear what people think about which direction the 
implementation should go, and what features they'd like to see.


Here's a feature-request (in the long-run). But I'm not sure if I understand 
correctly that a new, improved Guix-daemon enables us to implement this. Sorry 
if I'm wrong. I'd like to be able to step through the build-process of a 
package phase-by-phase and see what changed after each phase by looking at the 
file-tree.

Re: [workflow] Automatically close bug report when a patch is committed

[Vagrant Cascadian]

On 2023-09-10, Liliana Marie Prikler wrote:
> Am Donnerstag, dem 07.09.2023 um 09:12 -0700 schrieb Vagrant Cascadian:
>> I am much more comfortable with the "Fixes" convention of:
>> 
>>   Fixes: https://issues.guix.gnu.org/NNN
> I like the idea, but we should also consider the bugs.gnu.org address
> here as well as the convention of putting it into angular brackets.  In
> fact, I might even prefer it if the convention was
>   Fixes: Bug description 
> where bug description is a (possibly empty) name for the bug such as
> "Emacs hangs when I press a key" or something.
>
>
> As for when to send it, remember that we already send a bunch of mails
> to guix-comm...@gnu.org as our commit hook?  I think it shouldn't be
> too hard to search for the fixes line and send it to debbugs control.

Well, the complication gets to be ... which branch did it land in? in
master, it's fairly obvious... you can just mark it as
done/closed/etc. I guess with other branches it makes sense to mark it
with the "pending" or maybe some more specific usertag
"pending-in-BRANCH"?

live well,
  vagrant


signature.asc
Description: PGP signature

Re: Can we provide another UI for patches than just email ?

[Edouard Klein]

Hi Simon, Saku

Thank you for your answers.

First, I should have slept on my email instead of sending it right away.
I was still dealing with the frustration of having made a fool of myself
with debbugs, which compounded with wrestling earlier with golang build
system. My tone was unnecessarily harsh, please accept my apologies for
that.

I'd like to be part of the solution, so Saku, I'd like to help
write the putative scripts you talk about. The main difficulty I see is
that they need to be configured to be able to send and receive email on
the user's behalf. I have some more patches to send over, I'll use that
as an excuse to see what I can come up with.



Second, I don't know what the general feeling is here towards debbugs, but
if moving away from it is something that may happen, my suggestion is not
to use anything with Pull Requests, as Simon you seem to have
understood. I worded my suggestion poorly. What I had in mind was a
(semi-)public-writable repo: code is pushed by developers, pulled and
reviewed by maintainers, and if satisfactory, rebased on to master.

If communication about the patch is needed, an email thread can be
started on the mailing list, CCing the commit author(s) whose email
address is known from the commits. A bash one liner starting with git
log ... | sed ... can create the email automatically from the proposed branch.

There is no need for any active code on the server, just SSH access or
maybe public git:// access, and a place to store the repos.
See
https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm?ref_type=heads#L526
the os/git function for an example on how to set that up on a guix
server.
See https://the-dam.org/docs/tutorials/git-hosting.html for how to set
it up and https://the-dam.org/docs/tutorials/private-git.html for how to
create repos once it is set up.

Details need to be ironed out, such as whether to make it world-writable
or to have a vetting process, whether to have a public-writable repo
segregated from the official one, etc. but I think a reasonable
configuration can be found. It seems simpler than git email with or
without debbugs, for both maintainers and reviewers, but I'll admit I
only have experience working with small teams, and not running a large
and successful open source project, so I can't be very assertive there.



Lastly, I think the lack of reviewers stems from the lack of
contributors. I for one feel like I could give good reviews on trivial
patches like new packages and package updates. Just check the repo is
the official one, check the licence is good, and check that it, and its
descendents, builds. I would gladly review a few packages a week, if I
did not dread interacting with git email. Again, not insurmontable, but
friction is a powerful thing.
If we could onboard contributors more easily, we would have a larger
pool of maintainers to pick from. Guix is freaking awesome, but very
hard to pick up. It took me a few years to get up to speed.



Anyway, thanks for the great software and for the time and guidance
y'all give on help-guix and here. Sorry for the ranty email.

Cheers,

Edouard.

Simon Tournier  writes:

> Hi Edouard,
>
> It is very important to speak up, although that makes me sad to read
> such poor feedback experience.
>
> Well, the friction is about Debbugs.  Maybe I repeat myself: Debbugs is
> initially thought to be a bug tracker system and not a patch track system.
>
> Do not take me wrong, I am not trying to convince you.  Instead, I am
> just trying to explain that email workflow is not so much different and
> the main annoyances you point come from Debbugs and not emails,
> somehow. :-)
>
> On Thu, 14 Sep 2023 at 10:51, Edouard Klein  wrote:
>
>> Before anybody tries to explain to me that git send-email is easier than
>> I think, what you have to beat is:
>
> Well, the complete workflow you have in mind is:
>
> 1.  $ git remote add guix-patches WHATEVER  #only once
> 2.  $ git push -u guix-patches master:some-unique-name
> 3.  … send a Pull Request … #only once
>
> Instead, the email workflow somehow reads:
>
> a.  … send an Email to guix-patc...@gnu.org #only once
> b.  $ git config sendemail.to 12...@debbugs.gnu.org #only once
> c.  $ git send-email --base=auto -v  origin
>
> and the order is flipped:
>
> a == 3   #only once
> b == 1   #only once
> c == 2
>
> Yeah, the order appears awkward. But that’s because of Debbugs; else it
> would not be very different.
>
> The advantage is that contributing does not require from you to have a
> public Git repository, it requires from you only an email address.
>
> The drawback (plural!), bah you already know them. :-)
>
> Again, thank you for your feedback.
>
> Cheers,
> simon

Re: [workflow] Automatically close bug report when a patch is committed

[Maxim Cournoyer]

Hello,

Le 14 septembre 2023 12:30:41 GMT-04:00, Vagrant Cascadian  
a écrit :

[…]

>> I doubt the Change-Id idea would help much closing *bugs* on the
>> bug-guix tracker, but I'd expect it to be useful to close already merged
>> (but forgotten on the guix-patches tracker) *patches*.
>
>Well, all of the "bugs" I was referring to were submitted patches
>tracked at debbugs.gnu.org via the issues.guix.gnu.org
>frontend... weather they were submitted via guix-patches@ or bug-guix@
>or n...@debbugs.gnu.org... :)
>
>They are all "bugs" to me, though "issues" seems a bit more neutral in
>tone and I like it better, but new packages or package updates are just
>generally wishlist bugs/issues and not inherrently something broken,
>though they may sometimes fix things or be related to fixes...

Let's call these patch series to avoid confusion :-)

>
>> The Change-Ids would be added automatically without the user having to
>> install anything, so from the time it's deployed we'd have a means to
>> identify which patch series were all merged.
>
>Well, they would still have to install the commit hook, or did I miss
>something?

Yes! Hooks can be configured upon building the Guix checkout, the same way its 
now done for our pre-push hook :-)

Maxim

Re: How can we decrease the cognitive overhead for contributors?

[Sarthak Shah]

I think that quite a few Guix users end up not committing to Guix because
of how daunting and strange the process seems.
In particular, having an alternate, easy-to-use interface for updating
package definitions specifically could be very useful.

The greatest strength of Guix is that it can be very easily integrated into
Guile programs.
I believe that it should not be very hard to create a user interface (web,
ncurses or GUI) that searches for a package's location (the package record
has a field that accommodates this) and then displays it, which the user
can then edit. We can then have the program build the package based on this
new definition, and if the user is satisfied with it it can format a patch
based on the edit and all the contributor will have to do is e-mail it.

This could greatly reduce the barrier to entry for contributing newer
package versions, but it could also open up the door to spam/misuse if we
host it on the web.

Regards,
Sarthak.

On Wed, 13 Sep, 2023, 17:54 Fannys,  wrote:

>
> Ekaitz Zarraga  writes:
>
> >> > This is what I mean when I say many times emacs is kind of mandatory,
> >> > and
> >> > this thread is kind of a demonstration of what I meant because the
> main
> >> > discussion evolved to: you can use this or that in emacs to ease the
> >> > dev
> >> > experience.
> >>
> >>
> >> One of the benefits of my being able to attend Guix Days was seeing
> >> peoples' workflows and stacks in person.
> >>
> >> As such, one of my conclusions having (already) committed to Guix was
> >> that I needed to master Emacs prior to Guile
> >> (Im highly flow orientated).
> >
> > But again, even if this is a great option for you, it might be a really
> bad
> > option for some other people. Everybody does not have the time to spend
> > learning emacs, or other specific tool. It's ok if the workflow suggests
> that
> > but it's not great if we have no other alternative.
> >
> > It's not accessible and imposes a barrier in some people.
>
> Yeah agreed. And we should be consious of that.
> Ironically by mandating Emacs and Email we force people to use specific
> tools while at the same time even though the same people will complain(!)
> against vendor lock-in
> like github.
>
>

Re: Can we provide another UI for patches than just email ?

[Simon Tournier]

Hi Edouard,

It is very important to speak up, although that makes me sad to read
such poor feedback experience.

Well, the friction is about Debbugs.  Maybe I repeat myself: Debbugs is
initially thought to be a bug tracker system and not a patch track system.

Do not take me wrong, I am not trying to convince you.  Instead, I am
just trying to explain that email workflow is not so much different and
the main annoyances you point come from Debbugs and not emails,
somehow. :-)

On Thu, 14 Sep 2023 at 10:51, Edouard Klein  wrote:

> Before anybody tries to explain to me that git send-email is easier than
> I think, what you have to beat is:

Well, the complete workflow you have in mind is:

1.  $ git remote add guix-patches WHATEVER  #only once
2.  $ git push -u guix-patches master:some-unique-name
3.  … send a Pull Request … #only once

Instead, the email workflow somehow reads:

a.  … send an Email to guix-patc...@gnu.org #only once
b.  $ git config sendemail.to 12...@debbugs.gnu.org #only once
c.  $ git send-email --base=auto -v  origin

and the order is flipped:

a == 3   #only once
b == 1   #only once
c == 2

Yeah, the order appears awkward. But that’s because of Debbugs; else it
would not be very different.

The advantage is that contributing does not require from you to have a
public Git repository, it requires from you only an email address.

The drawback (plural!), bah you already know them. :-)

Again, thank you for your feedback.

Cheers,
simon

Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)

[Simon Tournier]

Hi,

On Thu, 14 Sept 2023 at 18:51, Ludovic Courtès  wrote:

> For the record, that’s a 6% increase:
>
> --8<---cut here---start->8---
> $ guix size guix | tail -1
> total: 633.0 MiB
> $ guix size guix git-minimal | tail -1
> total: 675.7 MiB
> --8<---cut here---end--->8---
>
> (Of course it all adds up; I’m not saying we can dismiss it.)
>
> In the context of  plus the lack of
> GC in libgit2 discussed in , my
> inclination is to include that hard dependency on Git.

And considering bug#65924 [1], it is not 6% but more.  Because
currently git-minimal is broken and coreutils and potentially
util-linux would also be part of the closure.

--8<---cut here---start->8---
$ guix size guix | tail -1
total: 633.0 MiB
$ guix size guix git-minimal coreutils | tail -1
total: 692.7 MiB
$ guix size guix git-minimal coreutils util-linux | tail -1
total: 706.6 MiB
--8<---cut here---end--->8---

Therefore. it is 9.4% or worse 11.6%.

1: bug#65924: git searches coreutils and util-linux commands in PATH
Maxim Cournoyer 
Wed, 13 Sep 2023 14:00:09 -0400
id:87fs3iuf6e@gmail.com
https://yhetil.org/guix/87fs3iuf6e@gmail.com
https://issues.guix.gnu.org/msgid/87fs3iuf6e@gmail.com


> That’s not a happy choice for me, but it has the advantage of solving
> two immediate problems.

Three. :-)

Once git-minimal and plumbing Git commands around, some slow
procedures using libgit2 will be replaced by faster ones.  And I also
have in mind some Git repository normalization that differs from SWH;
having plain Git commands would also easy that part.

Just to point that the introduction of git-minimal as hard dependency
of Guix means that libgit2 will be slowly removed from the picture,
IMHO.

Cheers,
simon

Re: [workflow] Automatically close bug report when a patch is committed

[Liliana Marie Prikler]

Am Donnerstag, dem 14.09.2023 um 11:42 +0200 schrieb Giovanni Biscuolo:
> OK! :-)  Let's see how this relates to the 2 use cases we are talking
> about:
> 
> 1. Use "Fixes:" (et al) in commit msg to tell "the hook" to close the
> bug.
> 
> This "action" implies that the commit we are pushing upstream
> "Applies:" to that bug report; it has no added value.
> 
> 2. Use 'Change-Id'...
> 
> This also implies that the commit we are pushing upstream "Applies:"
> to that bug report related to that [PATCH]; no added value also.
> 
> So, when and only when we will implement a 'Change-Id' requirement
> adding an 'Applies' metadata is not useful for linking [PATCH]es to a
> bug report.
> 
> Did I miss something?
On "Fixes:" vs. "Applies:" etc: the point of having more keywords is
that "Fixes:" implies a bug, whereas more general wording is preferable
when pushing a patch that simply updates a package.  I just wrote some
example wordings and wanted all of us to agree on the one that makes
"the most sense"; alas it got misinterpreted.

> 
> > Maybe if Vagrant put something like:
> > 
> > Fixes:  that could cause problems?  But
> > then the URL is different, so we could filter out these, so I don't
> > see the problem, if we use URLs.
> 
> Yes we are saying the same thing! :-)
> 
> Sorry I've made confusion but Vagrant's concern was expressed
> _before_ someone proposed (maybe Liliana) to use namespaced URIs.
> 
> Vagrant please: do you confirm that using URLs "Fixes:
> " is OK for your usecase?
I actually prefer cool URLs myself, my format was 
"Fixes: [optional description] "

> > 
> To be cristal clear: I think that "the other proposal" (that is use
> "Fixes:" and alike in commit msg to close the provided bug num) will
> be **superseeded** when all the tools to manage (first of all: CLI
> query tool) the 'Change-Id' preudo-header/footer :-D
Well I still very much prefer human readable footers, but I'm perhaps a
little lonely in this debate.

Cheers
> > > 

Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)

[Ludovic Courtès]

Hi!

Maxim Cournoyer  skribis:

> So given there's no technical reasons not to use libgit2, I'd use that
> and keep the closure size down.

For the record, that’s a 6% increase:

--8<---cut here---start->8---
$ guix size guix | tail -1
total: 633.0 MiB
$ guix size guix git-minimal | tail -1
total: 675.7 MiB
--8<---cut here---end--->8---

(Of course it all adds up; I’m not saying we can dismiss it.)

In the context of  plus the lack of
GC in libgit2 discussed in , my
inclination is to include that hard dependency on Git.

That’s not a happy choice for me, but it has the advantage of solving
two immediate problems.

I would revisit it as soon as libgit2 supports shallow clones (which is
coming, as you write) and GC (or a workaround to that effect).  SHA256
may also soon be a requirement: we’ll need to be able to clone repos
that use it.

How does that sound?

Ludo’.

Re: comparing commit-relation using Scheme+libgit2 vs shellout plumbing Git

[Simon Tournier]

Hi Ludo,

On Thu, 14 Sep 2023 at 12:30, Ludovic Courtès  wrote:

>but I don’t think
> we can get a decent throughput if we shell out for all these things
> (assuming ‘git’ can even give us raw data).

Do you consider that Magit does not have a decent throughput?
Do you consider that Git-Annex does not have a decent throughput?

To my knowledge, they shell out Git plumbing commands; one using Emacs
Lisp and the other Haskell.

And some porcelain Git commands that we all are using daily are Bash
scripts calling plumbing Git commands that shell out.  (Or were Bash
scripts before being replaced by C builtin).

For example, git-rebase, git-pull, git-log, etc.

https://github.com/git/git/commit/55071ea248ef8040e4b29575376273e4dd061683
https://github.com/git/git/commit/b1456605c26eb6bd991b70b0ca0a3ce0f02473e9
https://github.com/git/git/commit/e3a125a94d34d22a8ca53e84949a1bb38cd6e425

The task is probably complex and boring, I agree.  However, I am not
convinced the issue is about a “decent throughput”.  The best would be
to have the same performance using libgit2 and using plumbing Git
command for one example: say ’commit-relation’.  Or another one. :-)

Otherwise, I believe what I am seeing. ;-)

Cheers,
simon

Re: [workflow] Automatically close bug report when a patch is committed

[Simon Tournier]

Hi,

On Thu, 14 Sep 2023 at 12:27, Giovanni Biscuolo  wrote:

> Please can you expand what troubles do you see in automatically adding
> 'Change-Id:' using a hook-commit-msg like
> https://gerrit-review.googlesource.com/Documentation/cmd-hook-commit-msg.html
> ?

1. The hook must be installed.
2. The hook must not be in conflict with user configuration.
3. The generated Change-Id string can be mangled by some user unexpected
   action.
4. etc.

Many things can rail out on user side.  For an example, base-commit is
almost appearing systematically in submitted patches almost 3 years
later.  The patches of some submissions are badly formatted.  Etc.

Whatever the implementation, I am not convinced that the effort is worth
the benefits.  And I am not convinced it will help in closing the
submissions when the patches have already been applied.

That’s said, I am not against the proposal.  I just have mixed feelings
and before deploying I strongly suggest to review if the proposal covers
the intent.

Cheers,
simon

Re: Implementing the guix-dameon in Guile

[Ludovic Courtès]

Christopher Baines  skribis:

> I'm not very familiar with actors, but I guess that's similar to having
> a bunch of cooperating fibers which handle different things.

Yes.  Specifically, actors here are a fiber together with a channel;
each actor looks like:

  (let loop ((state …))
(match (get-message channel)
  ;; handle request and call ‘loop’
  ))

The actor keeps serving requests it gets on its channel.

It makes quite a difference when you start framing it this way.  When I
first fiberized Cuirass years ago, I’d just use ‘spawn-fiber’ here and
there where concurrency was needed.  Thinking in terms of actors makes
the code clearer, allows you to think about “concurrency units”, respect
separation of concerns, and so on.  (Our Spritely comrades would explain
this much better!)

Ludo’.

Re: [workflow] Automatically close bug report when a patch is committed

[Vagrant Cascadian]

On 2023-09-13, Maxim Cournoyer wrote:
> Vagrant Cascadian  writes:
>> On 2023-09-09, Maxim Cournoyer wrote:
>>> The Change-Id stays the same unless you manually edit it out of your
>>> commit message when amending / rebasing, so the commit hash may change
>>> while the Change-Id stays the same.  So you can rebase your feature
>>> branch on master and share a v2, whose existing commits will have the
>>> same Change-Ids (newly added commits would get their own Change-Id
>>> trailer).
>>
>> Ok, that makes some sense.
>>
>> Although the majority of bugs in the cleanup work I did were actually
>> filed by someone else entirely... so the Change-Id will not help with
>> those. Not a reason not to implement it, but not consistent with some of
>> the triggers of this thread. :)
>
> I doubt the Change-Id idea would help much closing *bugs* on the
> bug-guix tracker, but I'd expect it to be useful to close already merged
> (but forgotten on the guix-patches tracker) *patches*.

Well, all of the "bugs" I was referring to were submitted patches
tracked at debbugs.gnu.org via the issues.guix.gnu.org
frontend... weather they were submitted via guix-patches@ or bug-guix@
or n...@debbugs.gnu.org... :)

They are all "bugs" to me, though "issues" seems a bit more neutral in
tone and I like it better, but new packages or package updates are just
generally wishlist bugs/issues and not inherrently something broken,
though they may sometimes fix things or be related to fixes...


> The Change-Ids would be added automatically without the user having to
> install anything, so from the time it's deployed we'd have a means to
> identify which patch series were all merged.

Well, they would still have to install the commit hook, or did I miss
something?


live well,
  vagrant


signature.asc
Description: PGP signature

[fr] Moment de convivialité Guix@Paris en septembre

[Tanguy LE CARROUR]

(Warning: this email is in french because the meeting is supposed
to be held in French… and in person.)

Bonjour Guix,

Ce jeudi 28 septembre à 19h, se tiendra la première édition de Guix@Paris
ouverte au public.


## Programme

Il n'y a pas vraiment de programme. Le but est que les utilisat·rice·eur·s
de Guix —ou futur·e·s utilisat·rice·eur·s !—, dans la région parisienne se
rencontrent et qu'ielles puissent discuter de Guix, Guile ou autres.

L'idée est surtout de passer un moment convivial en mettant des visages
sur des noms. Se rencontrer quoi ! 

Il n'y a pas d'expérience pré-requise et vous êtes tout·e·s les bienvenu·e·s.


## Logistique

Vous pouvez nous dire si vous comptez venir en répondant à ce message,
mais vous serez aussi les bienvenu·e·s au dernier moment ! 

Nous nous occupons des tables, des chaises, des prises électriques et
du vidéo-projecteur. Il y aura aussi de quoi boire et grignoter, mais
vous pouvez apporter ce qui vous ferait plaisir.


## Accès

Nous serons accueilli·e·s dans les locaux de l'[April][1], elle même hébergée
par [Easter-eggs][2] :

```
Association April
44/46 rue de l'Ouest (cour intérieure)
Bâtiment 8
75014 Paris

Stations de Métro : Gaîté, Montparnasse, Pernety.

OpenStreetMap : .
```

[1]: https://april.org "April, promouvoir et défendre le logiciel libre"
[2]: https://easter-eggs.com/Presentation-d-Easter-eggs "Easter-eggs"


Au plaisir de vous y rencontrer !

-- 
Tanguy

Re: Implementing the guix-dameon in Guile

[Christopher Baines]

Ludovic Courtès  writes:

>> Rewrites are risky because you only get the value right at the end,
>> therefore the priority is to get a minimal but viable implementation in
>> Guile that can be switched to, and not to get distracted on adding or
>> improving functionality unnecessarily. That is better done once the new
>> implementation has been adopted.
>
> In the past I wondered, as Maxim wrote, whether we could move
> incrementally—after all, a fair bit of “the daemon” is already in Scheme
> (there’s substitute support and other helps, plus (guix store …) etc.)
> I’m not sure that’s feasible or desirable though.
>
> My take today :-) is that ‘wip-guile-daemon’ is a great starting point.
> We could aim towards having a minimal ‘guix daemon’ (space) command that
> would coexist with ‘guix-daemon’ and that people could try out soonish
> (I think Caleb got it to build derivations back then).  Eventually less
> adventurous people will use it, and at some point it’ll be sufficiently
> mature that we can default to ‘guix daemon’ instead of ‘guix-daemon’.

Yep, my hope is that this can happen over the next year.

> Technically, I think it should be a single-threaded Fibers program,
> building on the experience we got from the Coordinator, shepherd, etc.
> That should allow us to do everything in one process (in contrast, the
> C++ implementation forks for every incoming connection, which then
> significantly complicates the implementation of locking, etc.)

Yep, that sounds good to me.

> I imagine the daemon could be structured as a set of actors (it’s really
> my thing these days ;-)), with an eye on facilitating code sharing and
> interaction with the Coordinator, Cuirass, and all that.

I'm not very familiar with actors, but I guess that's similar to having
a bunch of cooperating fibers which handle different things.

And maybe not in the short term, but if the Guile daemon implementation
is flexible and extensible enough, it should be possible to replace the
build coordinator with some extensions in and around the guix daemon.


signature.asc
Description: PGP signature

Re: Cuirass actors

[Thompson, David]

On Thu, Sep 14, 2023 at 11:31 AM Ludovic Courtès  wrote:
>
> Hi!
>
> "Thompson, David"  skribis:
>
> > I'm curious to hear more about your inter-process transport needs!
>
> I’d like to have actors running in separate processes on the same
> machine.  I wouldn’t want them to communicate over Tor or TCP+TLS;
> rather, I’d like them to use AF_UNIX sockets, “abstract sockets”, pipes,
> socketpairs—one of these local IPC mechanisms.
>
> My understanding is that there could be an ocapn “netlayer” based on
> that.  Does that make sense?  Is this something you’re considering?

Thanks for clarifying! Yes, we definitely intend to provide local IPC
netlayers such as UNIX domain sockets.  We actually have a test
netlayer that uses UNIX domain sockets, so some of this work has
already been done.

- Dave

Re: [bug#62047] [PATCH 0/2] '--with-input' & co. no longer replace hidden packages

[Ludovic Courtès]

Hi Greg,

(It’s been two months but I completely missed this message.)

Greg Hogan  skribis:

> On Wed, Mar 8, 2023 at 7:03 AM Ludovic Courtès  wrote:
>>
>> Hello,
>>
>> This change makes things like:
>>
>>   guix build --with-input=guile=guile-next guix -n --no-grafts
>>
>> more useful and tractable.
>>
>> Low-level rewrites are still possible for packages not marked
>> as hidden in 'commencement.scm', such as glibc:
>>
>>   guix build --with-latest=glibc hello -n
>>
>> Thoughts?
>>
>> Ludo'.
>>
>> Ludovic Courtès (2):
>>   packages: Use SRFI-71 instead of SRFI-11.
>>   packages: 'package-input-rewriting/spec' ignores hidden packages.

[...]

> This has broken, for example, building clang with a newer version of
> gcc using package-input-rewriting/spec.

As in:

   guix build clang --with-c-toolchain=clang=gcc-toolchain@12

?  Or some other command?

> What do you think of adding a hidden? property to enable the old
> behavior?

Maybe, but I’m not sure I fully understand the problem.

> Alternatively, why are gcc, binutils, and libc used as build-time
> dependencies rather than gcc-toolchain? gcc-toolchain could be
> rewritten as a non-hidden package and use of the toolchain would
> seemingly better support profiles created with
> package->development-manifest.

We could replace gcc, binutils, etc. with just ‘gcc-toolchain’; this
will need testing of course, but I cannot think of any issue it would
cause.  (That’s a ‘core-updates’ change though.)

Thanks,
Ludo’.

Re: Implementing the guix-dameon in Guile

[Ludovic Courtès]

Christopher Baines  skribis:

> I think compatibility is a priority, and although I haven't looked too
> much in to the details yet, I think it's quite realistic.
>
> I think it's very important for Guix to keep compatibility with older
> daemons, and on the daemon side, I'd want to see the Guile
> implementation work with older Guix's, as well as it being possible to
> switch back and forth between the implementations without issue.

+1.  The derivation format and what goes in the build environment (on
GNU/Linux) are set in stone.

The RPC protocol could evolve if needed, preserving backward
compatibility.

Ludo’.

Re: Cuirass actors

[Ludovic Courtès]

Simon Tournier  skribis:

> IIUC, the current two “builder” backend are:
>
>  + local ’guix-daemon’: the queue of derivations is processed using one
>strategy – the one implemented in C++,
>
>  + ’cuirass remote-server’: the queue of derivations is processed using
>another strategy – implemented in Guile relying on ZeroMQ (something
>like steal of work, if I remember correctly).
>
> Is it correct?

Yes.

> The Build Coordinator also implements the other actors “channel
> updater”, “evaluator”, etc., right?

No.

> From my rough understanding, the first aim of Build Coordinator is the
> implementation of the “builder”.  Is it correct?

Yes.

> My two questions are:
>
>  1. Is the Build Coordinator able to only process the queue of
> derivations?  (being a “builder” backend).  And if yes, what is its
> strategy for processing?

Yes, you give it derivations and it builds them; as for what its
scheduling strategy is, I don’t know!

>  2. In this picture of actor model, would it make sense (or would it be
> possible) to replace the “builder” actor from the Cuirass one to the
> Build Coordinator one?

Yes, that’s exactly the message I tried to convey.  :-)

Thanks,
Ludo’.

Re: Cuirass actors

[Ludovic Courtès]

Hi!

"Thompson, David"  skribis:

> I'm curious to hear more about your inter-process transport needs!

I’d like to have actors running in separate processes on the same
machine.  I wouldn’t want them to communicate over Tor or TCP+TLS;
rather, I’d like them to use AF_UNIX sockets, “abstract sockets”, pipes,
socketpairs—one of these local IPC mechanisms.

My understanding is that there could be an ocapn “netlayer” based on
that.  Does that make sense?  Is this something you’re considering?

Thanks,
Ludo’.

Re: Implementing the guix-dameon in Guile

[Ludovic Courtès]

Hi Christopher,

Christopher Baines  skribis:

> My plan is to focus on this over the next year. I left my previous day
> job quite a few months ago now to take a bit of a break, that's the main
> reason I've been able to spend more time trying to push forward some of
> the QA stuff. With some monetary support from NLNet [2], I'm planning to
> continue this break and focus for the next year on getting a Guile
> implementation of the guix-daemon written and adopted.
>
> 2: https://nlnet.nl/project/GuixDaemon-Guile/

Yay, this is great news!! But also: thank you for your long-term
commitment, it’s an invaluable contribution to the project.

> Rewrites are risky because you only get the value right at the end,
> therefore the priority is to get a minimal but viable implementation in
> Guile that can be switched to, and not to get distracted on adding or
> improving functionality unnecessarily. That is better done once the new
> implementation has been adopted.

In the past I wondered, as Maxim wrote, whether we could move
incrementally—after all, a fair bit of “the daemon” is already in Scheme
(there’s substitute support and other helps, plus (guix store …) etc.)
I’m not sure that’s feasible or desirable though.

My take today :-) is that ‘wip-guile-daemon’ is a great starting point.
We could aim towards having a minimal ‘guix daemon’ (space) command that
would coexist with ‘guix-daemon’ and that people could try out soonish
(I think Caleb got it to build derivations back then).  Eventually less
adventurous people will use it, and at some point it’ll be sufficiently
mature that we can default to ‘guix daemon’ instead of ‘guix-daemon’.

Technically, I think it should be a single-threaded Fibers program,
building on the experience we got from the Coordinator, shepherd, etc.
That should allow us to do everything in one process (in contrast, the
C++ implementation forks for every incoming connection, which then
significantly complicates the implementation of locking, etc.)

I imagine the daemon could be structured as a set of actors (it’s really
my thing these days ;-)), with an eye on facilitating code sharing and
interaction with the Coordinator, Cuirass, and all that.

I hope this makes sense.  Time will tell!

Ludo’.

Re: Implementing the guix-dameon in Guile

[Josselin Poiret]

Hi Chris,

Christopher Baines  writes:

> Hey!
>
> I think this has been talked about for a while [1], but I want to make it
> happen. Currently the guix-daemon is still similar to the nix-daemon
> that it was forked from, and is implemented in C++. I think that a Guile
> implementation of the guix-daemon will simplify Guix and better support
> hacking on and around the daemon to add new features and move Guix
> forward.
>
> 1: 
> https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/ROADMAP.org#n71
>
> My plan is to focus on this over the next year. I left my previous day
> job quite a few months ago now to take a bit of a break, that's the main
> reason I've been able to spend more time trying to push forward some of
> the QA stuff. With some monetary support from NLNet [2], I'm planning to
> continue this break and focus for the next year on getting a Guile
> implementation of the guix-daemon written and adopted.

Great and ambitious project! I am also interested in it and am willing
to work on it (although at a much slower pace than you might).

> 2: https://nlnet.nl/project/GuixDaemon-Guile/
>
> Rewrites are risky because you only get the value right at the end,
> therefore the priority is to get a minimal but viable implementation in
> Guile that can be switched to, and not to get distracted on adding or
> improving functionality unnecessarily. That is better done once the new
> implementation has been adopted.
>
> While I think there's a substantial amount of work to do, progress
> towards a Guile guix-daemon has already been made. There was a least one
> GSoC project which did make progress, and there's Guile implementations
> of some of the functionality in Guix already.
>
> Still though, I'd like to hear what people think about which direction
> the implementation should go, and what features they'd like to see. Even
> if those are not essential to make the Guile implementation viable, it
> still might inform the direction to take.

I think the #1 feature for me would be to have it completely
unpriviledged using mount namespaces, so that you could still build
software without needing to run the daemon on the system.  You won't be
able to run the built software without using namespaces as well though,
but that still a step in the right direction imo.

WDYT?
-- 
Josselin Poiret


signature.asc
Description: PGP signature

Re: Cuirass actors

[Thompson, David]

Hey Ludo,

On Wed, Sep 13, 2023 at 5:08 PM Ludovic Courtès  wrote:
>
> With this actor split, one could implement another “builder” backend,
> for instance one that talks to a Build Coordinator process.  It’s also
> obviously close to the programming model encouraged by Goblins, which
> should make eventual migration easier (and when that happens, if Goblins
> provides an inter-process transport, we’ll no longer need the custom
> “bridge” and we’ll be able to move actors from one process to another
> much more easily.)

We're obviously very excited by this development over at Spritely. :)

I'm curious to hear more about your inter-process transport needs!

- Dave

Re: [emacs]: snapshots against latest versions

[Munyoki Kilyungi]

Hi Cayetano!  Allow me to chime in, although I'm
green.

Cayetano Santos 
aliandika:

> Hi Guix,
>
>   Following a recent patch to an snapshot of an emacs package
>   (emacs-mastodon), where latest stable (tagged) release dates back from
>   a long time, the question of whether to send patches for non stable
>   (tagged) versions is pertinent or not.
>
>   Of course, the answer is clear: we only package stable. Guix manual
>   (22.4.3 Version Numbers) clearly states that "We usually package only
>   the latest version of a given free software project ... Occasionally,
>   we package snapshots of upstream’s version control system (VCS)
>   instead of formal releases.  This should remain exceptional.". Fine
>   with that.
>
From my experience, this depends.  In the python
system, updating some things can have ripple
effects that propagate to other packages, and
sometimes in a not so nice way.  For packages that
have ripple effects, I find useful to package to
the latest version upto where things are stable.
Also, sometimes, a recent change can introduce a
security bug, so in such a case, unless a patch is
provided, I see no need to be in the latest
version.

[...]

-- 
(Life is like a pencil that will surely run out,
but will leave the beautiful writing of life.)
(D4F09EB110177E03C28E2FE1F5BBAE1E0392253F
(hkp://keys.openpgp.org))


signature.asc
Description: PGP signature

Re: [workflow] Automatically close bug report when a patch is committed

[Giovanni Biscuolo]

Maxim Cournoyer  writes:

[...]

> I like the 'Closes: ' trailer idea; it's simple.  However, it'd need to
> be something added locally, either the user typing it out (unlikely for
> most contributors) or via some mumi wizardry (it's unlikely that all
> users will use mumi), which means its usage (and value) would depend on
> how motivated individuals are to learn these new tricks.

I agree: the ratio, or better usecase, of my /trivial/ (in design, not
in implementation) initial proposal [1] was to try to help committers
closing bugs "in one go" by adding proper information to the commit
message, e.g. "Closes: #N"

It was _not_ intended for contributors, also because they could _not_
know that **specific** patch in a patch series will really close a
**whole** bug report: that's only a judgement of the committer.

Also, my ratio was influenced by my misunderstanding of a series of bug
closing actions performed by Vagrant (see [1] for details): the problem
in the majority (all?) of those cases was **not** that the committer
simply forgot to close the related bug report /but/ that bug reports
containing (different) patches for the _same_ package were not linked
each other: the solution to this class of problems in obviously not
"Automatically close bug report when a patch is committed", it's
something else [2]

> On the other hands, having Change-Ids added by a pre-commit hook
> automatically would means the user doesn't need to do anything special
> other than using git, and we could still infer useful information at any
> time (in a server hook, or as a batch process).
>
> For this reason, I think we could have both (why not?  Change-Ids by
> themselves provide some value already -- traceability between our git
> history and guix-patches).

I agree: just having 'Change-Ids' alone already provide some added
value, even if we still miss the tooling (server side git hook, batch
processing

Thanks!  Gio'


[1] id:8734zrn1sc@xelera.eu 
https://yhetil.org/guix/8734zrn1sc@xelera.eu/

[2] id:87msxyfhmv@xelera.eu https://yhetil.org/guix/87msxyfhmv@xelera.eu

-- 
Giovanni Biscuolo

Xelera IT Infrastructures


signature.asc
Description: PGP signature

Re: comparing commit-relation using Scheme+libgit2 vs shellout plumbing Git

[Ludovic Courtès]

Simon Tournier  skribis:

> On my machine, I get something less spectacular for a history with 1000
> commits in between.
>
> scheme@(guix-user)> ,time (commit-relation* 1000th newest)
> $1 = ancestor
> ;; 0.128948s real time, 0.082921s run time.  0.046578s spent in GC.
> scheme@(guix-user)> ,time (commit-relation 1000th newest)
> $2 = ancestor
> ;; 4.588075s real time, 5.521358s run time.  1.404764s spent in GC.
>
> I did something very similar as wolf is proposing and named it
> ’commit-relation*’.

That’s an order of magnitude.  Probably it could be a bit less if we put
some effort in it (‘commit-relation’ is implemented in a fairly naive
way.)

That said, ‘commit-relation’ is just one example.  I’d encourage
interested people to look at (guix git-authenticate) to get a feel of
what we need.  Most of it is quite pedestrian, like
‘load-keyring-from-reference’ or ‘commit-signing-key’, but I don’t think
we can get a decent throughput if we shell out for all these things
(assuming ‘git’ can even give us raw data).

Ludo’.

Re: [workflow] Automatically close bug report when a patch is committed

[Giovanni Biscuolo]

Simon Tournier  writes:

[...]

>>  maybe ChangeIds really trump the explicit tags proposed by Giovanni
>> or myself here.  Whether that justifies the cognitive overhead of
>> juggling them around on every submission remains to be shown or
>> disproven.
>
> I agree.  I am not convinced by the benefits and I already see some
> troubles.

Please can you expand what troubles do you see in automatically adding
'Change-Id:' using a hook-commit-msg like
https://gerrit-review.googlesource.com/Documentation/cmd-hook-commit-msg.html
?

Thanks, Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures


signature.asc
Description: PGP signature

Re: [workflow] Automatically close bug report when a patch is committed

[Giovanni Biscuolo]

Hi Liliana

Liliana Marie Prikler  writes:

> Am Mittwoch, dem 13.09.2023 um 11:27 -0400 schrieb Maxim Cournoyer:

[...]

> I do wonder how the ChangeId would work in practice.

It's a «tag to track commits across cherry-picks and rebases.»

It is used by Gerrit to identify commits that belong to the same review:
https://gerrit-review.googlesource.com/Documentation/user-changeid.html

We could use it for the same purpose and instead of building a web
application for code review, "simply" count that all 'Change-Id's in a
patchset have been pushed to the Guix official repo to declare the
related bug report closed.

> Since it's not really assigned by the committer, it would have to be
> generated "on the fly" and attached to the mail in between

Not to the mail, to the commit msg! [1]

> which could result in all kinds of nasty behaviour like unstable Ids
> or duplicated ones.

No, modulo hook script bugs obviously.

> Also, if we can automate this for ChangeIds, we could also automate
> this for patch-sets – the last patch in the series just gets the
> Closes: tag added by mumi.  

The idea is that, but we don't need to add "Closes" to the commit msg
(via post-receive hook), we "just" need the hook to send an email to
-done on behalf of the committer (the committer, not the
contributor).

> Furthermore, I'm not convinced that it would ease the issue of
> forgotten bugs as you can't really apply them to the past.

No, this 'Change-Id' is not intended for past bug reports since we
**must not** rewrite past commits _because_ commit messages are
/embedded/ in commit objects.

...but for this purpose we could use git-notes, **if** wanted:
https://git-scm.com/docs/git-notes :-D

> So the practical use is limited to the case where you intentionally
> cherry- pick this or that commit from a series.

No: the practical use is that for each guix-patch bug report we can
count how many [PATCH]es are left to be committed and act accordigly,
for example notify all involved parties (contributor, committer,
'X-Debbugs-CC's) that N/M patches from the series are still to be merged
upstream... or close the bug report if zero patches are left.

> How we want to deal with that case could be a discussion in its own
> right, and maybe ChangeIds really trump the explicit tags proposed by
> Giovanni or myself here.  Whether that justifies the cognitive
> overhead of juggling them around on every submission remains to be
> shown or disproven.

There will be no additional cognitive overhead for contributors since
'Change-Id' will be automatically managed, they can simply ignore it.

> Beyond the scope of the discussion so far, it also doesn't help us with
> duplicate or superseded patches (e.g. two series on the mailing list
> propose a similar change, because one of them has already been
> forgotten).

No, IMO there is **no** solution to this problems other than "triaging"
(id:87msxyfhmv@xelera.eu
https://yhetil.org/guix/87msxyfhmv@xelera.eu/)

> Again, the explicit close tags would allow this case to be
> handled in an interpretable fashion.  In both cases, we do however also
> introduce the potential for incorrect tagging, which then needs to be
> resolved manually (more or less a non-issue, as it's the status quo).

There is no potential of incorret tagging when using a hook-commit-msg
[1] to add 'Change-Id'.

For the other method discussed here, there is no way to avoid users
mistyping 'Closes:' pseuto-headers in their commit messages: if mistuped
they will be ignored :-(


Cheeers, Gio'


[1] 
https://gerrit-review.googlesource.com/Documentation/cmd-hook-commit-msg.html

-- 
Giovanni Biscuolo

Xelera IT Infrastructures


signature.asc
Description: PGP signature

Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)

[Ludovic Courtès]

Hi,

Josselin Poiret  skribis:

> My opinion is that the preferred API for Git is still the UNIX one via
> plumbing commands.  Anything else is trying to catch up to it, and then
> we get into this conundrum that we want to do everything in Scheme, but
> we're unable to do it as well as Git itself.  If I had to choose, a
> Guile library wrapping the Git commands would be the best, especially
> since we're managing long-living checkouts, something libgit2 doen't
> seem too interested in.

I have mixed feelings here.  Clearly, I don’t think a Unix command can
ever be as rich and efficient as a “proper library”.

Are alternative Git implementations doomed to always try to catch up?
My intuition would be “no”, because not so much changes in Git as an
on-disk format and protocol.

There is one big change coming up though: SHA256 support (now officially
supported in Git).  Is it being discussed in libgit2?

Maxim Cournoyer  skribis:

> Josselin Poiret  writes:

[...]

>> There's still the `git gc` problem though.
>
> It's klunky, but a workaround is to locally clone the checkout anew
> using libgit2, as suggested here [0].
>
> [0]  https://github.com/libgit2/libgit2/issues/3247#issuecomment-486585944

That doesn’t work, at least with libgit2 1.3.2:

  https://issues.guix.gnu.org/65720#7

> We could also try to contribute to libgit2 toward adding proper
> support for a 'gc' action.

I share this sentiment: if we’re gonna depend on it, we’ve gotta invest
in it.  We’re benefiting from it so we shouldn’t be mere consumers.

I have to admit I don’t see myself doing it right now, but I would
definitely encourage others to do so.

Now, as a corollary to what I wrote above: if we don’t invest in it, we
should be prepared to drop it.

Ludo’.

Re: Guix pull speed

[Ludovic Courtès]

Hello!

Josselin Poiret  skribis:

> After looking a bit more into guix pull speed, or to be more precise the
> "Computing Guix derivation..." step, which is not substitutable.  I've
> come to the conclusion that the thing that takes the majority of the
> time is loading the files that define the packages that the new Guix
> needs to build itself.  These files are not compiled yet, and worse,
> loading just (gnu packages guile) ends up loading 361 other package
> files.  You can generate a package graph in GraphML with `guix graph -t
> module -b graphml guile`, and use e.g. networkx to analyze it.

Nice (we should add this trick under “Invoking guix graph” for instance;
we already mention xdot there.)

> You can compare with a compiled check-out of guix by just running the
> following in a `guix repl`:
>
> (use-modules (guix self) (guix monad-repl))
> ,run-in-store (guix-derivation (getcwd) "0.0-git" #:pull-version 1)
>
> which takes at most 5 seconds on my laptop.
>
> One idea I had was to move all the packages that are looked up in (guix
> self) to their own little bubble, to avoid having to load extra stuff.
> However, this is not currently possible because some of them do have
> non-trivial dependency graphs.  I've identified these problematic
> inputs: guile-avahi guile-ssh guile-git guile-gnutls guix-daemon (it
> pulls in all other dependencies itself) po4a graphviz

Yeah, I don’t think the idea of an independent subset of the module
graph is achievable, because sooner or later glibc starts depending on
Python, which starts depending on Rust, which pulls in Qt.  Compiling
with ‘-Wunused-modules’ might allow us to clean up the module graph a
bit though.

> What could be done about this?  Another solution would be to somehow
> build Guix without any of the dependencies and then add them in later,
> similar to what is done with build-aux/build-self.scm to be able to load
> (guix self) in the first place.  That seems quite complex though.

Most of the time is spent evaluating (gnu packages …) modules;
evaluation is pretty slow, and from what I remember of previous
profiling sessions, this is largely due to psyntax.  We should
concentrate on that.


Another approach: the resulting .drv is a deterministic result.
Wouldn’t it be nice if we could make the whole “Computing Guix
derivation” process a derivation itself?  That way it could be
substituted.

Nix introduced (or considered introducing) “recursive Nix”, whereby a
derivation build process can talk to the daemon “from the inside”,
creating additional derivations on the way.  This could be one
possibility, but it’s a lot of work with unclear outcomes.

In Guix, we can use the (guix …) modules inside the build process, and
we have almost everything we need to compute a set of derivations “in
the abstract” (we’d need a  backend that, instead of
making RPCs, would store .drv and “sources” in memory or in a file).  So
we could create a derivation that returns let’s say a nar containing the
closure of the .drv (the “Guix derivation”), which we would then import.

There’s “just” one gotcha with this plan: grafts.  To compute grafts, we
potentially need to build things, and for that we’d need to actually
talk to the daemon.


Maybe there are other possibilities, such as adding a
“builtin:build-self” derivation builder in the daemon.

Food for thought!

Ludo’.

Re: [workflow] Automatically close bug report when a patch is committed

[Giovanni Biscuolo]

Hi Maxim and Vagrant,

I'm sorry for some of the inconprehensions.

Finally I think we got a useful design overall for the _two_ user cases
and the related pieces of metadata and algorithms, now it's time for
_two_ implementations proposals, that also means _code_...

If nihil obstat, I'm going to open 2 bug reports (severity: wishlist)
and will take ownership... but only to coordinate the work, since I miss
some competence /and/ some authorization (for example to get and/or
install the server side git hook)

Maxim Cournoyer  writes:

[...]

>> Anyway, I agree with Liliana that having more keyworks will help humans
>> better capture (and remember) the implied semantics (that we should
>> definitely document, anyway); for this reason my proposal is to accept
>> all this _lowercased_ keywords (all followed by a ":" with NO spaces in
>> between): fix, fixes, close, closes, resolve, resolves, do, done.
>
> OK, I now get the point; we're not discussing synonyms but various
> actions

Yes :-D

[...]

>>> If we choose this simple scheme where the top commit of a series can be
>>> annotated with Debbugs control commands, I'd opt for:
>>>
>>> --8<---cut here---start->8---
>>> Applies: #bug-number
>>> --8<---cut here---end--->8---
>>
>> Uh I think I get it: you mean we could use a keyword in the commit
>> message to allow the committer to effectively link a commit to a
>> #bug-number, right?
>
> Yes!

OK! :-)  Let's see how this relates to the 2 use cases we are talking
about:

1. Use "Fixes:" (et al) in commit msg to tell "the hook" to close the
bug.

This "action" implies that the commit we are pushing upstream "Applies:"
to that bug report; it has no added value.

2. Use 'Change-Id'...

This also implies that the commit we are pushing upstream "Applies:" to
that bug report related to that [PATCH]; no added value also.

So, when and only when we will implement a 'Change-Id' requirement
adding an 'Applies' metadata is not useful for linking [PATCH]es to a
bug report.

Did I miss something?

> I think my thought train went that way while Liliana and yours
> were more focused on a post push action to close *fixed* issues,
> right?

Yes, it's a (super?) /brainstorming/ :-)

> What I described here was a general process by which we could close
> *patches* series that were forgotten in an 'open' state.

Yes: until we miss the 'Change-Id' metadata, we cannot do [1] nothing
for forgotten patches series.

[...]

>> Namespace has been assumed as part of the proposed URI to try address
>> Vagrant's concerns [1]:
>>
>>
>>So... I maintain the guix package in Debian, and want to make sure
>>that whatever bug-closing indicator guix upstream uses, does not end up
>>triggering when I push new upstream versions to salsa.debian.org ... and
>>start incorrectly marking incorrect bug numbers on bugs.debian.org that
>>were meant for debbugs.gnu.org.
>
> I don't understand how this risk could be triggered; we're strictly
> talking about commits made in the Guix repository, not in one of
> Debian's?  Why/how would a Guix commit message trigger a Debian Debbugs
> server action?  Maybe if Vagrant put something like:
>
> Fixes:  that could cause problems?  But then
> the URL is different, so we could filter out these, so I don't see the
> problem, if we use URLs.

Yes we are saying the same thing! :-)

Sorry I've made confusion but Vagrant's concern was expressed _before_
someone proposed (maybe Liliana) to use namespaced URIs.

Vagrant please: do you confirm that using URLs "Fixes:
" is OK for your usecase?

[...]

>> Fixes: [optional bug description] 
>>
>> (here, URI is :#)
>>
>> ...but then you, Maxim, suggested [3] this form:
>>
>> Fixes: bug#65738 (java-ts-mode tests)
>
> Note that we can stick with the  URL and
> achieve the same result

Thinking twice about this point, now I see that using the URL is
**much** better than , simply because URLs can be
copy/pasted in a browser for users not using the bug-reference Emacs
feature or any other similar feature in their preferred IDE, if
available.

> with some extra config (see: bug#65883).

Fantastic!

[...]

>> The automatic email message will be sent to our "bug control and
>> manipulation server" [5], with this header:
>>
>>
>> From: GNU Guix git hook 
>> Reply-To:  <>
>> To: cont...@debbugs.gnu.org
>>
>>
>> and this body:
>>
>>
>> package guix
>> close  []
>> quit
>>
>>
>> The "Reply-To:" (I still have to test it) will receive a notification
>> from the control server with the results of the commands, including
>> errors if any.
>>
>> Then, the documentation for the close command [5] states:
>
> Note that 'close' is deprecated in favor of 'done', which does send a
> reply.

Sorry I'm not finding 'done' (and the deprecation note) here:
https://debbugs.gnu.org/server-control.html

Maybe do you mean that we should not use the control server but send a
message to -d...@debbugs.gnu.org?

Like:

Re: Guix CLI options through manifest.scm or guix.scm

[Ludovic Courtès]

Hi,

Distopico  skribis:

> Is it possible to define/express CLI options from manifest.scm or
> guix.scm? For example, I have
>
> ```
> guix shell \
>  -m manifest.scm \
>  --container -F -N -P \
>  --share=/opt/android-sdk \
>  --share=$HOME/.android/avd \
>  --share=$HOME/.gradle/ \
>  --preserve='^DISPLAY$' \
>  --preserve='^XAUTHORITY$' \
>  --preserve='^DBUS_' \
>  --expose=$XAUTHORITY \
>  --expose=/var/run/dbus
> ```

There are two things relevant to ‘guix shell’: (1) what packages go
inside the environment, and (2) how the environment is built (container
vs. pure, networking or not, etc.).

The manifest only takes care of #1.

We need a non-CLI way to express #2.  I don’t think manifests are a good
fit for that; perhaps we need a new  abstraction, which
might mean we need a new ‘environment.scm’ file or similar.

Anyhow, I agree that something’s missing here.

Thanks,
Ludo’.

Re: Can we provide another UI for patches than just email ?

[Saku Laesvuori]

> Dear Guix,
> 
> I've just blown up Guix's debbugs by opening (and then closing) 33 bugs
> instead of sending a 33-commits patch series.
> 
> I've also seen that there is a huge discussion on the cognitive overhead
> of contributing. I will read it as soon as I can spare more time on this.
> 
> I find git email's workflow extremely frustrating and confusing. I know
> it works for most of you who actually commit, but I would like to point
> out that I'm one more person to be tripped by its complexity, and who
> cares enough to say so here. There are probably even more people who
> don't speak up.

I have done that too, but I think the problem is caused by debbugs, not
by git send-email.

> Therefore, there is a huge cost in keeping up using it: less people will
> be able to provide patches, and software will get stale.
> 
> For example, ou current Go version is 1.17 ! From 2021 ! The current is
> 1.21. I suspect that if contributing was easier, we wouldn't have a 2
> years delay in our versions.

I don't think so, because we are already getting more contributions than
are getting reviewed. If contributions were easier to send but reviewing
them wouldn't change we'd just have that 2 years of delay in applying
the patches.

> [...]
>
> Before anybody tries to explain to me that git send-email is easier than
> I think, what you have to beat is:
> git remote add guix-patches WHATEVER #only once
> git push -u guix-patches master:some-unique-name
> 
> You can then push, pull, rebase, whatever, from the command line (or
> magit in my case), without leaving your dev context. Can't be easier
> than that.

It would be possible to have a git send-email -based workflow that would
look something like this:

$ git clone ...
[ make changes ]
$ git send-email origin/master
[ get feedback, edit and rebase ]
$ git send-email origin/master --in-reply-to=msgid
[ repeat ]

We could even have a script that would submit changes and with that the
workflow could be as simple as:

$ git clone ...
$ ./script new
[ make changes ]
$ ./script send
[ get feedback, edit and rebase ]
$ ./script send
[ repeat ]

The problem is not using email, it is that the tooling we currently have
or recommend requires doing more manual and error-prone steps than is
actually necessary.


signature.asc
Description: PGP signature

Re: Swineherd: Guix System container manager

[Ricardo Wurmus]

Simon Tournier  writes:

>> Of course the Swineherd is also available as a Guix package called
>> “swineherd”.
>
> Hum, I did (or adding guile):
>
> guix time-machine -q -- shell swineherd
>
> Then I do not know what to do.  What are the basic steps for testing it?

There are no executables.

The basic steps for setting things up are in the included manual.

You’ll first need to set up a root disk for all your disposable
container home directories.  See the “Getting Started” chapter in the
manual on instructions how to get this set up for testing.

You then launch a new Shepherd process with the included
shepherd-config.scm (it’s at $out/share/swineherd/shepherd-config.scm),
which registers the “swineherd” and “swineherd-http-api” services, and
starts the former.

You can then try the Shepherd actions documented in the “Command Line
Interface” section of the manual.

I haven’t tried loading the included shepherd-config.scm into my PID 1,
because I don’t need it and I don’t want to freeze my PID 1 due to bugs
:) But with a properly set GUILE_LOAD_PATH (including the Swineherd
modules) it should also work with PID 1.

-- 
Ricardo

Re: Implementing the guix-dameon in Guile

[Christopher Baines]

Vagrant Cascadian  writes:

> On 2023-09-13, Christopher Baines wrote:
>> I think this has been talked about for a while [1], but I want to make it
>> happen. Currently the guix-daemon is still similar to the nix-daemon
>> that it was forked from, and is implemented in C++. I think that a Guile
>> implementation of the guix-daemon will simplify Guix and better support
>> hacking on and around the daemon to add new features and move Guix
>> forward.
>>
>> 1: 
>> https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/ROADMAP.org#n71
> ...
>> Let me know if you've got any comments or questions!
>
> Sounds great!
>
> My only real concern, as someone maintaining guix packages in Debian, is
> to make sure that we do not break compatibility with being able to use
> an older daemon, as Debian stable/bookworm is still at guix 1.4.x and it
> would be nice to not have to force people to manually upgrade the daemon
> (e.g. and even if a newer version lands in a future Debian stable
> release, in general it will stuck using that version for some years as
> well).
>
> I have noticed occasional issues with the Debian packages of guix having
> compatibility issues when newer versions of guile-git/libgit2,
> guile-ssh/libssh2, etc. get introduced, and wonder if the same would
> hold true of a daemon?
>
> In Guix, by design you wouldn't really notice these sorts of problems as
> it is always generally built with the current version, but Debian does
> rely on ABI compatibility for package upgrades... I might be able to
> keep better track of these types of issues in Debian, although various
> guile-* modules that depend on C libraries seem to avoid the normal
> detection mechanisms to trigger rebuilds in Debian.
>
> That is a bit of a tangent, but it reminded me about that issue...

I think compatibility is a priority, and although I haven't looked too
much in to the details yet, I think it's quite realistic.

I think it's very important for Guix to keep compatibility with older
daemons, and on the daemon side, I'd want to see the Guile
implementation work with older Guix's, as well as it being possible to
switch back and forth between the implementations without issue.


signature.asc
Description: PGP signature

Can we provide another UI for patches than just email ?

[Edouard Klein]

Dear Guix,

I've just blown up Guix's debbugs by opening (and then closing) 33 bugs
instead of sending a 33-commits patch series.

I've also seen that there is a huge discussion on the cognitive overhead
of contributing. I will read it as soon as I can spare more time on this.

I find git email's workflow extremely frustrating and confusing. I know
it works for most of you who actually commit, but I would like to point
out that I'm one more person to be tripped by its complexity, and who
cares enough to say so here. There are probably even more people who
don't speak up.

Therefore, there is a huge cost in keeping up using it: less people will
be able to provide patches, and software will get stale.

For example, ou current Go version is 1.17 ! From 2021 ! The current is
1.21. I suspect that if contributing was easier, we wouldn't have a 2
years delay in our versions.

I do not want to open a flamewar about the merits of git email. I want
to point out that I personnaly find it difficult and that I'm not the
only one.

Now, do we want to address this as a community, and welcome developers
who may have good patches to contribute, but who can't spare at least ten
hours to get confortable with git email ?

An alternative solution would be to provide a public or semi-public git
remote, and just let people git push their branches there.

I suspect a git hook can then provide integration with debbugs if
necessary, and I would be willing to assist writing it.

In the current state, the UX is just unbearable, and I have in the past
(and likely will again) just not bothered sending patches because the
experience is so time consuming and infuriating.

Before anybody tries to explain to me that git send-email is easier than
I think, what you have to beat is:
git remote add guix-patches WHATEVER #only once
git push -u guix-patches master:some-unique-name

You can then push, pull, rebase, whatever, from the command line (or
magit in my case), without leaving your dev context. Can't be easier
than that.

Sorry about blowing up the debbugs. I'll do my best to avoid it in the
future.

Cheers,

Re: Implementing the guix-dameon in Guile

[Christopher Baines]

Maxim Cournoyer  writes:

>> The transformation toward a Guile daemon is a point of consistency and
>> pride for the project and therefore unlikely to be second-guessed or
>> reverted. My recommendation is to replace the daemon gradually—working
>> from (apply system* (command-line) downward—and mainline your
>> incremental changes as quickly as possible.
>
> I had some musing about the daemon recently; I was thinking libguile
> could be added to our old C++ daemon, which could then replace its
> functions piece-wise with Scheme implemented ones?
>
> At the end, we'd have almost everything in Scheme, at which point we
> could rewrite the main loop to Scheme and say farewell to the old
> daemon.
>
> Would that approach makes any sense?  I know of at least one C++ project
> integrating libguile, and that's the jami-daemon, so it could perhaps
> provide some clues as to how to proceed to do so.

I think it's a viable approach, but probably not one for me. Since I
don't really write C++, I think that trying to replace parts of the C++
code and keep things working would take more work than just writing
Guile.


signature.asc
Description: PGP signature

Re: How can we decrease the cognitive overhead for contributors?

[Ricardo Wurmus]

Fannys  writes:

>> But again, even if this is a great option for you, it might be a really bad
>> option for some other people. Everybody does not have the time to spend
>> learning emacs, or other specific tool. It's ok if the workflow suggests that
>> but it's not great if we have no other alternative.
>>
>> It's not accessible and imposes a barrier in some people.
>
> Yeah agreed. And we should be consious of that.
> Ironically by mandating Emacs and Email we force people to use specific
> tools while at the same time even though the same people will complain(!) 
> against vendor lock-in
> like github.

We don’t *mandate* the use of Emacs.  It’s just a common recommendation
because it works so well with text and is trivially extensible, so it’s
a common target for helper tools.  Surely we also wouldn’t call a
recommendation to use a shell script “vendor lock-in” just because it
needs Bash.

Emacs works well with text, and text is all that’s needed in a
patch-based workflow, which is in fact vendor agnostic.

Of course this doesn’t mean that it is as accessible as we’d want.

-- 
Ricardo

Re: How can we decrease the cognitive overhead for contributors?

[Ricardo Wurmus]

MSavoritias  writes:

> Simon Tournier  writes:
>
>> Hi,
>>
>> On Tue, 29 Aug 2023 at 12:53, MSavoritias  wrote:
>>
>>> Do you know if there are any plans to write a scheme bug/patching
>>> system? Because looking a bit into it, it doesn't seem like its that
>>> actively developed so maybe we would be better served by one in scheme.
>>> Or Sourcehut of course as somebody wrote in another email.
>>> Since not much would change with sr.ht anyways.
>>
>> The only work I know is named tissue by Arun Isaac.  Arun also
>> contributes to Mumi (one web front-end of the venerable Debbugs instance
>> of the GNU project).
>>
>> https://tissue.systemreboot.net/
>> https://archive.fosdem.org/2023/schedule/event/tissue/
>>
>> Well, I am not convinced by the argument « we would be better served by
>> one in scheme ».  For instance, Cuirass or Build Coordinator or even
>> Mumi, all are implemented in Scheme and they are receiving very few
>> contributions.
>>
>> https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git
>> https://git.savannah.gnu.org/cgit/guix/build-coordinator.git/
>> https://git.savannah.gnu.org/cgit/guix/mumi.git/
>>
>>
>> Cheers,
>> simon
>
> Its more about:
> If debbugs has issues and we dont fully control it and can make those
> improvements, why not something in scheme instead of sr.ht?

It’s an independent axis.  Limitations in Debbugs cannot be fixed
centrally because we don’t control the server.  If we did decide to
operate a bug tracker by ourselves it would suit us to pick one that
didn’t also require a lot of effort to work around annoyances.

The fact that we’ve used Scheme for a number of tools that don’t see
many contributions aside from those who initially wrote them is a good
reminder to adjust our expectations.


-- 
Ricardo

Re: to PR or not to PR, is /that/ the question?

[Simon Tournier]

Hi,

On Thu, 14 Sept 2023 at 08:53, Giovanni Biscuolo  wrote:

> To remain /in topic/: I think that adopting a «web based PR model» will
> definitely _not_ decrease the cognitive overhead for contributors.

What could potentially decrease the cognitive overhead for
contributors when using a "web based PR model" is all the usual glue
(services) around this very same "web based PR model".  Somehow, the
discussion is about these services behind the web-button.  I agree
with Katherine, quoting [1]:

I know everyone is focusing on email vs. web-forge, but I am trying to
draw attention to the root causes of the complexity, and enumerating
possible solutions to these.

IMHO, I think we have identified some of these causes of complexity.

Last, let me say it again: I am perfectly happy with the email workflow. :-)

Cheers,
simon


1: Re: How can we decrease the cognitive overhead for contributors?
Katherine Cox-Buday 
Tue, 05 Sep 2023 13:15:48 -0600
id:13416fee-8c7d-b145-48b9-0fbec2251...@gmail.com
https://lists.gnu.org/archive/html/guix-devel/2023-09
https://yhetil.org/guix/13416fee-8c7d-b145-48b9-0fbec2251...@gmail.com

Re: [workflow] Automatically close bug report when a patch is committed

[Andreas Enge]

Hello,

Am Wed, Sep 13, 2023 at 09:14:52PM +0200 schrieb Liliana Marie Prikler:
> I do wonder how the ChangeId would work in practice.  Since it's not
> really assigned by the committer, it would have to be generated "on the
> fly" and attached to the mail in between, which could result in all
> kinds of nasty behaviour like unstable Ids or duplicated ones.

this one would be easy to solve: it could just be the hash of something.
For instance, it could actually be the commit "number" itself of the
commit when it is first created.

Andreas

Re: Implementing the guix-dameon in Guile

[Simon Tournier]

Hi,

Really cool!  Congrats Chris! :-)

On Wed, 13 Sep 2023 at 13:13, Maxim Cournoyer  wrote:

> I had some musing about the daemon recently; I was thinking libguile
> could be added to our old C++ daemon, which could then replace its
> functions piece-wise with Scheme implemented ones?

Maxim, you are proposing to keep the C++ and gradually replace some
parts by implementing these parts in Guile and add a C layer via
libguile to call the Guile code from the C++, right?

If I remember correctly from the last discussion on Guile guix-daemon, I
proposed the converse.  However, the C layer of Maxim’s proposal seems
much easier. ;-)

Some gradual replacements seem a good strategy for the feedback loop.
However, the final design would be more or less the same as the current
one.  Although, I do not know if another design would be possible. :-)

Chris, all the best for this challenging project!  It’s nice that you
have a grant and I am convinced all the ingredients are here for making
this project a success. :-)

Cheers,
simon

Re: Cuirass actors

[Simon Tournier]

Hi,

It is really cool! :-)

On Wed, 13 Sep 2023 at 23:08, Ludovic Courtès  wrote:

>   - The "builder" spawns derivation builds.  There are currently two
> implementations: the local builder sends build requests to the local
> 'guix-daemon' process, while the remote build delegates builds to
> 'cuirass remote-server'.

[...]

> With this actor split, one could implement another “builder” backend,
> for instance one that talks to a Build Coordinator process.

IIUC, the current two “builder” backend are:

 + local ’guix-daemon’: the queue of derivations is processed using one
   strategy – the one implemented in C++,

 + ’cuirass remote-server’: the queue of derivations is processed using
   another strategy – implemented in Guile relying on ZeroMQ (something
   like steal of work, if I remember correctly).

Is it correct?

The Build Coordinator also implements the other actors “channel
updater”, “evaluator”, etc., right?  From my rough understanding, the
first aim of Build Coordinator is the implementation of the “builder”.
Is it correct?

My two questions are:

 1. Is the Build Coordinator able to only process the queue of
derivations?  (being a “builder” backend).  And if yes, what is its
strategy for processing?

 2. In this picture of actor model, would it make sense (or would it be
possible) to replace the “builder” actor from the Cuirass one to the
Build Coordinator one?

Cheers,
simon

 

Re: to PR or not to PR, is /that/ the question?

[Giovanni Biscuolo]

Hi Simon

Simon Tournier  writes:

> Thank you for your detailed message.  Sorry, I have not read all the
> details because I have been lost.

Sorry!  Forgive me since I am not able to summarize the details without
losing content.

> What do you want to explain?

In the context of this increasing cognitive overheading thread...

As I wrote: «I don't think thay adopting a «web based PR model»
(whatever the term "pull request" means, editor note) for the Guix
project is a good idea.»

To remain /in topic/: I think that adopting a «web based PR model» will
definitely _not_ decrease the cognitive overhead for contributors.

Also, replacing (or adding to) the current "integration request" email
based workflow with a "web based PR model" (that _imply_ a web based
patch review strictly connected with the choosen "forge"?) would be an
/overwelming/ cognitive overhead for all.

Last but not least, I also think that adding a patch review system like
Gerrit or Gerrit, Phabricator, and Review Board would not only
_increase_ the overall cognitive overhead but is also /technically/
_broken_.

Why I think all this things?  Well, I'm sorry but it's a _very_ long
story and is all written down in the /details/.
(id:87y1ha9jj6@xelera.eu aka 
https://yhetil.org/guix/87y1ha9jj6@xelera.eu/)

[...]

I'm done with this thread, sorry; I'll drift alone :-) 

Happy hacking, Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures


signature.asc
Description: PGP signature

Re: Swineherd: Guix System container manager

[Tanguy LE CARROUR]

Hi Simon,

Quoting Simon Tournier (2023-09-13 14:58:38)
> On Wed, 13 Sep 2023 at 11:06, Ricardo Wurmus  wrote:
> [...]
> >
> > The Swineherd was designed to be used with Shepherd on foreign distros,
> > so it does not assume to be running on top of Guix System (for better or
> > worse).
> 
> Oh!  It is very promising.  Really cool!
> 
> > Of course the Swineherd is also available as a Guix package called
> > “swineherd”.
> 
> Hum, I did (or adding guile):
> 
> guix time-machine -q -- shell swineherd
> 
> Then I do not know what to do.  What are the basic steps for testing it?

I haven't tried it (yet), but I've read this: 
.

Regards,

-- 
Tanguy