bug#58697: [bug] guix refresh nftables crashes

[Maxim Cournoyer]

Hi,

宋文武  writes:

> Maxime Devos  writes:
>
>> On 22-10-2022 03:55, kiasoc5 via Bug reports for GNU Guix wrote:
>>> % guix refresh nftables
>>> [...]
>>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>>> In procedure getaddrinfo: Servname not supported for ai_socktype
>>> ```
>>
>> I can reproduce this locally and don't know the cause.
>>
>
> It happens when the origin have both 'mirror://' and 'http://' urls,
> current the html updater check for any url match 'http' or 'https', but
> when updating it will just pick the first url, so when the first is
> 'mirror://' this error will come.

I ended up with honoring the first element of the URI, since support for
mirror:// URIs in (guix gnu-maintenance) was added recently (see commit
bdaef69556f68595e5ec0db1710bf8ad208abe20, "gnu-maintenance: Allow mirror
URLs to fallback to the generic HTML updater.").

The fix is pushed with commit 2a7f031ca9.  Let me know if anything is
still causing problems!

-- 
Thanks,
Maxim

bug#65804: Bug report: "You found a bug"

[Lasse Schlör]

Hello,

The command that ran into this bug was:
`guix shell --manifest=guix-psychopy-manifest.scm`

The contents of `guix-psychopy-manifest.scm` were as follows:
```
(use-modules (guix inferior) (guix channels)
 (srfi srfi-1))

(define channels
  (list (channel
 (name 'guix)
 (url "https://git.savannah.gnu.org/git/guix.git;)
 (commit
  ;; Last commit with Python 3.8
  "f1eea19c9ae27e5d275b083bbf280e5b59e5e57a"

(define inferior
  (inferior-for-channels channels))

(packages->manifest
 (list (first (lookup-inferior-packages inferior "python"))
   (specification->package "git")))
```

The command fetched/downloaded a lot of packages without issues. Then, the 
following output happened:
[…]
```
fetching path `/gnu/store/rw5z0mmdlcwkdzvr94bgihwlnhzwyky4-bzip2-boot0-1.0.8'...
-guix substitute: warning: while fetching 
https://ci.guix.gnu.org/nar/gzip/rw5z0mmdlcwkdzvr94bgihwlnhzwyky4-bzip2-boot0-1.0.8:
 server is somewhat slow
guix substitute: warning: try `--no-substitutes' if the problem persists
guix substitute: warning: download from 
'https://ci.guix.gnu.org/nar/gzip/rw5z0mmdlcwkdzvr94bgihwlnhzwyky4-bzip2-boot0-1.0.8'
 failed, trying next URL
-guix substitute: warning: while fetching 
https://ci.guix.gnu.org/nar/lzip/rw5z0mmdlcwkdzvr94bgihwlnhzwyky4-bzip2-boot0-1.0.8:
 server is somewhat slow
guix substitute: warning: try `--no-substitutes' if the problem persists
guix substitute: warning: download from 
'https://ci.guix.gnu.org/nar/lzip/rw5z0mmdlcwkdzvr94bgihwlnhzwyky4-bzip2-boot0-1.0.8'
 failed: 504, "Gateway Time-out"
retrying download of 
'/gnu/store/rw5z0mmdlcwkdzvr94bgihwlnhzwyky4-bzip2-boot0-1.0.8' with other 
substitute URLs...
guix substitute: warning: download from 
'https://ci.guix.gnu.org/nar/gzip/rw5z0mmdlcwkdzvr94bgihwlnhzwyky4-bzip2-boot0-1.0.8'
 failed, trying next URL
guix substitute: warning: download from 
'https://ci.guix.gnu.org/nar/lzip/rw5z0mmdlcwkdzvr94bgihwlnhzwyky4-bzip2-boot0-1.0.8'
 failed: 504, "Gateway Time-out"
guix substitute: error: failed to find alternative substitute for 
'/gnu/store/rw5z0mmdlcwkdzvr94bgihwlnhzwyky4-bzip2-boot0-1.0.8'
fetching path `/gnu/store/rw5z0mmdlcwkdzvr94bgihwlnhzwyky4-bzip2-boot0-1.0.8' 
(empty status: '')
fetching path 
`/gnu/store/gcl0d5i1hfg4s0fv21dgaf7r5z3i16zd-coreutils-boot0-8.32'...
Backtrace:
  15 (primitive-load 
"/gnu/store/fnyqpbdxbm4ipwvq9vfmxpkphdlqjcg3-compute-guix-derivation")
In ice-9/eval.scm:
155:9 14 (_ _)
159:9 13 (_ #(#(#(#(#(#(#(#(#(#(#(#(#(# 
?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?))
In ./guix/store.scm:
  2049:24 12 (run-with-store # 
# ?)
\   1883:8 11 (_ #)
In ./guix/gexp.scm:
   258:18 10 (_ #)
   1123:2  9 (_ #)
982:2  8 (_ #)
843:4  7 (_ #)
In ./guix/store.scm:
  1931:12  6 (_ #)
   1358:5  5 (map/accumulate-builds # 
# ?)
  1369:15  4 (_ # 
("/gnu/store/jwi2rqsz16lipgfl1h3084z8hiincnwv-guile-git-?" ?) ?)
  1369:15  3 (loop #f)
   715:11  2 (process-stderr # _)
In ./guix/serialization.scm:
   101:11  1 (read-int #)
 79:6  0 (get-bytevector-n* # 8)

./guix/serialization.scm:79:6: In procedure get-bytevector-n*:
ERROR:
  1. :
  file: #f
  port: #
guix shell: error: You found a bug: the program 
'/gnu/store/fnyqpbdxbm4ipwvq9vfmxpkphdlqjcg3-compute-guix-derivation'
failed to compute the derivation for Guix (version: 
"f1eea19c9ae27e5d275b083bbf280e5b59e5e57a"; system: "x86_64-linux";
host version: "6113e0529d61df7425f64e30a6bf77f7cfdfe5a5"; pull-version: 1).
Please report it by email to .
```

As requested by Guix, I am reporting the bug with this email.

I assume this bug happened upon a temporary loss of the internet connection. 
Running the command anew continued fetching/downloading just fine.

Kind regards
Lasse Schlör

bug#65847: system container gathering entropy takes forever

[oscar . quijano--- via Bug reports for GNU Guix]

New containers get stuck with a message similar to the following one:  

  

guile: warning: failed to install locale  

system container is running as PID 85878W  

ARNING: (guile-user): imported module (guix build utils) overrides core
binding `delete'  

Run 'sudo guix container exec 85878 /run/current-system/profile/bin/bash
--login'  

or run 'sudo nsenter -a -t 85878' to get a shell into it.  

  

WARNING: (guile-user): imported module (guix build utils) overrides core
binding `delete'  

making '/gnu/store/gkqmm80naf3zw2n20ml11q7xb2nbnglg-system' the current
system...  

WARNING: (guile-user): imported module (guix build utils) overrides core
binding `delete'  

setting up setuid programs in '/run/setuid-programs'...  

populating /etc from /gnu/store/wigi6gny24gpk2inqy19xswsbplqa6fc-etc...  

WARNING: (guile-user): imported module (guix build utils) overrides core
binding `delete'  

WARNING: (guile-user): imported module (guix build utils) overrides core
binding `delete'  

Please wait while gathering entropy to generate the key pair;  

this may take time...  

  

and the message stays there forever, I have left it running even for more than
half an hour and it doesn't move from there. This is happening even with basic
system container definitions. The previos message was generated when running a
system container with the following system definition:  

  

(use-modules (gnu)  

(gnu services web))  

  

(operating-system  

  

(host-name "container")  

  

(timezone "Europe/Berlin")  

  

(file-systems (cons (file-system  

(device (file-system-label "does-not-matter"))  

(mount-point "/")  

(type "ext4"))  

%base-file-systems))  

  

(bootloader (bootloader-configuration  

(bootloader grub-bootloader)  

(targets '("/dev/sdX"  

  

(services %base-services))  

  

  

I also tried sharing /dev/random and /dev/urandom with the host and it still
gets stuck there.  

Any ideas about what could be causing this?  

  

bug#55907: VFIO kernel module fails to capture PCI device

[Lars Rustand]

Hello Nick,

Did you ever figure this out? I am struggling with the same problem.


Thank you,

- Lars

bug#65770: xdg-desktop-portal-kde only works if system is reconfigured with gnome-desktop-service-type

[Zheng Junjie]

Hello, can you try it?

>From 09104e9b8f13868751dc2d5f514b01a6c286e790 Mon Sep 17 00:00:00 2001
Message-ID: <09104e9b8f13868751dc2d5f514b01a6c286e790.1694010267.git.zhengjun...@iscas.ac.cn>
From: Zheng Junjie 
Date: Wed, 6 Sep 2023 22:23:11 +0800
Subject: [PATCH] gnu: xdg-desktop-portal-kde: Add miss input.

* gnu/packages/freedesktop.scm (xdg-desktop-portal-kde): Add miss input.
[propagated-inputs]: add xdg-desktop-portal.
---
 gnu/packages/freedesktop.scm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gnu/packages/freedesktop.scm b/gnu/packages/freedesktop.scm
index 0a1c9bffb3..e60bff05dd 100644
--- a/gnu/packages/freedesktop.scm
+++ b/gnu/packages/freedesktop.scm
@@ -2936,6 +2936,8 @@ (define-public xdg-desktop-portal-kde
   libxkbcommon
   kio-fuse
   wayland-protocols))
+(propagated-inputs
+ (list xdg-desktop-portal))
 (synopsis "Backend implementation for xdg-desktop-portal using Qt/KF5")
 (description "This package provides a backend implementation
 for xdg-desktop-portal that is using Qt/KF5.")

base-commit: 6113e0529d61df7425f64e30a6bf77f7cfdfe5a5
-- 
2.41.0

bug#65774: python updater clears inputs, leaves propagated-inputs empty

[Sergio Pastor Pérez]

Hi,

I'm afraid you lost me here. Why is the `inputs' field empty? I don't
understand what you meand with the checking of 'python-'.

Keep in mind that I have very little experince with python packaging so
I'm may not understand you due to my lack of background.


Maxim Cournoyer  writes:

> Hi,
>
> I've had the Python updater produce this when attempting to update
> fontmake:
>
> gnu/packages/fontutils.scm:780:2: warning: fontmake: 'propagated-inputs'
> field not found; leaving it unchanged
>
> gnu/packages/fontutils.scm:780:2: warning: fontmake: expected
> 'propagated-inputs' value: (python-attrs python-fontmath
> python-fonttools python-glyphslib python-ufo2ft python-ufolib2)
>
> --8<---cut here---start->8---
> modified   gnu/packages/fontutils.scm
> @@ -779,16 +779,16 @@ (define-public psautohint-font-data
>  (define-public fontmake
>(package
>  (name "fontmake")
> -(version "3.4.0")
> +(version "3.7.1")
>  (source (origin
>(method url-fetch)
>(uri (pypi-uri "fontmake" version ".zip"))
>(sha256
> (base32
> -"0fc5c9csjpy1aa4c03p7nvjgls5wjplhmmf42n0cmvrlh6cm7wl3"
> +"0ib7fvwgwazm7qfj4a3rkqkb40xfbj40rnvsmkvl2isg2ky3vg9m"
>  (build-system python-build-system)
> -(inputs (list python-fontmath python-glyphslib))
> -(native-inputs (list unzip python-setuptools-scm))
> +(inputs (list))
> +(native-inputs (list zip))
>  (home-page "https://github.com/googlefonts/fontmake;)
>  (synopsis
>   "Compile fonts from sources (UFO, Glyphs) to binary (OpenType, 
> TrueType)")
> --8<---cut here---end--->8---
>
> The choice of using inputs here was conscious, as it is a command, not a
> library.  Perhaps it could check if the name starts with 'python-' or
> not?  It's a bit 'magic', but it would help.

bug#65391: Acknowledgement (People need to report failing builds even though we have ci.guix.gnu.org for that)

[Dr. Arne Babenhauserheide]

Hi,


I’m skipping a lot to get only to the most important points (save time
for us all).

Simon Tournier  writes:
> Instead, it is QA that builds “pre-commit“ (patches).  Thanks to
> tireless Chris’s work since years, we have some tools for monitoring the
> impact of one change on the whole package set.  Somehow, if I have
> correctly understood, QA uses the Build Coordinator to list all the
> derivations and then build all the new ones generated by the change.
>
> So the answer to your question is yes. :-) Aside, help is welcome for
> improving QA.

So something was missing there that let the change to the ocaml package
slip through this january. This should have raised red flags somewhere.

Do we have documentation on the process? (link?)

>> Since a manifest is strictly dependent on all packages defined in it,
>> removing a single referenced package means that the manifest is broken:
>> no update works anymore. No security updates come in anymore — even if
>> the package in question worked locally. This is a situation we should
>> not cause.
…
> What I am proposing is: if the same package is still failing after
> several X  or attempts, then we mark it as ‘broken’ and it
> becomes a candidate for a removal.  People who care raise their hand.
> And we have a better idea about the real status.

This means with the current functionality that the manifest is broken at
that point. Nothing can be updated anymore. I’ve been in that situation
a few times already with broken packages and it caused weeks of not
being able to update because I didn’t have the time to investigate.

That’s why I wrote the following:

> If we had a way to have placeholder packages (similar to the renamings)
> that emit warnings for missing packages but do not break the build, that
> would reduce the damage done by removing a package. But I think such a
> mechanism must be in place and tested before adding a rule to remove
> packages.

This would cause us to collect a slowly growing list of removed packages
that will be ignored (except for the warning) in manifests.

That way we would avoid breaking the setup when removing a package.


(define-public-removed the-package-variable
  (removed-package
(name "the-package-name")
(reason-for-removal "upstream stopped working a decade ago")))


The key difference between your scenario "some package is broken and I
cannot install it" and my scenario "I have a package in my manifest that
gets removed, breaking my manifest" is that mine is much more painful
because an update breaks changing a working system.

In my scenario I don’t just see "oh, this doesn’t work, let’s choose
another way", but a way I’ve been using and building on gets broken.

Also I experienced that at least twice already. That I had to go and
investigate before I could add a package to my manifest, because the
manifest was broken by a removed package. In at least one instance I had
not been able to update for several weeks before that and didn’t have
time and energy to investigate.

Once I had missed that my system had not updated in months, because I
did reconfigure in a cron job and a removed package had broken
/etc/config.scm


And we actually select for such breakage, because I cannot see locally
whether a package failed on CI, so while I can see (and have to fix)
packages that fail locally, on-CI-failures are invisible.


So instead of removing a package, I think the first step in a process
should be to warn everyone with that package in the manifest that it’s
broken on CI ⇒ add a warning to that package, like the rename warnings.

If no one takes it up for a few months, replace it with a
removed-package placeholder that warns to clean up the manifest. And
just keep that placeholder in place to avoid breaking manifests.


Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de


signature.asc
Description: PGP signature

bug#51540: Update

[Charles via Bug reports for GNU Guix]

Hello, is there any update on this? It seems that the help tags are still not 
available for plugins.

bug#65391: People need to report failing builds even though we have ci.guix.gnu.org for that

[Csepp]

Simon Tournier  writes:

> Hi,
>
> On Mon, 11 Sept 2023 at 09:33, Csepp  wrote:
>
>> That is not a package problem but a Guix interface problem.  I have been
>> saying for a while that there needs to be an option to disable all
>> non-trivial local builds by default when you know your machine can't
>> handle them.
>
> IMHO, your proposal is orthogonal with the issue at hand: broken
> packages.  Other said, the issue is: how to deal with the set of
> packages that will not build and we already know it (since weeks,
> months or even years for some).
>
> My workstation can handle all the compilations that are required.  My
> laptop is able offload to it.  The issue about broken packages is not
> about the resources.  It is about burning resources for nothing.
>
> About the issue you are speaking about, we already had discussions in
> this direction -- you are not the only one saying "the fix needs to do
> X" for a while but please keep in mind that "talking does not cook the
> rice". ;-)  Well, maybe you could open a ticket with a concrete
> use-case.
>
> Cheers,
> simon

I was hoping to get some consensus on whether this is actually a
bug/feature that others consider worth tracking, so I kept discussion of
it mostly to guix-devel, but sure, I can make a proper issue for it.

bug#63331: bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts

[Ludovic Courtès]

Hi,

Vivien Kraus  skribis:

> Le lundi 11 septembre 2023 à 16:36 +0200, Ludovic Courtès a écrit :
>> Eventually, when users are all running recent versions of
>> ‘guix-daemon’ with support for “builtin:git-download” (2–4
>> years from now?), we’ll be able to use “builtin:git-download”
>> unconditionally and thus be sure there are no risks of
>> derivation cycles.
>
> Do foreign distros need to update their guix package as well? If that
> is the case, the provided time frame might be optimistic.

At some point, we can change clients to print a warning saying that
their daemon is outdated if it lacks “builtin:git-download”.  That
should help speed things up.

>> Note that the patch series adds a hard dependency on Git.
>> This is because the existing ‘git-fetch’ code depends on Git
>
> I applaud the switch to the regular git program from libgit2, as I
> would then be able to pull from my cgit "dumb" server instead of having
> to maintain a mirror.

Nothing changes here: ‘git-fetch’ already uses Git.

Ludo’.

bug#63277: python-anaconda-client and conda fail

[Simon Tournier]

Hi,

On Mon, 11 Sep 2023 at 10:08, Hugo Buddelmeijer  wrote:

> The conda package seems to be fixed, and python-anaconda-client also
> builds. So I think this can be closed.

Thanks for the follow-up.  I am closing.

Feel free to reopen if I misread.

Cheers,
simon

bug#65391: Acknowledgement (People need to report failing builds even though we have ci.guix.gnu.org for that)

[Simon Tournier]

Hi Arne,

( I have not re-read all the thread. )

On Mon, 11 Sep 2023 at 10:30, "Dr. Arne Babenhauserheide"  
wrote:

>> Well, I do not think that any policy will mark a package for removal on
>> the first build failure.  However, if the same package is still failing
>> after several X  or attempts, it means something is wrong.
>> Marking it as a candidate for removal implies:
>>
>>  1. check if the failure is from CI when it builds locally,
>>  2. keep a set of packages that we know they are installable.

> This is a good example, but not for removing broken packages. For
> perl6-xml-writer removing the package would keep breakage in Guix.
>
> I just checked the build, and this looks like a Guix packaging error

This is exactly the effect if we have a policy. :-)

Please, do not read a policy for the removal of broken packages as an
automatic process.  As you, I think an automatic process for removing
would be a bad thing about the user experience.

Maybe I misunderstand what a policy is.  For me, a policy is a plan that
is used as a basis for making decisions, a policy helps in reaching
conclusion which then can lead to some actions.

Somehow this discussion is the implementation of the policy I am
proposing and that would help the maintenance, IMHO. I have manually
marked this package for removal and…

> that breaks the tests due to a change to some unrelated package:

…surprise, surprise, someone has checked. :-)

A policy for removal about the broken packages would allow to know what
to do.  If the same package is still failing after several X 
or attempts, it means something is wrong.

Currently, either you hit a broken package when doing some Guix
operations.  And that is a very poor experience, IMHO.  Either one have
to open the dashboard from CI [1], select some red buttons and
investigate.  And we can count with few fingers the number of people
doing that.

1: https://ci.guix.gnu.org/eval/741273/dashboard


> Disabling the tests makes the package build and work.

Here is the point of my proposal to have a policy for removal of broken
packages: automatically check how many times they have failed to build
and automatically tag them when they are considered problematic.  If no
one care and these tagged packages are not fixed, then let remove them.

It would drastically help in the maintenance.  Otherwise, your help is
very welcome in monitoring all the failures. :-)


> So here, removing a package would start at the wrong place: some change
> between 2021-02-01 and 2021-04-30 broke the perl6-tap-harness and we did
> not detect that.

Yes, that’s where QA should help: detect unrelated change that have a
long distance impact on unrelated packages.

Changes to the branching/commit policy
Christopher Baines 
Thu, 08 Jun 2023 15:24:37 +0100
id:87y1kuyqew@cbaines.net
https://yhetil.org/guix/87y1kuyqew@cbaines.net
https://lists.gnu.org/archive/html/guix-devel/2023-06

[bug#63459] [PATCH] doc: Rewrite the branching strategy.
Christopher Baines 
Fri, 12 May 2023 08:55:20 +0100

id:f339d15842370b97558b704593848e318462b68d.1683878120.git.m...@cbaines.net

https://yhetil.org/guix/f339d15842370b97558b704593848e318462b68d.1683878120.git.m...@cbaines.net

https://issues.guix.gnu.org/msgid/f339d15842370b97558b704593848e318462b68d.1683878120.git.m...@cbaines.net



> This does not mean that there will never be a case in which a package
> has to be removed, but given that both cases you showed are likely
> self-induced breakage due to changes that should have been rejected as
> breaking seemingly unrelated packages, it rather looks like the
> situation where removing the package is the right way forward is the
> exceptional case.

We are miscommunicating.  Or we have a very different vision about what
should be the reliability of Guix.

As a regular user, I need perl6-tap-harness, so I type “guix install
perl6-tap-harness”, and bang, it fails.

As a regular user, I do not mind if the problem is coming from some
change between 2021-02-01 and 2021-04-30 or if it comes from something
else.  What I want is that “guix install perl6-tap-harness” just works.

Having a clear policy for removal – again not an automatic removal
procedure – would help all, IMHO.


> The norm is that our CI should have detected a problem in the commit
> causing the breakage.
>
> Can we automatically rebuild all inheriting packages when a package gets
> changed?

CI builds all the commits pushed to Savannah.  Not exactly all but
that’s another story and it does not matter for this discussion.

AFAIK, no one is checking that the commit they are pushing does not lead
to break something.  Else they would not push it I guess. ;-)

Instead, it is QA that builds “pre-commit“ (patches).  Thanks to
tireless Chris’s work since years, we have some tools for monitoring the
impact of one change on the whole package set.  Somehow, if I have
correctly 

bug#65832: [PATCH] guix: shell: Don't whitelist / by typo in `shell-authorized-directories'.

[Ludovic Courtès]

Hi,

Janneke Nieuwenhuizen  skribis:

> From: Janneke Nieuwenhuizen 
> Date: Wed, 6 Sep 2023 10:52:17 +0200
> Subject: [PATCH] guix: shell: Don't whitelist / by typo in
>  `shell-authorized-directories'.
>
> Fixes 
>
> * guix/scripts/shell.scm (authorized-shell-directory?): After warning,
> continue LOOP to return valid query result for DIRECTORY.

Thanks a lot for finding, reporting, and fixing this issue!

Ludo’.

bug#63331: Guile-GnuTLS/Git circular dependency and built-in git checkouts

[Vivien Kraus via Bug reports for GNU Guix]

Hello!

Le lundi 11 septembre 2023 à 16:36 +0200, Ludovic Courtès a écrit :
> Eventually, when users are all running recent versions of
> ‘guix-daemon’ with support for “builtin:git-download” (2–4
> years from now?), we’ll be able to use “builtin:git-download”
> unconditionally and thus be sure there are no risks of
> derivation cycles.

Do foreign distros need to update their guix package as well? If that
is the case, the provided time frame might be optimistic.

> Note that the patch series adds a hard dependency on Git.
> This is because the existing ‘git-fetch’ code depends on Git

I applaud the switch to the regular git program from libgit2, as I
would then be able to pull from my cgit "dumb" server instead of having
to maintain a mirror.

Vivien

bug#65720: Guile-Git-managed checkouts grow way too much

[wolf]

On 2023-09-08 19:08:05 +0200, Ludovic Courtès wrote:
> Hello!
> 
> Josselin Poiret  skribis:
> 
> > Right, although I wouldn't necessarily say that the former doesn't have
> > a proper API, but rather that it has a Unix-oriented API.  That leads to
> > performance issues on e.g. Windows but on Linux I'm not sure there's
> > much of a difference.
> 
> [...]
> 
> > We could consider replacing the guile-git dependency with another
> > library built directly on top of git-minimal, and have this be a
> > dependency of Guix.  Not ideal though, and not really scalable either:
> > we can't just add every VCS as direct dependencies.
> 
> I cannot imagine a viable implementation of things like ‘commit-closure’
> and ‘commit-relation’ from (guix git) done by shelling out to ‘git’.

I am sure I must be missing some part of the contract of the function, but at
least the commit-relation seems fairly straightforward:

(define (shelling-commit-relation old new)
  (let ((h-old (oid->string (commit-id old)))
(h-new (oid->string (commit-id new
(cond ((eq? old new)
   'self)
  ((zero? (git-C %repo "merge-base" "--is-ancestor" h-old h-new))
   'ancestor)
  ((zero? (git-C %repo "merge-base" "--is-ancestor" h-new h-old))
   'descendant)
  (else
   'unrelated

I would argue it is even somewhat more readable than the current implementation.

> I’m quite confident this would be slow

My version is ~2000x faster compared to (guix git):

Guix: 1048.620992ms
Git:  0.532143ms

Again, I am sure I must have miss something, either in the implementation or in
the measurements, because it is pretty hard to believe there is so much room for
improvement.

The full script I used is attached to this email.

> and brittle.

In general git plumbing command are design to have stable CLI interface in order
to be usable in scripting.  So I am not sure where the brittleness would come
from.

> 
> It looks like there’s no option other than carrying the two
> implementations.

Assuming I made no mistake (hard to believe), it is probably worth exploring the
feasibility of just shelling out to the git binary some more.

> 
> ~~~
> 
> Years ago, Andy Wingo sketched a plan for GNU hackers to implement Git
> in pure Scheme.  That was on April 1st though, so people mistakenly
> assumed it was a joke and the project was never carried out.
> 
> I digress, but I wonder: is there not even a viable Haskell or OCaml
> implementation of Git?
> 
> Thanks,
> Ludo’.
>

W.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.
#!/bin/sh
# -*-scheme-*-
exec guile -s "$0" "$@"
!#

(use-modules (git)
 (guix git))

(define %repo "/tmp/guix-fork")

(define h1 "72745172d155e489936f694d6b9013cb76272370")
(define h2 "6d60d7ccba5a8e06c17d55a1772fa7f4529b5eff")
(define h3 "c3db650680f995f0556d3ddce567cdc1c33e4603")

;;; r has to still be defined when the commit-relation is called.  There is *no*
;;; error, but it always returns 'unrelated.  Quite a footgun.
(define r (repository-open %repo))
(define c1 (commit-lookup r (string->oid h1)))
(define c2 (commit-lookup r (string->oid h2)))
(define c3 (commit-lookup r (string->oid h3)))

(define (git-C dir . args)
  (apply system* "git" "-C" dir args))

(define (shelling-commit-relation old new)
  (let ((h-old (oid->string (commit-id old)))
(h-new (oid->string (commit-id new
(cond ((eq? old new)
   'self)
  ;; In real code, git-C should probably return #t (for 0), #f (for 1)
  ;; or raise (for anything else).
  ((zero? (git-C %repo "merge-base" "--is-ancestor" h-old h-new))
   'ancestor)
  ((zero? (git-C %repo "merge-base" "--is-ancestor" h-new h-old))
   'descendant)
  (else
   'unrelated

;;; Make sure it actually works.
(let ((tests `((,c1 . ,c1)
   (,c1 . ,c2)
   (,c2 . ,c1)
   (,c1 . ,c3
  (for-each (λ (c)
  (format #t "Guix: ~a\nGit:  ~a\n\n"
  (commit-relation (car c) (cdr c))
  (shelling-commit-relation (car c) (cdr c
tests))

(define (time proc)
  (let* ((start (get-internal-run-time))
 (_ (proc))
 (end   (get-internal-run-time)))
(exact->inexact (* 1000 (/ (- end start) internal-time-units-per-second)

(format #t "Guix: ~ams\nGit:  ~ams\n"
(time (λ () (commit-relation c1 c2)))
(time (λ () (shelling-commit-relation c1 c2


signature.asc
Description: PGP signature

bug#65720: Guile-Git-managed checkouts grow way too much

[Ludovic Courtès]

Ludovic Courtès  skribis:

> It would also be pretty bad for closure size:
>
> $ guix size guile-git | tail -1
> total: 106.6 MiB
> $ guix size guile-git git-minimal | tail -1
> total: 169.8 MiB
>
> It’s also not clear concretely how we’d add that dependency.  Try
> invoking ‘git’ from $PATH and print a warning if it doesn’t work?

A solution to this particular problem is coming:

  https://issues.guix.gnu.org/65866

Ludo’.

bug#63331: Guile-GnuTLS/Git circular dependency

[Ludovic Courtès]

Ludovic Courtès  skribis:

> The longer-term solution is to add a “builtin:git-download” derivation
> builder, just like we have “builtin:download”.  The implementation
> should be relatively easy, but we’ll have to be able to deal with
> daemons that lack this builtin possibly for several years.

Patch available!

  https://issues.guix.gnu.org/65866

Ludo’.

bug#65716: bug#65718: Importing a toolchain packages causes top-level dependency cycles

[Maxim Cournoyer]

Hello,

Maxim Cournoyer  writes:

[...]

> A very simple reproducer on current master:
>
> modified   gnu/packages/firmware.scm
> @@ -43,6 +43,7 @@ (define-module (gnu packages firmware)
>#:use-module (gnu packages admin)
>#:use-module (gnu packages autotools)
>#:use-module (gnu packages assembly)
> +  #:use-module (gnu packages avr)
>#:use-module (gnu packages backup)
>#:use-module (gnu packages base)
>#:use-module (gnu packages bash)
>
>
> And then:
>
> $ ./pre-inst-env guix build hello

[...]

> ;;; loading /home/maxim/src/guix/gnu/packages/unicode.scm
> error: cross-gcc: unbound variable

There were multiple top-level problems in the avr and embedded modules,
which I've fixed in bug#65860 [0] (pending review).

I've added some documentation w.r.t. rules to avoid cyclic module
dependencies to hopefully educate people like myself about the problem
and how to avoid it in the future.

On a related note, I've also sent some improvement to guile-devel [1] so
that we can track the nested level of loaded modules when setting
'%load-verbosely' to #t.

[0]  https://issues.guix.gnu.org/65860
[1]  https://lists.gnu.org/archive/html/guile-devel/2023-09/msg00021.html.

-- 
Thanks,
Maxim

bug#52182: [cuirass] remote-worker process freeze

[Maxim Cournoyer]

Hi,

Ludovic Courtès  writes:

> Hello!
>
> I’m optimistically closing this bug because the implementation of
> ‘cuirass remote-worker’ has now changed to a single Fibers process:
>
>   https://lists.gnu.org/archive/html/guix-devel/2023-07/msg00096.html
>   
> https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=de8586080e04677cbe34c58f34715757ac61eea3

Thanks, this reads fantastic!

-- 
Thanks,
Maxim

bug#65787: time-machine is doing too much network requests

[Simon Tournier]

Oops, missing diff for clarity. :-)

On Mon, 11 Sep 2023 at 11:41, Simon Tournier  wrote:

> If yes, here two examples:

Adding ’pk’ where ’remote-fetch’ and ’branch-lookup’ are called.

diff --git a/guix/git.scm b/guix/git.scm
index 1cb87a45607b..0209826c5c00 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -234,8 +234,10 @@ (define (resolve-reference repository ref)
   (let resolve ((ref ref))
 (match ref
   (('branch . branch)
-   (let ((oid (reference-target
-   (branch-lookup repository branch BRANCH-REMOTE
+   (let ((oid (begin
+(pk 'branch-lookup 'NETWORK)
+(reference-target
+  (branch-lookup repository branch BRANCH-REMOTE)
  (object-lookup repository oid)))
   (('symref . symref)
(let ((oid (reference-name->oid repository symref)))
@@ -483,8 +485,10 @@ (define* (update-cached-checkout url
  ;; Only fetch remote if it has not been cloned just before.
  (when (and cache-exists?
 (not (reference-available? repository ref)))
-   (remote-fetch (remote-lookup repository "origin")
- #:fetch-options (make-default-fetch-options)))
+   (begin
+ (pk 'remote-fetch 'NETWORK)
+ (remote-fetch (remote-lookup repository "origin")
+   #:fetch-options (make-default-fetch-options
  (when recursive?
(update-submodules repository #:log-port log-port
   #:fetch-options (make-default-fetch-options)))

Cheers,
simon

bug#65787: time-machine is doing too much network requests

[Simon Tournier]

Hi Ludo,

> Maybe this is the purpose of your message:
> reducing Git remote accesses in those cases? 

Yes. :-)

On Sun, 10 Sep 2023 at 22:10, Ludovic Courtès  wrote:

>  It would probably be easier if you could come with
> an example where there’s Git-related network traffic where there
> shouldn’t.

Do we agree that the both Guile-Git procedures ’remote-fetch’ and
’branch-lookup’ are doing network traffic?

If yes, here two examples:

--8<---cut here---start->8---
$ ./pre-inst-env guix pull -q -p /tmp/new
Updating channel 'guix' from Git repository at 
'https://git.savannah.gnu.org/git/guix.git'...

;;; (remote-fetch NETWORK)

;;; (branch-lookup NETWORK)
Authenticating channel 'guix', commits 9edb3f6 to 2eb6df5 (83 new commits)...
Building from this channel:
  guix  https://git.savannah.gnu.org/git/guix.git   2eb6df5
--8<---cut here---end--->8---

Well, that’s not perfect… it becomes much worse:

--8<---cut here---start->8---
$ ./pre-inst-env guix time-machine -q -- describe

;;; (remote-fetch NETWORK)

;;; (branch-lookup NETWORK)

;;; (remote-fetch NETWORK)

;;; (branch-lookup NETWORK)
Updating channel 'guix' from Git repository at 
'https://git.savannah.gnu.org/git/guix.git'...

;;; (remote-fetch NETWORK)

;;; (branch-lookup NETWORK)
building 
/gnu/store/z8jwdgr7z6i8c00msdm2kzfv0n3zp25v-module-import-compiled.drv...
--8<---cut here---end--->8---

Do we agree that is suboptimal?


> ‘cached-channel-instance’ has 3 cases:
>
>   1. Obvious cache hit: This is when CHANNELS specifies the commit of
>  each target channel (this happens for example when you type ‘guix
>  time-machine -q --commit=a4c35c607cfd7d6b0bad90cfcc46188d489e1754)
>  *and* the combination of channels is already in
>  ~/.cache/guix/inferiors.  This is the optimal case: the Git repo
>  doesn’t even need to be opened.

Yes.


>   2. Cache hit: CHANNELS are pinned, but refer to tags (like “v1.2.0”)
>  or short commit IDs (like “a4c35c6”).  In that case,
>  ‘channel-full-commit’ opens the Git repo to resolve the identifier.
>  After that, we go to case #1 or #4.

That’s suboptimal.  Currently, it reads:

--8<---cut here---start->8---
$ guix describe
Generation 28   sept. 06 2023 14:54:50  (current)
  guix 6113e05
repository URL: https://git.savannah.gnu.org/git/guix.git
commit: 6113e0529d61df7425f64e30a6bf77f7cfdfe5a5

$ ./pre-inst-env guix time-machine -q --commit=6113e05 -- describe

;;; (remote-fetch NETWORK)

;;; (remote-fetch NETWORK)
Updating channel 'guix' from Git repository at 
'https://git.savannah.gnu.org/git/guix.git'...

;;; (remote-fetch NETWORK)
Computing Guix derivation for 'x86_64-linux'... |  C-c C-c
--8<---cut here---end--->8---

Using patch from #65352 [1], it removes these useless traffic:

--8<---cut here---start->8---
$ ./pre-inst-env guix time-machine -q --commit=6113e05 -- describe
Updating channel 'guix' from Git repository at 
'https://git.savannah.gnu.org/git/guix.git'...
Computing Guix derivation for 'x86_64-linux'... |  C-c C-c
--8<---cut here---end--->8---

Using the proposed patch, it becomes optimal, IMHO.  Well, let discuss
it in #65352 – comments are welcome. :-)

>   3. Cache hit: CHANNELS are not pinned—i.e., they refer to a branch,
>  not a commit.  In that case we first need to perform a ‘git fetch’
>  and then we go to #1 or #4.

I hope that I am convincing you that this case is suboptimal: it does 3
’git fetch’ and more 3 others lookup requiring network.  So it is about
6 network access when only one is necessary, IMHO.


>   4. Cache miss: ‘reference-available?’ returns #f for the channel
>  commits, we got through ‘remote-fetch’ followed by
>  ‘build-derivations’.

This case is part of #2 and discussed in #65352, IMHO.


Cheers,
simon


1: [bug#65352] [PATCH v2] DRAFT git: Avoid touching the network unless needed 
in 'reference-available?'.
Simon Tournier 
Wed, 06 Sep 2023 16:17:08 +0200
id:32d3fb5066e0b20e200dabef0fba897634e21dda.1694009405.git.zimon.touto...@gmail.com
https://yhetil.org/guix/32d3fb5066e0b20e200dabef0fba897634e21dda.1694009405.git.zimon.touto...@gmail.com
https://issues.guix.gnu.org/msgid/32d3fb5066e0b20e200dabef0fba897634e21dda.1694009405.git.zimon.touto...@gmail.com

bug#65720: Digression about Git implementations (was Re: bug#65720: Guile-Git-managed checkouts grow way too much)

[Simon Tournier]

Hi Ludo,

On Fri, 08 Sep 2023 at 19:08, Ludovic Courtès  wrote:

> Years ago, Andy Wingo sketched a plan for GNU hackers to implement Git
> in pure Scheme.  That was on April 1st though, so people mistakenly
> assumed it was a joke and the project was never carried out.

Well, that is a piece of work. :-)

Maybe there is an hope with: git-std-lib.

Subject: Proposal/Discussion: Turning parts of Git into libraries
From: Emily Shaffer 
To: Git List 
Date: Fri, 17 Feb 2023 13:12:23 -0800   

https://lore.kernel.org/git/CAJoAoZ=Cig_kLocxKGax31sU7Xe4==bgzc__bg2_pr7krnq...@mail.gmail.com/

And some patches are starting to float around.
https://public-inbox.org/git/20230810163346.274132-1-calvin...@google.com/


> I digress, but I wonder: is there not even a viable Haskell or OCaml
> implementation of Git?

It depends on what means “viable”. :-)

https://github.com/mirage/ocaml-git
https://hackage.haskell.org/package/git

Irmin [1] is an OCaml library for building mergeable, branchable
distributed data stores – A Distributed Database Built on the Same
Principles as Git.  And irmin relies on ocaml-git.

1: https://github.com/mirage/irmin

Then there is a pure Go implementation and another using Java.

https://git-scm.com/book/en/v2/Appendix-B%3A-Embedding-Git-in-your-Applications-go-git
https://git-scm.com/book/en/v2/Appendix-B%3A-Embedding-Git-in-your-Applications-JGit

I do not know all that are “viable”.  Well, I do not know if ’git gc’ is
implemented.  And I do not know which plumbing is implemented and which
porcelain is available.

Last, SWH uses dulwich [2] which is a pure Python implementation of Git.

2: https://www.dulwich.io/

To my knowledge, there is no “dulwich gc” but they implement “dulwich
fsck” and “dulwich repack”.

Back on 10 Years of Guix or at UNESCO on February – I do not remember
exactly when – we were discussing about implementation of Git.  And we
mentioned an implementation in Rust.  Maybe this one:

https://github.com/Byron/gitoxide

Cheers,
simon

bug#65391: Acknowledgement (People need to report failing builds even though we have ci.guix.gnu.org for that)

[Dr. Arne Babenhauserheide]

Simon Tournier  writes:
> On Wed, 30 Aug 2023 at 12:39, "Dr. Arne Babenhauserheide"  
> wrote:
>> Please don’t remove packages that are broken on the CI. I often had a
>> case where no substitute was available but the package built just fine
>> locally. This is not a perfect situation (nicer would be to track why it
>> doesn’t come from CI — sometimes it’s just a resource problem on the
>> CI), but if you removed a package I use that would break all updates for
>> me.
>
> Well, I do not think that any policy will mark a package for removal on
> the first build failure.  However, if the same package is still failing
> after several X  or attempts, it means something is wrong.
> Marking it as a candidate for removal implies:
>
>  1. check if the failure is from CI when it builds locally,
>  2. keep a set of packages that we know they are installable.
>
> For instance, ocaml4.07-* packages are failing since more or less April.
>
> https://data.guix.gnu.org/repository/1/branch/master/package/ocaml4.07-ppxlib/output-history
>
> Does it make sense to keep them?  For another example, some perl6-*
> packages are failing since… 2021.
>
> https://data.guix.gnu.org/repository/1/branch/master/package/perl6-xml-writer/output-history
>
> Does it make sense to keep them?

This is a good example, but not for removing broken packages. For
perl6-xml-writer removing the package would keep breakage in Guix.

I just checked the build, and this looks like a Guix packaging error
that breaks the tests due to a change to some unrelated package:
/gnu/store/ap404x14l604wm0gvaj439ga2vjzwnl7-perl6-tap-harness-0.0.7/bin/prove6: 
/gnu/store/ap404x14l604wm0gvaj439ga2vjzwnl7-perl6-tap-harness-0.0.7/bin/.prove6-real:
 perl6: bad interpreter: No such file or directory

Disabling the tests makes the package build and work.

So here, removing a package would start at the wrong place: some change
between 2021-02-01 and 2021-04-30 broke the perl6-tap-harness and we did
not detect that.

This is a problem that would get hidden by removing broken packages.

The problem is that we (large inclusive we that stands for all users of
Guix) did not track down this problem that causes the build to fail.

From this I see two distinct cases:

- packages broken upstream
- packages broken by changes in Guix

If a package is broken upstream and not going to get fixed and this
requires regular patching in Guix, I agree that we have to remove it at
some point.

If however a change in Guix breaks packages, that change should get
rolled back / reverted and fixed, so it does not break the packages.

8 |   ocaml-migrate-parsetree
  ^^^
Error: Library "ocaml-migrate-parsetree" not found.

This likely means that a change in the inherited package removed the
input, but the breakage wasn’t detected.

And that’s actually what happened in
386ad7d8d14dee2103927d3f3609acc63373156a
Fri Jan 13 10:54:36 2023 +

This commit broke ocaml4.07-ppxlib by cleaning up the inputs of
ocaml-ppxlib (not naming names, this is not about shaming but about
detecting the deeper problem).

It should have been rejected (somehow) by CI. The change it would have
required is this:

diff --git a/gnu/packages/ocaml.scm b/gnu/packages/ocaml.scm
index 8ff755aea9..042432be9a 100644
--- a/gnu/packages/ocaml.scm
+++ b/gnu/packages/ocaml.scm
@@ -6845,6 +6845,9 @@ (define-public ocaml4.07-ppxlib
  (base32
   "0my9x7sxb329h0lzshppdaawiyfbaw6g5f41yiy7bhl071rnlvbv"
  (build-system dune-build-system)
+ (propagated-inputs
+  (modify-inputs (package-propagated-inputs ocaml-ppxlib)
+(prepend ocaml-migrate-parsetree)))
  (arguments
   `(#:phases
 (modify-phases %standard-phases

So for both the cases you named for removal, such a removal would have
caused us to miss actual problems in our process.

This does not mean that there will never be a case in which a package
has to be removed, but given that both cases you showed are likely
self-induced breakage due to changes that should have been rejected as
breaking seemingly unrelated packages, it rather looks like the
situation where removing the package is the right way forward is the
exceptional case.

The norm is that our CI should have detected a problem in the commit
causing the breakage.
(this is reasoning from only two datapoints, so take it with a grain of
salt …)

Can we automatically rebuild all inheriting packages when a package gets
changed?

> The usual situation is that CI is able to build the packages.  The set
> of packages that CI is not able to build is very limited and it is the
> exception.
>
> Having a rule to deal with the regular broken packages appears to me a
> good thing and very helpful to keep Guix reliable.  And that rule cannot
> be based on rare exceptional cases.

A rule should work with known cases, otherwise it causes known breakage.

Also see above: in the two cases you selected, removing the package
would be the wrong path forward.

> On Wed, 30 

bug#63277: python-anaconda-client and conda fail

[Hugo Buddelmeijer]

The conda package seems to be fixed, and python-anaconda-client also
builds. So I think this can be closed.

On Mon, 8 May 2023 at 14:22, Hugo Buddelmeijer  wrote:
>
> Sending the patches seems to have worked out just fine. Except that
> number 2 got in my own spam folder.
>
> I forgot that I also needed to update python-defusedxml to 0.7.1.
>
> Using guix does feel a bit like a moving target. It is good to learn
> how to fix such issues yourself, because they will pop-up often. The
> Contributing section [1] of the manual is a good place to start.
>
> Something unrelated: my desktop (gnome) looks sooo much sharper it
> seems, so kudos to whoever improved that!
>
> Cheers,
> Hugo
>
> [1] https://guix.gnu.org/manual/en/html_node/Contributing.html

bug#52182: [cuirass] remote-worker process freeze

[Ludovic Courtès]

Hello!

I’m optimistically closing this bug because the implementation of
‘cuirass remote-worker’ has now changed to a single Fibers process:

  https://lists.gnu.org/archive/html/guix-devel/2023-07/msg00096.html
  
https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=de8586080e04677cbe34c58f34715757ac61eea3

Ludo’.

bug#65391: People need to report failing builds even though we have ci.guix.gnu.org for that

[Simon Tournier]

Hi,

On Mon, 11 Sept 2023 at 09:33, Csepp  wrote:

> That is not a package problem but a Guix interface problem.  I have been
> saying for a while that there needs to be an option to disable all
> non-trivial local builds by default when you know your machine can't
> handle them.

IMHO, your proposal is orthogonal with the issue at hand: broken
packages.  Other said, the issue is: how to deal with the set of
packages that will not build and we already know it (since weeks,
months or even years for some).

My workstation can handle all the compilations that are required.  My
laptop is able offload to it.  The issue about broken packages is not
about the resources.  It is about burning resources for nothing.

About the issue you are speaking about, we already had discussions in
this direction -- you are not the only one saying "the fix needs to do
X" for a while but please keep in mind that "talking does not cook the
rice". ;-)  Well, maybe you could open a ticket with a concrete
use-case.

Cheers,
simon

bug#65391: People need to report failing builds even though we have ci.guix.gnu.org for that

[Csepp]

(changing the subject back to the intended one.  I think the fact that
someone replies to an automated acknowledgement email like once a week
says indicates that the emails are not communicating clearly what their
purpose is. anyways, on to the actual issue at hand.)

Simon Tournier  writes:

> Hi,
>
> On Tue, 29 Aug 2023 at 10:45, Maxim Cournoyer  
> wrote:
>
>> It's frustrating for users when a package is missing, but it's also
>> frustrating/inefficient for maintainers to stumble upon broken packages
>> when checking if an upgrade broke dependent packages (it takes time to
>> build them just to find out they fail, and researching they already
>> did), so a balance is needed.
>
> There is nothing worse as an user to have this experience:
>
> guix search foobar
>
> oh cool, foobar is there, let try it,
>
> guix shell foobar
>
> … wait …
> … stuff are building …
> … laptop is burning …
> … wait …
> Bang!
>
> Keeping broken packages is just annoyances.  Contributor are annoyed
> because as said by the paragraph above.  And user are annoyed as
> described just above.
>
> I am in favor to set a policy for removing then.
>
> The question is the way to detect them.  QA can do whatever we want but
> until people are helping Chris because, IMHO, Chris is already enough
> busy to keep stuff running, we probably need to keep our process simple
> enough in order to stay actionable and avoid some vacuum of “coulda,
> shoulda or woulda”.  For what my opinion is worth on that. :-)
>
> Cheers,
> simon

That is not a package problem but a Guix interface problem.  I have been
saying for a while that there needs to be an option to disable all
non-trivial local builds by default when you know your machine can't
handle them.
Alternatively the CI could record some basic resource utilization
information, so users could for example set a limit on RAM.  (Although
this gets tricky for parallel builds.)

bug#65720: Guile-Git-managed checkouts grow way too much

[Csepp]

Simon Tournier  writes:

> Hi,
>
> On Fri, 08 Sep 2023 at 19:09, Ludovic Courtès  wrote:
>
 It would also be pretty bad for closure size:

 --8<---cut here---start->8---
 $ guix size guile-git | tail -1
 total: 106.6 MiB
 $ guix size guile-git git-minimal | tail -1
 total: 169.8 MiB
 --8<---cut here---end--->8---

 It’s also not clear concretely how we’d add that dependency.  Try
 invoking ‘git’ from $PATH and print a warning if it doesn’t work?
 But then, what about applications like Cuirass and hpcguix-web?
>>>
>>> I think we can rely on something like,
>>>
>>> guix shell -C git-minimal -- git gc
>>
>> We’re talking about the implementation of a cache (meant to speed up
>> operations), that would actually fill said cache plus do a whole bunch
>> of expensive operations?  Nah.  :-)
>
> I do not think.  If I understand correctly, we need to run “git gc” at
> some point, therefore git-minimal needs to me around.  The question is
> how and when.
>
> Well, maybe I am missing what the bug is about.  For me, it is about
> running ‘git gc’ for cleaning the Git checkout cache, no?
>
>
> Solution #1.  Add git-minimal as inputs.  It increases the closure and
> the extra load (on average) is about the ratio between the rate of “guix
> pull” and the rate of the git-minimal changes.
>
> Assuming, that people are running “guix pull” once per week and say “git
> gc” is run after 50 pulls.  (These both number are totally arbitrary and
> based on my personal estimate).
>
> Data Service [1] tells:
>
> 2023-07-07 15:45:22 2023-09-08 21:22:08
> 2023-05-11 16:10:48 2023-07-07 14:21:45
> 2023-05-01 16:40:08 2023-05-11 14:36:16
> 2023-04-25 13:34:54 2023-05-01 15:19:55
> 2023-04-25 13:34:54 2023-09-08 21:22:08
> 2023-03-06 17:22:28 2023-04-25 12:27:33
> 2023-01-17 23:49:19 2023-03-06 16:48:43
> 2022-11-08 13:06:42 2023-01-17 15:11:47
> 2022-10-08 05:14:46 2022-11-08 09:56:31
> 2022-09-06 15:00:08 2022-10-08 04:15:43
> 2022-08-13 22:02:31 2022-09-06 12:58:52
> …
>
> It means that an user will download ~10 times git-minimal for nothing.
>
>
> Solution #2.  The one I am proposing. :-)  Download git-minimal only
> when Guix needs it for running “git gc”.  Yeah, there is probably a
> small overload with some operations.  But, I bet this overload is much
> smaller than the one of solution #1.
>
> Well, it depends on the number of times people are updating the cache vs
> the rate of change of git-minimal.
>
> For sure, if one updates 100 times per week the cache, having
> git-minimal as inputs is far better.  But I do not think that the
> regular usage on average. :-)
>
> That’s why I am proposing to have an option for turning off this “git
> gc“ operation.
>
> Well, we have lived since years without running ‘git gc’ so running it
> once per year on average is probably enough to keep the cache size
> reasonable.  And git-minimal is changing every month.
>
>
> Maybe, there is some solution #3. ;-)
>
> Cheers,
> simon
>
>
> 1: 
> https://data.guix.gnu.org/repository/1/branch/master/package/git-minimal/output-history

Please don't create another situation like with guix system roll-back,
where a crucial sysadmin operation doesn't work without network access.
Or at least make it configurable, so things that are likely to be needed
for future operations are pre-fetched.

bug#65720: Guile-Git-managed checkouts grow way too much

[Csepp]

Ludovic Courtès  writes:

> Hello!
>
> Josselin Poiret  skribis:
>
>> Right, although I wouldn't necessarily say that the former doesn't have
>> a proper API, but rather that it has a Unix-oriented API.  That leads to
>> performance issues on e.g. Windows but on Linux I'm not sure there's
>> much of a difference.
>
> [...]
>
>> We could consider replacing the guile-git dependency with another
>> library built directly on top of git-minimal, and have this be a
>> dependency of Guix.  Not ideal though, and not really scalable either:
>> we can't just add every VCS as direct dependencies.
>
> I cannot imagine a viable implementation of things like ‘commit-closure’
> and ‘commit-relation’ from (guix git) done by shelling out to ‘git’.
> I’m quite confident this would be slow and brittle.
>
> It looks like there’s no option other than carrying the two
> implementations.
>
> ~~~
>
> Years ago, Andy Wingo sketched a plan for GNU hackers to implement Git
> in pure Scheme.  That was on April 1st though, so people mistakenly
> assumed it was a joke and the project was never carried out.
>
> I digress, but I wonder: is there not even a viable Haskell or OCaml
> implementation of Git?
>
> Thanks,
> Ludo’.

For sake of completeness:
There is an alternative implentation in C for Plan 9 that I've used and
is now mature enough that the 9front project switched to it from
Mercurial.
It might be possible to compile it with the plan9port compiler wrapper.

There is also a Git implementation in OCaml that some MirageOS
unikernels use to serve static content from a git repository.
Also the Irmin "database" is based on git and is written in OCaml.