bug#65714: guix-copy 32-bit integer limit

[Ludovic Courtès]

Hi Efraim,

Efraim Flashner  skribis:

> `guix copy` seems unable to deal with store items larger than
> 2**32 bytes (about 4.3 GiB).
>
> (ins)efraim@3900XT ~$ guix copy --from=192.168.1.196 
> /gnu/store/0f8pgl9cyh8bknl50nw6yj9wykzm6ymc-texlivetexmf-20230313
> retrieving 1 store item from '192.168.1.196'...
> guix copy: error: implementation cannot deal with > 32-bit integers
>
> (ins)efraim@3900XT ~$ ssh 192.168.1.196 du -sch 
> /gnu/store/0f8pgl9cyh8bknl50nw6yj9wykzm6ymc-texlivetexmf-20230313
> 7.5G/gnu/store/0f8pgl9cyh8bknl50nw6yj9wykzm6ymc-texlivetexmf-20230313
> 7.5Gtotal

Hmm that vaguely rings a bell, but we’ve had things like texlive-texmf
from the start and I think we’ve definitely been able to fetch them via
the offload/copy code.

Are you able to copy other store items from that host?  Could there be
another issue leading to a bogus diagnostic?

Thanks,
Ludo’.

bug#65720: Guile-Git-managed checkouts grow way too much

[Ludovic Courtès]

Ludovic Courtès  skribis:

> As reported by Tobias on IRC (in the context of ‘hpcguix-web’),
> checkouts managed by Guile-Git appear to grow beyond reason.  As an
> example, here’s the same ‘.git’ managed with Guile-Git and with Git:
>
> $ du -hs 
> ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq
> 6.7G
> /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq
> $ du -hs .git
> 517M.git

Unsurprisingly, GC makes a big difference:

--8<---cut here---start->8---
$ cp -r 
~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq 
/tmp/checkout
$ (cd /tmp/checkout/; git gc)
Enumerating objects: 717785, done.
Counting objects: 100% (717785/717785), done.
Delta compression using up to 4 threads
Compressing objects: 100% (154644/154644), done.
Writing objects: 100% (717785/717785), done.
Total 717785 (delta 569440), reused 710535 (delta 562274), pack-reused 0
Enumerating cruft objects: 103412, done.
Traversing cruft objects: 81753, done.
Counting objects: 100% (64171/64171), done.
Delta compression using up to 4 threads
Compressing objects: 100% (17379/17379), done.
Writing objects: 100% (64171/64171), done.
Total 64171 (delta 52330), reused 58296 (delta 46792), pack-reused 0
Expanding reachable commits in commit graph: 133730, done.
$ du -hs /tmp/checkout
539M/tmp/checkout
--8<---cut here---end--->8---

> It would seem that libgit2 doesn’t do the equivalent of ‘git gc’.

Confirmed: .

My inclination for the short term would be to work around this
limitation by (1) finding a heuristic to determine is a checkout has
likely accumulated too much cruft, and (2) considering such checkouts as
expired (thereby forcing a re-clone) or running ‘git gc’ on them if
‘git’ is available.

I can’t think of a good heuristic for (1).  Birth time could be one, but
we’d need statx(2):

--8<---cut here---start->8---
$ stat 
~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq | 
tail -4
Access: 2023-09-04 23:13:54.668279105 +0200
Modify: 2023-09-04 11:34:41.665385000 +0200
Change: 2023-09-04 11:34:41.661629102 +0200
 Birth: 2021-08-09 10:48:17.748722151 +0200
--8<---cut here---end--->8---

Lacking statx(2), we can approximate creation time by looking at
‘.git/config’:

--8<---cut here---start->8---
$ stat 
~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/config
 | tail -3
Modify: 2021-08-09 10:50:28.031760953 +0200
Change: 2021-08-09 10:50:28.031760953 +0200
 Birth: 2021-08-09 10:50:28.031760953 +0200
--8<---cut here---end--->8---

This strategy can be implemented like this:

diff --git a/guix/git.scm b/guix/git.scm
index ebe2600209..ed3fa56bc8 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -405,7 +405,16 @@ (define cached-checkout-expiration
 
   ;; Use the mtime rather than the atime to cope with file systems mounted
   ;; with 'noatime'.
-  (file-expiration-time (* 90 24 3600) stat:mtime))
+  (let ((ttl (* 90 24 3600))
+(max-checkout-retention (* 9 30 24 3600)))
+(lambda (file)
+  (match (false-if-exception (lstat file))
+(#f 0) ;FILE may have been deleted in the meantime
+(st (min (pk 'ttl (+ (stat:mtime st) ttl))
+ (pk 'maxttl (match (false-if-exception
+  (lstat (in-vicinity file ".git/config")))
+(#f +inf.0)
+(st (+ (stat:mtime st) max-checkout-retention))
 
 (define %checkout-cache-cleanup-period
   ;; Period for the removal of expired cached checkouts.

Namely, a cached checkout as considered as “expired” after 9 months.  In
my case, it gives this:

--8<---cut here---start->8---
scheme@(guix git)> (cached-checkout-expiration 
"/home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/")

;;; (ttl 1701596081)

;;; (maxttl 1651827028)
$6 = 1651827028
--8<---cut here---end--->8---

Of course having to re-clone entire repositories every 9 months is
ridiculous, but storing gigabytes of packs is worse IMO (I’m
specifically thinking about the Guix repo, which every users copies via
‘guix pull’).

Thoughts?

Thanks,
Ludo’.

bug#65741: The URW fontconfig dilemma

[Bruno Victal]

During my attempt to build the enblend-enfuse documentation [1],
it would fail and display the following message:
--8<---cut here---start->8---
Error: fontconfig: Didn't find expected font family. Perhaps URW Type 1 fonts 
need installing?
--8<---cut here---end--->8---

Naturally searching for the URW fonts in guix didn't yield anything
other than some texlive packages so I ended up writing a definition
for it.
It turned out to be unnecessary as I later found out 'font-ghostscript'
provides just that. (an issue of discoverability perhaps?)

Nonetheless my newly written package (which should be “merged” into
the definition of 'font-ghostscript' with a better description and
perhaps a rename) is an update of what font-ghostscript provides and
should work just the same, in theory. (spoilers: it doesn't)

Intrigued by this difference and after some doc reading and several
arcane commands thrown in desperation, I've distilled them to this:

With the existing font-ghostscript package:
--8<---cut here---start->8---
$ guix shell -C font-ghostscript fontconfig
[env]$ fc-match 'Nimbus Sans L'
n019003l.pfb: "Nimbus Sans L" "Regular"
--8<---cut here---end--->8---

With the new font-urw-base35 package (definition below):
--8<---cut here---start->8---
$ ~/src/guix/pre-inst-env guix shell -C font-urw-base35 fontconfig
[env]$ fc-match 'Nimbus Sans L' 
DejaVuSans.ttf: "DejaVu Sans" "Book"
--8<---cut here---end--->8---

The source of the font-urw-base35 has a file for fontconfig at
'fontconfig/urw-fallback-backwards.conf' that contains this:
--8<---cut here---start->8---




…
  
Nimbus Sans L

  Nimbus Sans

  
…
--8<---cut here---end--->8---

So unless I misunderstood the fontconfig documentation and the comment,
'fc-match' should have found the newer Nimbus Sans, not DejaVu Sans.
What gives?


The definition for this font-urw-base35 package:
--8<---cut here---start->8---
;; Note: add #:use-module (guix build-system copy) to gnu/packages/fonts.scm.
;; when trying this out.
;; See 

 for more information.
(define-public font-urw-base35
  (let ((commit "3c0ba3b5687632dfc66526544a4e811fe0ec0cd9")
(revision "0")
(base-version "20200910"))
(package
  (name "font-urw-base35")
  (version (git-version base-version revision commit))
  (source
   (origin
 (method git-fetch)
 (uri (git-reference
   (url "https://github.com/ArtifexSoftware/urw-base35-fonts;)
   (commit commit)))
 (file-name (git-file-name name version))
 (sha256
  (base32
   "11mc9r2ap80jmh0w5jmypyq9hzyi9g9g4hcj45d5xn66m3jnlgjq"
  (build-system copy-build-system)
  (arguments
   (list
#:install-plan
#~`(("fonts/" "share/fonts/urw-base35"
 #:exclude ("README" "COPYING"))
("appstream/" "share/metainfo"
 #:exclude ("README.md"))
("fontconfig/" "share/fontconfig/conf.avail/"
 #:exclude ("README.md")))
#:phases
#~(modify-phases %standard-phases
(add-before 'install 'configure
  (lambda _
;; Set font priority for fontconfig.
(with-directory-excursion "fontconfig"
  (for-each (lambda (f)
  ;; As recommended in the README.md.
  (rename-file f
   (string-append "61-"
  (basename f
(find-files "." "\\.conf$"
#:fail-on-error? #t
  (home-page "https://github.com/ArtifexSoftware/urw-base35-fonts;)
  (synopsis "URW++ base 35 font set.")
  (description "This package provides substitutes for the 35 fonts
required by Adobe Postscript(c) Language Level 2 specification.")
  (license license:agpl3
--8<---cut here---end--->8---

[1]: 


-- 
Furthermore, I consider that nonfree software must be eradicated.

Cheers,
Bruno.

bug#63726: time-machine without options does not get the latest commit

[Simon Tournier]

Hi,

On Thu, 25 May 2023 at 23:03, Ludovic Courtès  wrote:

>> Now, what I am missing.  The manual says:
>>
>>As for ‘guix pull’, the absence of any options means that the latest
>> commit on the master branch will be used.  The command
>>
>>  guix time-machine -- build hello
>>
>>will thus build the package ‘hello’ as defined in the master branch,
>> which is in general a newer revision of Guix than you have installed.
>> Time travel works in both directions!

[...]

> Should we fix the doc or should we fix the code?…

All seems fixed so let close it?

Cheers,
simon

bug#65740: No fallback to SWH for .guix-channel dependencies

[Simon Tournier]

Hi,

Consider this channels.scm file:

--8<---cut here---start->8---
$ cat /tmp/channels.scm
(list (channel
(name 'guix)
(url "https://git.savannah.gnu.org/git/guix.git;)
(branch "master")
(commit
  "710d9050524213a83e4ce9efe9765d7fbc233839"))
  (channel
(name 'bimsb)
(url "https://github.com/BIMSBbioinfo/guix-bimsb.git;)
(branch "master")
(commit
  "240a599f77dab7dcb8d2ed091b90056e58a46c51")))
--8<---cut here---end--->8---

Then, assume Github is down forever.  Guix will lookup to Software
Heritage (SWH) for the channel bimsb.  So far, so good.  However, then
it fails:

--8<---cut here---start->8---
$ guix time-machine -C /tmp/channels.scm -- describe
Updating channel 'guix' from Git repository at 
'https://git.savannah.gnu.org/git/guix.git'...
Updating channel 'bimsb' from Git repository at 
'https://github.com/BIMSBbioinfo/guix-bimsb.git'...
/tmp/channels.scm:95:6: warning: channel 'bimsb' lacks 'introduction' field but 
'.guix-authorizations' found
Updating channel 'guix-past' from Git repository at 
'https://gitlab.inria.fr/guix-hpc/guix-past'...
guix time-machine: error: Git error: failed to resolve address for 
gitlab.inria.fr: Name or service not known
--8<---cut here---end--->8---

The main reason is because the channel bimsb contains the file
.guix-channel which reads,

--8<---cut here---start->8---
(channel
 (version 0)
 (dependencies
  (channel
   (name guix-past)
   (url "https://gitlab.inria.fr/guix-hpc/guix-past;))
  (channel
   (name guix-science)
   (url "https://github.com/guix-science/guix-science.git;
--8<---cut here---end--->8---

And Guix is not able to fallback to SWH for these channels.

Corollary, even if it was able, what is the correct revision of
the guix-past or guix-science channels?

This report is about two bugs:

 1. transparent fallback to SWH for .guix-channel dependencies

 2. pin all channels when running “guix describe”, even the ones from
   .guix-channel dependencies.

This #2 is annoying because it makes the workflow:

guix time-machine -C channels.scm -- shell -m manifest.scm

unpractical when user relies on channels with dependencies.

Cheers,
simon

bug#65716: bug#65718: Importing a toolchain packages causes top-level dependency cycles

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès  writes:

> Hello,
>
> Maxim Cournoyer  skribis:
>
>> Adding the Guile undocumented "(set! %load-verbosely #t)" to my local
>> near the top of my (guix config) module, I see, when running 'make':
>>
>> make[2] : on entre dans le répertoire « /home/maxim/src/guix »
>> [...]
>> Compiling Scheme modules...
>> [  0%] LOAD guix.scm
>> ;;; loading ./guix/build/syscalls.scm
>> ;;; loading ./guix/build/syscalls.scm
>
> [...]
>
> Looks like this is independent from your change no?  There’s no
> ergodox.scm in the list of loaded files AFAICS.
>
> Do you experience the problem without your patch?

A very simple reproducer on current master:

--8<---cut here---start->8---
modified   gnu/packages/firmware.scm
@@ -43,6 +43,7 @@ (define-module (gnu packages firmware)
   #:use-module (gnu packages admin)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages assembly)
+  #:use-module (gnu packages avr)
   #:use-module (gnu packages backup)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
--8<---cut here---end--->8---

And then:

--8<---cut here---start->8---
$ ./pre-inst-env guix build hello
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/srfi/srfi-71.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/io/ports.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/ice-9/binary-ports.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/base.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/ice-9/iconv.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/ice-9/rdelim.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/enums.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/conditions.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/exceptions.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/control.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/records/procedural.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/syntax-case.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/ice-9/optargs.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/system/base/pmatch.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/records/syntactic.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/hashtables.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/srfi/srfi-69.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/srfi/srfi-13.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/lists.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/rnrs/files.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/srfi/srfi-8.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/ice-9/receive.scm
;;; loading /home/maxim/src/guix/guix/build/utils.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/srfi/srfi-34.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/srfi/srfi-60.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/ice-9/ftw.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/ice-9/vlist.scm
;;; loading /home/maxim/src/guix/guix/build/syscalls.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/system/foreign.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/system/base/target.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/srfi/srfi-19.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/srfi/srfi-6.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/ice-9/i18n.scm
;;; loading 
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/share/guile/3.0/system/foreign-library.scm
;;; loading /home/maxim/src/guix/guix/combinators.scm
;;; loading /home/maxim/src/guix/guix/deprecation.scm
;;; loading /home/maxim/src/guix/guix/serialization.scm
;;; loading /home/maxim/src/guix/guix/monads.scm
;;; loading 

bug#54206: [aarch64] kodi fails because of just one test

[Ricardo Wurmus]

This seems to have been fixed in the meantime.  I was able to install
kodi on aarch64 with guix e365c26a34fa485f9af46538fcea128db681c33d.

-- 
Ricardo