Is Guix suitable for large monorepos?

[jgart]

Is Guix suitable for large monorepos?

How does Guix compare to pants in the python arena?

https://yewtu.be/watch?v=p4stnR1gCR4
https://www.pantsbuild.org/

Re: debbugs irritation Was: [WIP Patch] Adding an FHS container to guix shell

[Csepp]

zimoun  writes:

> Hi,
>
> On Thu, 21 Jul 2022 at 02:20, Csepp  wrote:
>> I was looking for this patch on issues.guix.gnu.org and could not figure
>> out why it's not there and I'd just like to point out that having like 6
>> mailing lists with at least two separate indexes is... not great :) I
>> appreciate everyone's work on Mumi and Debbugs but...  maybe this is an
>> avoidable problem?
>
> Which are these 6 mailing lists?  What do you mean by 2 separate
> indexes?
>
> For the good or not, the GNU project has only one Debbugs instance.
> Mumi is just reading this Debbugs instance database to display the
> information nicely.  Somehow, Mumi is just another frontend to the GNU
> Debbugs instance.  Nothing conceptually different from emacs-debbugs.
>
>
> Cheers,
> simon

Mumi and Debbugs have different search interfaces and seem to use
different ordering.
6 was a slight exaggeration, but development communication happens at
least on:
* patches
* devel
* bugs
And then there is also help, which often overlaps with bugs.
There are also a few others for CI, translation, etc.

IMHO these papercuts add up.  Browsing and cross-referencing issues and
patches is way harder than it is with other forges.
Not saying we need to switch, maybe it's easier to just add the missing
functionality.  Or maybe it doesn't matter to anyone else.

Translating news on weblate?

[Julien Lepiller]

Hi Guix!

I was thinking we could have our news file translated at weblate, which
would help having more people translate it. Attached is a script that
is able to generate a pot file from the news.scm file and a translated
news.scm file from the existing news.scm and a directory that contains
the po files and a LINGUAS file.

The goal is to regularly fetch the news.scm file, generate a new pot
that would automatically be updated on weblate. Translators can do
their work, and the result is regularly merged back (after being
checked that it doesn't break) in our repo.

The script tries to reformat the news in a standard way and to merge
existing copyright lines with the authorship lines in the po files. It
is also able to generate the initial po files that I would give
weblate, so translators don't have to redo translations that have
already been done.

I also attached the LINGUAS file with all languages for which we have
at least one entry in news.scm.

WDYT?


LINGUAS
Description: Binary data
#!/gnu/store/5rbr7gi8q7gpmb4gmqrpnpk55a4gjpkz-profile/bin/guile \
--no-auto-compile -s
!#

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2022 Julien Lepiller 
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see .

;;; Commentary:

;; This code is a helper for news file translation using Gettext.

;;; Code:

(use-modules (srfi srfi-1)
 (srfi srfi-9)
 (srfi srfi-19)
 (ice-9 match)
 (ice-9 pretty-print)
 (ice-9 rdelim)
 (ice-9 regex)
 (guix build po)
 (guix channels))

(define read-channel-news (@@ (guix channels) read-channel-news))
(define channel-news-entries (@@ (guix channels) channel-news-entries))
(define  (@@ (guix channels) ))
(define channel-news-entry (@@ (guix channels) channel-news-entry))

(define (print-msg msg)
  (if msg
(let* ((msg (string-join (string-split msg #\\) ""))
   (msg (string-join (string-split msg #\") "\\\""))
   (msg (string-join (string-split msg #\newline)
 "\\n\"\n\"")))
  (if (string-contains msg "\n") (string-append "\"\n\"" msg) msg))
""))

(define* (generate-po news-file pot-file #:optional lang)
  "Generate a pot file from a news file."
  (let ((news (call-with-input-file news-file read-channel-news)))
(call-with-output-file pot-file
  (lambda (port)
(format port "# SOME DESCRIPTIVE TITLE~%")
(format port "# Copyright (C) YEAR the authors of Guix (msgids) and the following authors (msgstr)~%")
(format port "# This file is distributed under the same license as the guix news file.~%")
(format port "# ~%")
(format port "msgid \"\"~%")
(format port "msgstr \"\"
\"Project-Id-Version: guix news checkout\\n\"
\"Report-Msgid-Bugs-To: bug-g...@gnu.org\\n\"
\"POT-Creation-Date: ~a\\n\"
\"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\\n\"
\"Last-Translator: FULL NAME \\n\"
\"Language-Team: LANGUAGE \\n\"
\"Language: \\n\"
\"MIME-Version: 1.0\\n\"
\"Content-Type: text/plain; charset=UTF-8\\n\"
\"Content-Transfer-Encoding: 8bit\\n\"~%~%" (date->string (current-date)))
(for-each
  (match-lambda
(($  commit tag title body)
 (let ((msgid1 (assoc-ref title "en"))
   (msgid2 (assoc-ref body "en"))
   (msgstr1 (assoc-ref title lang))
   (msgstr2 (assoc-ref body lang)))
   (when commit
 (format port "#. commit: ~a~%" commit))
   (when tag
 (format port "#. tag: ~a~%" tag))
   (format port "msgid \"~a\"~%" (print-msg msgid1))
   (format port "msgstr \"~a\"~%~%" (print-msg msgstr1))
   (when commit
 (format port "#. commit: ~a~%" commit))
   (when tag
 (format port "#. tag: ~a~%" tag))
   (format port "msgid \"~a\"~%" (print-msg msgid2))
   (format port "msgstr \"~a\"~%~%" (print-msg msgstr2)
  (channel-news-entries news))

(define (languages lingua)
  (with-input-from-file lingua
(lambda _
  (let loop ((line (read-line)))
(if (eof-object? line)
'()
;; else read linguas before comment
(let ((before-comment (car (string-split line #\#
 

Re: debbugs irritation Was: [WIP Patch] Adding an FHS container to guix shell

[zimoun]

Hi,

On Thu, 21 Jul 2022 at 02:20, Csepp  wrote:
> I was looking for this patch on issues.guix.gnu.org and could not figure
> out why it's not there and I'd just like to point out that having like 6
> mailing lists with at least two separate indexes is... not great :) I
> appreciate everyone's work on Mumi and Debbugs but...  maybe this is an
> avoidable problem?

Which are these 6 mailing lists?  What do you mean by 2 separate
indexes?

For the good or not, the GNU project has only one Debbugs instance.
Mumi is just reading this Debbugs instance database to display the
information nicely.  Somehow, Mumi is just another frontend to the GNU
Debbugs instance.  Nothing conceptually different from emacs-debbugs.


Cheers,
simon

Re: native-inputs: Go for completeness or minimalism?

[zimoun]

Hi Hartmut,

On Wed, 20 Jul 2022 at 10:33, Hartmut Goebel  
wrote:

> Personally I tend to minimal.

Me too.  Being minimal is better on all aspects, IMHO.

The only drawback is indeed “guix shell -D”.  But, people developing can
add the missing or extra packages.  To me, Guix provides the minimal
environment for building and running one package.

Otherwise, we could imagine to create two packages.

However, there is no consensus about this “minimalism”.  For instance,
some packages have multi-outputs which implies that “guix shell -D” is
not minimal.


Cheers,
simon

Re: Could Guix System eventually run on top of HyperbolaBSD ? slightly off topic

[zimoun]

Hi Raghav,

On Wed, 20 Jul 2022 at 12:03, Raghav Gururajan  
wrote:

> [1] The idea isn't off-topic, because, Guix is meant to be kernel 
> agnostic. So exploring HyperbolaBSD as a kernel option for Guix is with 
> in the prospect.

Hum, I do not think that Guix is agnostic about the standard C library.
Therefore, Guix is not meant to be kernel agnostic.

Well, it depends what we name Guix.  From my understanding, the
bootstrap chain is not kernel agnostic.  For instance, what is the size
of the current binary seed on the Hurd kernel?

>From my understanding, Guix is meant to be GNU which means Guix is using
many tools from GNU and these tools are often not kernel agnostic.


> (A) Guix System with GNU userspace and HyperbolaBSD kernel.

Debian folks did that:

   GNU userland packaged with .deb using GLibc
   running on the top of a modified FreeBSD kernel

The project is named Debian/kfreebsd [1] and it is a huge piece of work.
Well, we could imagine reuse their work and replace “Debian packages” by
Guix ones.

But… yes, a strong but!  It is a lot of work as Vagrant explains it [2].
Basically, Guile 3.0 is not working on Debian/kfreebsd so it requires
many patches to have Guile 3.0 running on the top of a BSD kernel.


1: 
2: 


> (B) Guix System with HyperbolaBSD userspace and HyperbolaBSD kernel.

It means port Guix to a BSD kernel.  Guix is strongly connected to the
GLibc and a BSD kernel is not using GLibc but another.  Therefore, it
requires to solve many difficult core issues.


On Wed, 20 Jul 2022 at 18:49, Maxime Devos  wrote:

> (C) Guix System with HyperbolaBSD userspace (except for GNU C library 
> and compiler) and Linux or Hurd kernel

It means port BSD applications.  For instance, about pf, from Wikipedia
[1]:

PF has been ported to many other operating systems, however
there are major differences in capabilities

and even, nothing about kernels other than BSD variants.

1: 


In all cases (A) or (B) or (C), it is the sort of project that would
take someone highly motivated and highly skilled busy for some time.


Cheers,
simon

gwl-0.5.0 released [stable]

[Ricardo Wurmus]

We are pleased to announce the release of the GNU Guix Workflow
Language version 0.5.0, representing 46 commits by two people,
incorporating the results of productive discussions among a number of
helpful people on the #guix and #guix-hpc IRC channels on libera.chat
and on the gwl-de...@gnu.org mailing list.

This is a maintenance release with a number of bug fixes.
For details see the NEWS excerpt below.

• About

  The Guix Workflow Language (GWL) provides an extension to GNU Guix's
  declarative language for package management to automate the
  execution of programs in scientific workflows.  The GWL can use
  process engines to integrate with various computing environments.
  See the manual for details:

https://workflows.guix.info/manual/

• Download

Here are the compressed sources and a GPG detached signature[*]:
  https://ftpmirror.gnu.org/gwl/gwl-0.5.0.tar.gz
  https://ftpmirror.gnu.org/gwl/gwl-0.5.0.tar.gz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify gwl-0.5.0.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 
BCA689B636553801C3C62150197A5888235FACAC

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69
  Automake 1.16.3
  Gnulib v0.1-3269-g03d7a6b1f

NEWS

* Changes in 0.5.0 (since 0.4.0)

** Package handling
- Package specifications may also contain outputs.

** Testing
- Integration tests verify that all examples from the documentation can be 
executed correctly.
- Integration tests include submission of jobs to a local Slurm cluster via 
DRMAA.

** Bug fixes
- Processes comparisons are now performed with an identity comparison.
  This was always intended in cache table lookups, but the wrong
  comparison procedure was used.

- Custom interpreters in code snippets would be miscompiled due to a
  regression. =compile-procedure= was modified to address this
  regression.
  
- With containerized execution, the contained script was not passed its
  arguments for inputs, outputs, and values.  Now the arguments are
  passed through.
  
- The container scripts were compiled with fixed values for process
  inputs and outputs; now they read these values from their arguments as
  in other execution modes.
  
- Set the job’s working directory when launching a DRMAA job.

-- 
Ricardo


signature.asc
Description: PGP signature

Re: Building, packaging and updating Guix with confidence

[Maxime Devos]

On 21-07-2022 18:10, Josselin Poiret wrote:


b...@bokr.com  writes:

Naively:

Why does "the" guix daemon per se need root access at all?

The main thing is that all files in the store end up being written by
the guix daemon user.  So if we want the files to be easily
substitutable, they'd need to have a fixed uid/gid, and the only one we
can guarantee is root.  Other than that, it needs to use a bunch of
Linux namespaces to isolate the builds from the rest of the system,
which depending on the kernel build-time configuration might not be
possible when unprivileged.


Also, resource savings on multi-user systems. And if the guix daemon is 
run as the regular user, then all other daemons (on Guix System) would 
need to be run as that user or as root to be able to access theirselves, 
which is bad from a security perspective.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Building, packaging and updating Guix with confidence

[Josselin Poiret]

Hello,

b...@bokr.com writes:
> Naively:
>
> Why does "the" guix daemon per se need root access at all?

The main thing is that all files in the store end up being written by
the guix daemon user.  So if we want the files to be easily
substitutable, they'd need to have a fixed uid/gid, and the only one we
can guarantee is root.  Other than that, it needs to use a bunch of
Linux namespaces to isolate the builds from the rest of the system,
which depending on the kernel build-time configuration might not be
possible when unprivileged.

Best,
-- 
Josselin Poiret

Re: Dealing with non-ASCII file names in BOOTSTRAP-ORIGIN

[Ludovic Courtès]

Hi!

Marius Bakke  skribis:

> I tried switching to GCC 11 on the core-updates branch, but it fails
> early when attempting to repack the GCC source code for GCC-BOOT0,
> because some files in its test suite contains non-ASCII characters:
>
> [... unpacking ...]
> patching file gcc/builtins.c
> Hunk #1 succeeded at 4623 with fuzz 1 (offset 1341 lines).
> Hunk #2 succeeded at 6097 with fuzz 2 (offset 2206 lines).
> patching file gcc/gimple-fold.c
> Hunk #1 succeeded at 665 (offset 9 lines).
> Hunk #2 succeeded at 766 with fuzz 2 (offset 16 lines).
> patching file libvtv/Makefile.in
> Hunk #1 succeeded at 14 with fuzz 1 (offset -1 lines).
> source is at 'gcc-11.3.0'
> applying 
> '/gnu/store/g0ba4l825z9i4l1jd5cqvl6m09xicdwa-gcc-9-strmov-store-file-names.patch'...
> applying 
> '/gnu/store/5705r4ajxl8lav1hz9xm19w75zdcz1n2-gcc-5.0-libvtv-runpath.patch'...
> find-files: 
> gcc-11.3.0/gcc/testsuite/go.test/test/fixedbugs/issue27836.dir/Äfoo.go: No 
> such file or directory
> Backtrace:
> In srfi/srfi-1.scm:
>  465: 19 [fold # result+visited)> ...]
> In ice-9/ftw.scm:
>  452: 18 [# result+visited)> # #]
>  450: 17 [loop "gcc" "gcc-11.3.0" ...]
> In srfi/srfi-1.scm:
>  465: 16 [fold # result+visited)> ...]
> In ice-9/ftw.scm:
>  452: 15 [# result+visited)> # #]
>  450: 14 [loop "testsuite" "gcc-11.3.0/gcc" ...]
> In srfi/srfi-1.scm:
>  465: 13 [fold # result+visited)> ...]
> In ice-9/ftw.scm:
>  452: 12 [# 
> # #]
>  450: 11 [loop "go.test" "gcc-11.3.0/gcc/testsuite" ...]
> In srfi/srfi-1.scm:
>  465: 10 [fold # result+visited)> ...]
> In ice-9/ftw.scm:
>  452: 9 [# 
> # #]
>  450: 8 [loop "test" "gcc-11.3.0/gcc/testsuite/go.test" ...]
> In srfi/srfi-1.scm:
>  465: 7 [fold # result+visited)> ...]
> In ice-9/ftw.scm:
>  452: 6 [# 
> # #]
>  450: 5 [loop "fixedbugs" "gcc-11.3.0/gcc/testsuite/go.test/test" ...]
> In srfi/srfi-1.scm:
>  465: 4 [fold # result+visited)> ...]
> In ice-9/ftw.scm:
>  452: 3 [# 
> # #]
>  474: 2 [loop "issue27836.dir" ...]
> In guix/build/utils.scm:
>  540: 1 [# result)> 
> "gcc-11.3.0/gcc/testsuite/go.test/test/fixedbugs/issue27836.dir/Äfoo.go" ...]
> In unknown file:
>?: 0 [scm-error misc-error #f "~A" ("find-files failed") #f]
>
> ERROR: In procedure scm-error:
> ERROR: find-files failed
>
> Deleting these files also don't work for the same reason, even when
> using the hex representation, i.e. (delete-file "\u00c4foo.go"), or with
> DELETE-FILE-RECURSIVELY.
>
> One workaround is to avoid the use of BOOTSTRAP-ORIGIN by applying the
> patches and snippet in phases, but that's suboptimal because it has to
> be done for all of GCC-BOOT0, LIBSTDC++, and GCC-FINAL.

To be clear, you’re calling ‘delete-file-recursively’ from a snippet,
right?

If so, maybe you could… avoid it somehow?  :-)

Other options, as discussed on IRC, include adding ‘glibc-utf8-locales’
to the environment, or calling out to Coreutils as you proposed…

Ludo’.

Re: Guix Without Scheme

[Csepp]

Munyoki Kilyungi  writes:

> [[PGP Signed Part:Undecided]]
> Hi Csepp!
>
> Csepp  anaandika:
>
>> jgart  writes:
>>
>>> Hi Guixers,
>>>
>>> I just wanted to share this presentation that Singpolyma gave titled "Guix 
>>> Without Scheme":
>>>
>>> https://archive.org/details/singpolyma-guix-without-scheme
>>>
>>> Through the course of the presentation, singpolyma demos how to build
>>> a Guix package with javascript as well as lua.
>>>
>>> What do people think of leveraging Guile's compiler tower to write Guix
>>> packages in lua, javascript, python, and other languages? 
>>>
>>> nix perhaps? Might be meta fun to write a Guix package in a guile 
>>> implemented nix frontend.
>>>
>>> Maybe we should think of Scheme as just one frontend among many to Guile's 
>>> compiler tower?
>>>
>>> Is it a future goal for Guix to fully support this unique feature?
>>>
>>> all best,
>>>
>>> jgart
>>
>> I like Lua but this doesn't seem to be a wise use of developer
>> resources.  Those languages have subtle differences that would surface
>> if this idea was pursued further.  The most obvious one: Scheme is very
>> permissive with what you can name your variables, while all the others
>> basically follow C's restrictions.  How would you map those names?  It
>> is not possible to create a bijection between sets of different
>> cardinality.  And then we haven't gotten into how the module systems are
>> different or how record systems aren't even standardized among Schemes.
>>
>> If you want to experiment with mixing languages, Racket is likely a
>> better starting point.
>>
>
> Using other languages, if simple enough, would be
> a nice way to have developers who fear the parens
> dip their toe in GNU Guix using Guile.  Nice for
> intro workshops/talks - maybe.
>
> That said - and from a very biased view - I think
> Scheme is simple enough for anyone, and outside
> hobby or exploratory programming, should be
> primarily used for GNU Guix hacking.  Perhaps,
> over time that may change.

For non-sexpy programming we already have different syntaxes for Scheme,
like Wisp.  Those don't change the semantics of the language though.