Re: Scheduling a new release?

[Christina O'Donnell]

Hi,

On 08/05/2024 14:01, Christopher Baines wrote:

I think it would be nice to have a new release, and indeed release more
often, I think the way to get there is for less things to be broken
between releases, such that releasing takes less effort in terms of
testing and fixing things.

To give some specific issues, I've run up against the recent issues with
nss [1][2] and I don't think we could release with the nss package as is
currently.

1: https://issues.guix.gnu.org/70662
2: https://issues.guix.gnu.org/70663


I can fix these by disabling tests, but I would prefer if someone with 
more experience packaging for guix could make a decision on it. 
Otherwise, I don't have any problem reducing the number of tests and 
disabling all tests on PowerPC at least.


I could also do some analysis if it was deemed necessary, inserting a 
patch to measure the timings of each test/cycle. Additionally, I could 
try packaging some of the versions between 0.88 and 0.98 to identify the 
exact change that could be to blame. However, both of these seem 
overkill, given the backlog of patches/issues we have left to get 
through, and the manpower we currently have to work with.


Would any of that be helpful?


...


Kind regards,

Christina

Re: bug#40316: nss not reproducible

[Christina O'Donnell]

Hi,

On 06/05/2024 11:12, Ludovic Courtès wrote:

Hi,

Christina O'Donnell  skribis:


Tangentially, given how long nss takes to build, do you think that
it'd be worth shaving it down to a single test pass? Currently it runs
each test up to 3 times, which takes ~1h on my machine with no other
build running. Running only the standard pass takes 2.5-3x less time,
which is a huge quality of life improvement.

Currently we run ./nss/tests/all.sh, which I suppose is what upstream
recommends to run tests.

For sure I’d be happy if the test suite could run faster, but does
upstream offer such an option?  When you say “a single pass”, is that
something upstream supports?
Yes, you can control the tests by setting environment variables 
NSS_TESTS to a list of tests and NSS_CYCLES to a list of 'cycles' (what 
I previously called passes). The default is:


"standard pkix threadunsafe"

* 'standard' runs all of the below tests with default settings: "cipher 
lowhash cert dbtests tools sdr crmf smime ssl ocsp merge pkits ec gtests 
ssl_gtests policy"


* 'pkix' runs the tests "lowhash libpkix cert tools ssl ocsp pkits ec 
gtests ssl_gtests policy" with PKIX enabled.


* 'thread_unsafe' runs "ssl ssl_gtests" with "THREAD_UNSAFE" enabled.

My thinking would be to run the thread_unsafe cycle normally, but to 
reduce the test overlap between standard and pkix however, I can't say 
that I'm knowledgeable enough of NSS to claim that that wouldn't leave 
gaps that might bite us some point down the line. So it might be best to 
leave it as is unless someone familiar with NSS can confirm that it'd be 
safe to disable some tests/cycles.


Kind regards,

Christina

Re: bug#40316: nss not reproducible

[Christina O'Donnell]

Hi Ludo',

This doesn't look directly related. I haven't seen anything like this 
occur when I build it.


Tangentially, given how long nss takes to build, do you think that it'd 
be worth shaving it down to a single test pass? Currently it runs each 
test up to 3 times, which takes ~1h on my machine with no other build 
running. Running only the standard pass takes 2.5-3x less time, which is 
a huge quality of life improvement.


Kind regards,

Christina

On 02/05/2024 09:15, Ludovic Courtès wrote:

Hi Christina,

Nice work!

Christina O'Donnell  skribis:


I've got as far as making nss 3.98 reproducible, however updating it to 3.99
results in 51 test failures. These are regressions, and worked correctly for
3.98. I'm not entirely sure what the issue is, but I've run out of time to
debug it this week, so I'm sending this patch up as is.

Not sure if this is related, but we’re seeing test failures due to
timing issues right now with 3.98:

   https://issues.guix.gnu.org/70693

Thank you!

Ludo’.

[PATCH 6/6] WIP: nss: Attempting to resolve FIPS regression.

[Christina O'Donnell]

There are 51 new test failures which all appear to be related to FIPS.

For example:

modutil -dbdir 
/tmp/guix-build-nss-3.99.drv-0/nss-3.99/tests_results/security/localhost.1/fips 
-fips true

WARNING: Performing this operation while the browser is running could cause
corruption of your security databases. If the browser is currently running,
you should exit browser before continuing this operation. Type
'q ' to abort, or  to continue:
A PKCS #11 module returned CKR_DEVICE_ERROR, indicating that a problem has 
occurred with the token or slot.
ERROR: Unable to switch FIPS modes.
cert.sh: #291: Enable FIPS mode on database for FIPS PUB 140 Test Certificate 
(11)  - FAILED
cert.sh ERROR: Enable FIPS mode on database for FIPS PUB 140 Test Certificate 
failed 11

Change-Id: If0d57bb9e129eb862fae1a28d9779c6100e0a23d
---
 gnu/packages/nss.scm | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/nss.scm b/gnu/packages/nss.scm
index 80667d8affe..a8fb6965c2c 100644
--- a/gnu/packages/nss.scm
+++ b/gnu/packages/nss.scm
@@ -134,6 +134,10 @@ (define-public nss
   (delete-file-recursively "nss/lib/sqlite")
 (build-system gnu-build-system)
 (outputs '("out" "bin"))
+;; (search-paths
+;;  (list (search-path-specification
+;; (variable "LD_LIBRARY_PATH")
+;; (files '("lib")
 (arguments
  (list
   #:make-flags
@@ -161,12 +165,15 @@ (define-public nss
 #$@(if (%current-target-system)
#~("CROSS_COMPILE=1")
#~())
+(string-append "NSS_FORCE_FIPS=1")
+(string-append "NSPR_LIB_DIR="
+   (string-append #$nspr "/lib"))
 (string-append "NSPR_INCLUDE_DIR="
(search-input-directory %build-inputs
"include/nspr"))
 ;; Add $out/lib/nss to RPATH.
 (string-append "RPATH=" rpath)
-(string-append "LDFLAGS=" rpath)))
+(string-append "LDFLAGS=" rpath " -L" #$nspr "/lib")))
   #:modules '((guix build gnu-build-system)
   (guix build utils)
   (ice-9 ftw)
@@ -203,6 +210,8 @@ (define-public nss
 (setenv "DOMSUF" "localdomain")
 (setenv "USE_IP" "TRUE")
 (setenv "IP_ADDRESS" "127.0.0.1")
+;; (setenv "LD_LIBRARY_PATH"
+;; (string-append (getenv "LD_LIBRARY_PATH")))
 
 ;; The "PayPalEE.cert" certificate expires every six 
months,
 ;; leading to test failures:
-- 
2.41.0

[PATCH 1/6] gnu: nss: Fix cross-compilation.

[Christina O'Donnell]

From: Zheng Junjie 

* gnu/packages/nss.scm (nss)[arguments]<#:make-flags>: When
cross-compilation, Add CROSS_COMPILE=1.
<#:phases>: When cross-compilation, Set env NATIVE_CC to gcc.

Change-Id: I5c9559a4b8cecf2cfc6c47d136d69c01a335faaf
Signed-off-by: Zheng Junjie 
---
 gnu/packages/nss.scm | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/gnu/packages/nss.scm b/gnu/packages/nss.scm
index 7e9ed49ead8..459e53bc1cf 100644
--- a/gnu/packages/nss.scm
+++ b/gnu/packages/nss.scm
@@ -154,6 +154,9 @@ (define-public nss
 (#$(target-linux?) "linux")
 (else ""
#~())
+#$@(if (%current-target-system)
+   #~("CROSS_COMPILE=1")
+   #~())
 (string-append "NSPR_INCLUDE_DIR="
(search-input-directory %build-inputs
"include/nspr"))
@@ -175,6 +178,10 @@ (define-public nss
 (lambda _
   (setenv "CC" #$(cc-for-target))
   (setenv "CCC" #$(cxx-for-target))
+  ;; TODO: Set this unconditionally
+  #$@(if (%current-target-system)
+ #~((setenv "NATIVE_CC" "gcc"))
+ #~())
   ;; No VSX on powerpc-linux.
   #$@(if (target-ppc32?)
  #~((setenv "NSS_DISABLE_CRYPTO_VSX" "1"))
-- 
2.41.0

[PATCH 0/6] WIP: nss: Update to 3.99

[Christina O'Donnell]

Hi,

I've got as far as making nss 3.98 reproducible, however updating it to 3.99
results in 51 test failures. These are regressions, and worked correctly for
3.98. I'm not entirely sure what the issue is, but I've run out of time to
debug it this week, so I'm sending this patch up as is.

Up to patch 3 build correctly. Patch 4 is the first one that fails.

The issue specifically seems to all be related to FIPS:

A PKCS #11 module returned CKR_DEVICE_ERROR, indicating that a problem has
occurred with the token or slot.

If someone could take a look at this and see if there's anything I've missded
then I'd appreciate that. Otherwise I'm free to pick it back up again on
Tuesday.

Let me know if you have any questions.

Kind regards,
Christina

Christina O'Donnell (4):
  gnu: nss: Make reproducible.
  gnu: nss: Update to 3.99.
  gnu: nss-certs: Update to 3.99.
  WIP: nss: Attempting to resolve FIPS regression.

Zheng Junjie (2):
  gnu: nss: Fix cross-compilation.
  gnu: nspr: Fix cross-compilation.

 gnu/packages/certs.scm| 24 +--
 gnu/packages/nss.scm  | 30 +++--
 .../patches/nss-Disable-library-signing.patch | 67 +++
 3 files changed, 111 insertions(+), 10 deletions(-)
 create mode 100644 gnu/packages/patches/nss-Disable-library-signing.patch


base-commit: 9a47ef6182b6a36354699efbdbedca17f24cd9b8
-- 
2.41.0

[PATCH 2/6] gnu: nspr: Fix cross-compilation.

[Christina O'Donnell]

From: Zheng Junjie 

* gnu/packages/nss.scm (nspr)[arguments]<#:configure-flags>: When
cross-compilation, Add HOST_CC=gcc.

Change-Id: I337f217f153f8cc3a713906643d6fab9115056e9
Signed-off-by: Zheng Junjie 
---
 gnu/packages/nss.scm | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/nss.scm b/gnu/packages/nss.scm
index 459e53bc1cf..0baafe2f373 100644
--- a/gnu/packages/nss.scm
+++ b/gnu/packages/nss.scm
@@ -71,7 +71,10 @@ (define-public nspr
   #~(list "--disable-static"
   "--enable-64bit"
   (string-append "LDFLAGS=-Wl,-rpath="
- (assoc-ref %outputs "out") "/lib"))
+ (assoc-ref %outputs "out") "/lib")
+  #$@(if (%current-target-system)
+ #~("HOST_CC=gcc")
+ #~()))
   ;; Use fixed timestamps for reproducibility.
   #:make-flags #~'("SH_DATE='1970-01-01 00:00:01'"
;; This is epoch 1 in microseconds.
-- 
2.41.0

[PATCH 4/6] gnu: nss: Update to 3.99.

[Christina O'Donnell]

gnu/packages/nss.scm (nss): Update to 3.99.

Change-Id: Iba6c9dc2956cc0febb62a1c471add899250fa489
---
 gnu/packages/nss.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/nss.scm b/gnu/packages/nss.scm
index b608a995577..80667d8affe 100644
--- a/gnu/packages/nss.scm
+++ b/gnu/packages/nss.scm
@@ -109,7 +109,7 @@ (define-public nss
 ;; IMPORTANT: Also update and test the nss-certs package, which duplicates
 ;; version and source to avoid a top-level variable reference & module
 ;; cycle.
-(version "3.88.1")
+(version "3.99")
 (source (origin
   (method url-fetch)
   (uri (let ((version-with-underscores
@@ -120,7 +120,7 @@ (define-public nss
   "nss-" version ".tar.gz")))
   (sha256
(base32
-"15il9fsmixa1r4446zq1wl627sg0hz9h67w6kjxz273xz3nl7li7"))
+"1g89ig40gfi1sp02gybvl2z818lawcnrqjzsws36cdva834c5maw"))
   ;; Create nss.pc and nss-config.
   (patches (search-patches "nss-3.56-pkgconfig.patch"
"nss-getcwd-nonnull.patch"
-- 
2.41.0

[PATCH 3/6] gnu: nss: Make reproducible.

[Christina O'Donnell]

gnu/packages/patches/nss-Disable-library-signing.patch: Disable library
signing to make the build reproducible.
gnu/packages/nss.scm (nss): Apply this new patch.

Change-Id: I7860bae219ecc4a79423a590c27a1097ae2e7874
---
 gnu/packages/nss.scm  |  3 +-
 .../patches/nss-Disable-library-signing.patch | 67 +++
 2 files changed, 69 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/nss-Disable-library-signing.patch

diff --git a/gnu/packages/nss.scm b/gnu/packages/nss.scm
index 0baafe2f373..b608a995577 100644
--- a/gnu/packages/nss.scm
+++ b/gnu/packages/nss.scm
@@ -124,7 +124,8 @@ (define-public nss
   ;; Create nss.pc and nss-config.
   (patches (search-patches "nss-3.56-pkgconfig.patch"
"nss-getcwd-nonnull.patch"
-   "nss-increase-test-timeout.patch"))
+   "nss-increase-test-timeout.patch"
+   "nss-Disable-library-signing.patch"))
   (modules '((guix build utils)))
   (snippet
'(begin
diff --git a/gnu/packages/patches/nss-Disable-library-signing.patch 
b/gnu/packages/patches/nss-Disable-library-signing.patch
new file mode 100644
index 000..b488d29dcad
--- /dev/null
+++ b/gnu/packages/patches/nss-Disable-library-signing.patch
@@ -0,0 +1,67 @@
+From 4734b834755822f962af29e9395daa7338084e21 Mon Sep 17 00:00:00 2001
+Message-ID: 
<4734b834755822f962af29e9395daa7338084e21.1714059680.git@mutix.org>
+From: Christina O'Donnell 
+Date: Thu, 25 Apr 2024 16:35:50 +0100
+Subject: [PATCH] nss: Disable library signing.
+
+---
+ nss/cmd/shlibsign/Makefile | 32 +---
+ 1 file changed, 1 insertion(+), 31 deletions(-)
+
+diff --git a/nss/cmd/shlibsign/Makefile b/nss/cmd/shlibsign/Makefile
+index a119205..7a85c1d 100644
+--- a/nss/cmd/shlibsign/Makefile
 b/nss/cmd/shlibsign/Makefile
+@@ -43,22 +43,9 @@ EXTRA_SHARED_LIBS += \
+ 
+ endif
+ 
+-
+-# sign any and all shared libraries that contain the word freebl
+-ifeq ($(NSS_BUILD_WITHOUT_SOFTOKEN),1)
++# Disable library signing as it's non-deterministic
+ CHECKLIBS =
+ CHECKLOC =
+-else
+-CHECKLIBS = $(DIST)/lib/$(DLL_PREFIX)softokn3.$(DLL_SUFFIX)
+-CHECKLIBS += $(wildcard $(DIST)/lib/$(DLL_PREFIX)freebl*3.$(DLL_SUFFIX))
+-ifndef NSS_DISABLE_DBM
+-CHECKLIBS += $(DIST)/lib/$(DLL_PREFIX)nssdbm3.$(DLL_SUFFIX)
+-endif
+-CHECKLOC = $(CHECKLIBS:.$(DLL_SUFFIX)=.chk)
+-
+-MD_LIB_RELEASE_FILES = $(CHECKLOC)
+-ALL_TRASH += $(CHECKLOC)
+-endif
+ 
+ ###
+ # (5) Execute "global" rules. (OPTIONAL)  #
+@@ -78,23 +65,6 @@ include $(CORE_DEPTH)/coreconf/rules.mk
+ 
+ include ../platrules.mk
+ 
+-SRCDIR = $(call core_abspath,.)
+-
+-%.chk: %.$(DLL_SUFFIX) 
+-ifeq ($(OS_TARGET), OS2)
+-  cd $(OBJDIR) ; cmd.exe /c $(SRCDIR)/sign.cmd $(DIST) \
+-  $(call core_abspath,$(OBJDIR)) $(OS_TARGET) \
+-  $(call core_abspath,$(NSPR_LIB_DIR)) $(call core_abspath,$<)
+-else
+-ifeq ($(CROSS_COMPILE),1)
+-  # do nothing
+-else
+-  cd $(OBJDIR) ; sh $(SRCDIR)/sign.sh $(call core_abspath,$(DIST)) \
+-  $(call core_abspath,$(OBJDIR)) $(OS_TARGET) \
+-  $(call core_abspath,$(NSPR_LIB_DIR)) $(call core_abspath,$<)
+-endif
+-endif
+-
+ libs: install
+ ifdef CHECKLOC
+   $(MAKE) $(CHECKLOC)
+
+base-commit: 2951778f8e8855bed24754a57ecc43f02a2843dd
+-- 
+2.41.0
+
-- 
2.41.0

[PATCH 5/6] gnu: nss-certs: Update to 3.99.

[Christina O'Donnell]

gnu/packages/certs.scm (nss-certs-3.88.1): New variable.
(nss-certs-3.98): Update and rename to nss-certs-3.99.
(nss-certs): Update to 3.99.

Change-Id: I2f5f737d44d08497d4f5e0e07557be36d2f1f070
---
 gnu/packages/certs.scm | 24 +++-
 1 file changed, 19 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/certs.scm b/gnu/packages/certs.scm
index 7078c7c8d11..7aa96493fbe 100644
--- a/gnu/packages/certs.scm
+++ b/gnu/packages/certs.scm
@@ -125,7 +125,7 @@ (define-public certdata2pem
 that was originally contributed to Debian.")
   (license license:isc
 
-(define-public nss-certs
+(define-public nss-certs-3.88.1
   (package
 (name "nss-certs")
 ;; XXX We used to refer to the nss package here, but that eventually caused
@@ -188,10 +188,10 @@ (define-public nss-certs
 (home-page "https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS;)
 (license license:mpl2.0)))
 
-(define-public nss-certs-3.98
+(define-public nss-certs-3.99
   (package
-(inherit nss-certs)
-(version "3.98")
+(inherit nss-certs-3.88.1)
+(version "3.99")
 (source (origin
   (method url-fetch)
   (uri (let ((version-with-underscores
@@ -202,7 +202,21 @@ (define-public nss-certs-3.98
   "nss-" version ".tar.gz")))
   (sha256
(base32
-"1kh98amfklrq6915n4mlbrcqghc3srm7rkzs9dkh21jwscrwqjgm"))
+"15il9fsmixa1r4446zq1wl627sg0hz9h67w6kjxz273xz3nl7li7"))
+  ;; Create nss.pc and nss-config.
+  (patches (search-patches "nss-3.56-pkgconfig.patch"
+   "nss-getcwd-nonnull.patch"
+   "nss-increase-test-timeout.patch"
+   "nss-Disable-library-signing.patch"))
+  (modules '((guix build utils)))
+  (snippet
+   '(begin
+  ;; Delete the bundled copy of these libraries.
+  (delete-file-recursively "nss/lib/zlib")
+  (delete-file-recursively "nss/lib/sqlite")))
+
+(define-public nss-certs
+  nss-certs-3.99)
 
 (define-public le-certs
   (package
-- 
2.41.0

Re: nss not reproducible

[Christina O'Donnell]

Hi,

I believe I have a fix for this, I'm just waiting on my machine to hurry 
up and confirm it, might end up running over night, then I'll send my 
patch up.


I'm doing two native builds and two cross-builds.

I've also updated to 3.99.

Kind regards,

Christina

On 25/04/2024 15:06, Christina O'Donnell wrote:

Hi Steve,


It would be good to confirm this one:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40316


Still fails to reproduce with those changes applied.

The culprit is in nss/cmd/shlibsign/shlibsign.c:

shlibSignHMAC generates a new key-pair each time it's run:

    /* Generate a DSA key pair */
    logIt("Generate an HMAC key ... \n");
    crv = pFunctionList->C_GenerateKey(hRwSession, ,
   hmacKeyTemplate,
PR_ARRAY_SIZE(hmacKeyTemplate),
   );

Three options:
 1. Disable library signing entirely.
 2. Seed the generation to be deterministic.
 3. Drop in a HMAC key-pair and patch the code to use that instead of 
generating.


2 and 3 defeat the point of the cryptographically secure supply chain 
as the private key can be obtained deterministically, so my vote would 
be simply  to not sign the libraries (1), which would be easier to 
maintain. We're not the primary distributor and users can verify our 
distribution of nss by running `guix challenge` anyway.


It looks like Zhen Junjie applied two patches to fix NSS 
cross-compilation on Master [0]


Building everything cross-compiled to ARM now.

Kind regards,

Christina

Re: Core updates status

[Christina O'Donnell]

Hi Steve,


It would be good to confirm this one:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40316


Still fails to reproduce with those changes applied.

The culprit is in nss/cmd/shlibsign/shlibsign.c:

shlibSignHMAC generates a new key-pair each time it's run:

    /* Generate a DSA key pair */
    logIt("Generate an HMAC key ... \n");
    crv = pFunctionList->C_GenerateKey(hRwSession, ,
   hmacKeyTemplate,
PR_ARRAY_SIZE(hmacKeyTemplate),
   );

Three options:
 1. Disable library signing entirely.
 2. Seed the generation to be deterministic.
 3. Drop in a HMAC key-pair and patch the code to use that instead of 
generating.


2 and 3 defeat the point of the cryptographically secure supply chain as 
the private key can be obtained deterministically, so my vote would be 
simply  to not sign the libraries (1), which would be easier to 
maintain. We're not the primary distributor and users can verify our 
distribution of nss by running `guix challenge` anyway.



It looks like Zhen Junjie applied two patches to fix NSS cross-compilation on 
Master [0]


Building everything cross-compiled to ARM now.

Kind regards,

Christina

Re: Core updates status

[Christina O'Donnell]

Okay, I'll let you know as soon as I know.

On 24/04/2024 14:17, Steve George wrote:

Hi,

You just need to checkout core-updates and then 'start building'!

It would be good to confirm this one:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40316

It looks like Zhen Junjie applied two patches to fix NSS cross-compilation on 
Master [0]

Maybe master and core-updates have diverged - I haven't had time to look.

To check for reproducibility do a normal build, and if that's successful do the 
same build with `--check`

Thanks,

Steve

[0] 
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=fb86bf658a9374d41b05c5e586bfc6a3150cc3cb
  and 
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=452e7673bfeb0a14cecb8e760dda2c436aa69047


On 24 Apr, Christina O'Donnell wrote:

Hi Steve,

On 24/04/2024 07:08, Steve George wrote:

Hi,

We're trying to stabilise and merge core-updates, help definitely wanted!

I'd love to help! Any of these issues novice-friendly?

Will there be a point release after core-updates is merged?

Kind regards,

Christina

Re: Core updates status

[Christina O'Donnell]

Hi Steve,

On 24/04/2024 07:08, Steve George wrote:

Hi,

We're trying to stabilise and merge core-updates, help definitely wanted!


I'd love to help! Any of these issues novice-friendly?

Will there be a point release after core-updates is merged?

Kind regards,

Christina

Re: Adding plumbing subcommand 'derivation'?

[Christina O'Donnell]

Correction:

On 18/04/2024 13:57, Christina O'Donnell wrote:

[...]

`guix store get-references`
`guix store get-referrers`
`guix store info` - dumps the `path-info`
`guix store put xyz` - puts xyz into the store as a fixed derivation 
binary blob.


[...]

I've just checked and get-references and get-referrers are available as:

`guix gc --references`
`guix gc --referrers`

Kind regards,
Christina

Re: Adding plumbing subcommand 'derivation'?

[Christina O'Donnell]

Hi,

In the interest of throwing ideas out there, and with the caveat that 
this is rather uninformed:


I think having  a derivation sub-command makes the most sense to me. E.g.

`guix derivation show` for `guix drv-show`
`guix derivation update` for `guix drv-drv` (or 'refresh' or 'fix')

I have also been thinking that it'd potentially be useful to have 
something like:


`guix derivation edit /gnu/store/...-xyz.drv`

Which then produces a temp directory that you may edit with whatever 
tools you like, then on command checks for changed files, recompute 
hashes recursively, and puts the modified derivations and back into the 
store. I think this would be mainly be useful as a debugging tool or for 
changes that no longer build, for cases where the new `guix derivation 
update` wouldn't work.


On the tangential subject of plumbing commands, I've also started to 
write a `guix store` command that simply exposes functions in (guix 
store) as a command line interface as a way of learning the internals of 
Guix.


This would have sub-commands like:

`guix store get-references`
`guix store get-referrers`
`guix store info` - dumps the `path-info`
`guix store put xyz` - puts xyz into the store as a fixed derivation 
binary blob.


I say this as a way of clarifying the design of `guix derivation`, not 
as extra stuff that needs go in at the same time. Most of these commands 
might be deemed unnecessary.


Alternatively, as you say, Ricardo:

`guix inspect get-references`
`guix inspect get referrers`
`guix inspect path-info`
`guix inspect show-derivation`

However, in that case, another home would need to be found for drv-drv 
at minimum since I wouldn't expect 'inspect' to modify anything.


Kind regards,

Christina

On 18/04/2024 09:40, Ricardo Wurmus wrote:

Hi Simon,


So I propose to add the plumbing command ’derivation’.  Any objection?

I think it's useful to have.  To avoid proliferation of sub-commands, do
you think we could put this under "inspect", a generic sub-command for
all sorts of as yet to be invented introspection tools?

Re: Online social and patch review session on Monday

[Christina O'Donnell]

Hi,

Just to say that I'm looking forward to today's session! I'll try my 
best to be less quiet this time :-)


Kind regards,

Christina

On 12/04/2024 12:33, Steve George wrote:

Hi,

Our next online social and patch review session is on Monday (15th April):

   17:00 UTC; 18:00 London; 19:00 Paris; 13:00 New York

Please come along if you'd like to hangout and chat about Guix, and/or if you'd 
like to do some patch reviews. The sessions are very informal and we can run 
multiple virtual rooms if people want to pair and do patch reviews together.

If you have *simple* patches suitable for new contributors to review that have 
been forgotten (e.g. QA isn't handling them), you can add them to the 
'patch-review-hackers-list':

https://debbugs.gnu.org/cgi-bin/pkgreport.cgi?tag=patch-review-hackers-list;users=guix

This is done through a usertag on debbugs, to add your issue send email to 
cont...@debbugs.gnu.org

Subject: 
Body:

user guix
usertag  + patch-review-hackers-list
quit

As usual all the details on the Wiki:

https://libreplanet.org/wiki/Group:Guix/PatchReviewSessions2024

Hope to see you there!

Steve / Futurile

Re: Coordinators for patch review session on Tuesday

[Christina O'Donnell]

Hi,

Thanks for your reply,


1. Changing the tag to reviewed-looks-good

It doesn't look like this worked. The way to do this is in the instructions are 
4. 'Set a user tag' [0], probably the easiest way is to send an email (I do get 
funny results sometimes with my email client):

Subject: setting usertag on 65938

user guix
usertag 65938 + reviewed-looks-good
quit

The first line is important it has to be 'user guix' for it to appear on the 
patch review reports [1]. I think I messed up the instructions in the Wiki - 
you have to have a + in between the bug number and the tag you want to set 
(sorry about that). Please try again.


Ah I got it this time. I was missing the 'user guix'. I didn't read the 
wiki and tried to look it up from the debbugs documentation.



This is really just a way of signalling that reviews are happening - so trying 
to keep us in sync. The usertags we're using are:

- patch-review-hackers-list
- under-review
- escalated-review-request
- waiting-on-contributor
- reviewed-looks-good
If I change the patch quite a lot, should I mark it as 
'escalated-review-request' instead of 'reviewed-looks-good'?


And should I remove them from the patch-review-hackers-list after I've 
responded

The patch changes all look reasonable to me, you've already done a lot:

Great, thanks! Good to know I'm doing things vaguely right!

1. You should add a reviewed-by trailer:
Reviews are contributions from our community (and work!) so we should recognise 
them and add trailers. It also helps the maintainer know who did the review and 
therefore the level of confidence.

Basically just add 'Reviewed-by: A Person  - [2]

Sure, do you want me resubmit these patches to add that?

It looks like your updated patch retriggered QA, so if you look here and the 
foolow the Data Service link on the right you can see it's building it:

   https://qa.guix.gnu.org/issue/65938

The last step will be for a maintainer to see that it's built correctly, see 
your review and to apply it - great job for a first patch review!

Wonderful! The first of many, I'm hoping.

Kind regards,
Christina

Re: Coordinators for patch review session on Tuesday

[Christina O'Donnell]

Hi,

This is just to say that I went to review [2], but ended up making the 
changes myself, so I've submitted modified patches for those packages. 
Hopefully they're of a quality that's worth pushing.


I'm going to be busy this weekend, but I'll see if I get time to do some 
reviewing later on. It's actually quite fun!


Kind regards,

Christina

[2] https://debbugs.gnu.org/cgi-bin/bugreport.cgi?users=guix;bug=56576

On 03/04/2024 12:00, Christina O'Donnell wrote:

Hi Steve,

On 02/04/2024 21:23, Steve George wrote:

Hi Christina - thanks for coming along today - I hope it was useful.


Yes I did find it helpful. Since I'm the least experienced out of 
everyone there, I just stayed quiet and tried to absorb as much as I 
could.


It was good to see that not everyone was using Emacs, and I'm going to 
try to start using Efraim's vi script for GTAGS in Guile.



There's good instructions on the Wiki on how to review patches:

https://libreplanet.org/wiki?title=Group:Guix/PatchReviewSessions2024

I would love feedback on how to improve them!

There's plenty of patches to review, I've been keeping a list of them 
for the patch review calls:


https://debbugs.gnu.org/cgi-bin/pkgreport.cgi?tag=patch-review-hackers-list;users=guix 



And the wiki page references some other reports.

Please pick some patches and have a go - if you want someone else to 
look at them feel free to ping here or on IRC!


Thank you for writing this up in so much depth! I've reviewed [1] and 
tried to tag it as reviewed-looks-good, though I don't think that has 
gone through. If you or someone else could take a look at it then I'd 
appreciate that. I plan on reviewing some more patches this evening.


Kind regards,
Christina

[1] https://debbugs.gnu.org/cgi-bin/bugreport.cgi?users=guix;bug=65938

Re: Coordinators for patch review session on Tuesday

[Christina O'Donnell]

Hi Steve,

On 02/04/2024 21:23, Steve George wrote:

Hi Christina - thanks for coming along today - I hope it was useful.


Yes I did find it helpful. Since I'm the least experienced out of 
everyone there, I just stayed quiet and tried to absorb as much as I could.


It was good to see that not everyone was using Emacs, and I'm going to 
try to start using Efraim's vi script for GTAGS in Guile.



There's good instructions on the Wiki on how to review patches:

https://libreplanet.org/wiki?title=Group:Guix/PatchReviewSessions2024

I would love feedback on how to improve them!

There's plenty of patches to review, I've been keeping a list of them for the 
patch review calls:

https://debbugs.gnu.org/cgi-bin/pkgreport.cgi?tag=patch-review-hackers-list;users=guix

And the wiki page references some other reports.

Please pick some patches and have a go - if you want someone else to look at 
them feel free to ping here or on IRC!


Thank you for writing this up in so much depth! I've reviewed [1] and 
tried to tag it as reviewed-looks-good, though I don't think that has 
gone through. If you or someone else could take a look at it then I'd 
appreciate that. I plan on reviewing some more patches this evening.


Kind regards,
Christina

[1] https://debbugs.gnu.org/cgi-bin/bugreport.cgi?users=guix;bug=65938

Re: Coordinators for patch review session on Tuesday

[Christina O'Donnell]

Hi Steve,

I just wanted to say that I enjoyed the first one of these and I'm 
looking forward to today's session. I did want to go to the last 
session, but I lost track of time and missed it!


I'm a new contributor who's only sent a few patches up, but these 
sessions have been helpful for making contributing feel less 
intimidating. I haven't quite felt comfortable enough to review any 
patches on my own, so I do like the idea of pair-reviewing patches with 
someone who's a bit more experienced.


See you this evening!

Kind regards,

Christina

On 31/03/2024 22:58, Steve George wrote:

Hi all,

The next patch-review session is taking place on Tuesday 2nd of March [0] and 
I'd love to try pair-programming where groups can actively work on some patch 
reviews.

Is anyone willing to 'co-ordinate' a pair programming session?

Last time I set-up a cloud host and installed Upterm (https://upterm.dev/) so 
that everyone could ssh into a session. We could run 4-5 simultaneous sessions 
where people could 'pair' to do patch reviews together.

To co-ordinate a session I'll give you SSH access and there are instructions on 
how to launch the Upterm session. We have written instructions on some basic 
tools to do the patch reviews - and as Guix is installed for each user you can 
add your own ;-)

Anyone up for it?

I'm also collecting simple patches for review onto a list so they can be 
assigned to each group:

https://debbugs.gnu.org/cgi-bin/pkgreport.cgi?tag=patch-review-hackers-list;users=guix

Feel free to add simple issues there!

Thanks,

Futurile / Steve

[0] 
https://libreplanet.org/wiki/Group:Guix/PatchReviewSessions2024#Patch_Review_Sessions_2024

Re: Mechanism for helping in multi-channels configuration (and Xapian index)

[Christina O'Donnell]

Hi Simon,

Sorry for the really long delay, I meant to reply after I'd had a good 
read through the conversation you linked, but I haven't had a chance to 
really get into it yet, but I have read enough to get a surface idea of 
the project. The project looks fun, and looks like it will help Guix 
users and developers so I'd be on board in principle.


On 15/02/2024 15:05, Simon Tournier wrote:

...

I think you mean ’fold-packages’.


   2. Have a script that determines the symbols needed by each file. (Macros
      make this more difficult, but.)

Well, this would be difficult, IMHO.  Somehow, it is what the compiler
does. :-)
I asked on guile-devel and Maxime suggested using `module-map` or 
`module-for-each` which map over all symbols in a module. Presumably 
this would know what's quotes literally and what's a proper symbol.

   3. Have both scripts have an incremental version that runs on diffs (for
      performance).
   4. Run this for every commit on every branch on every channel caching the
      result.
   5. Have a CI script keep this updated for new commits.
   6. Have a server track incompatibilities.

Here, I think the issue is that one server needs to track all the
channels.  And that’s a too strong assumption, IMHO.

I think the design should be something on channel maintainer side.
Somehow, the main Guix channel could be seen as a Git submodule from the
channel side and the issue is that information is not tracked.

There is this ’.guix-channel’ file which allows to describe channel
dependencies.  And the improvements could be to add more there.  The
question is what to add and how to add it.  Keeping in mind the
simplicity and the maintenance burden-free. :-)


Okay, this makes sense. I'm thinking that you could have something like 
a sqlite index that can be generated by running a script on the code. 
The index could exist on the server separate to the channel repo, 
pointed at by the .guix-channel file. The commit hook could: (1) update 
the local index to include the latest commit; (2) update the hash inside 
.guix-channel. Then a push hook could also push the index to the server.


It's a bit clunky because you've got this binary blob that you have to 
synchronize with the channel, and it's easy to get this wrong. Putting 
the index in the channel repo would bloat the channel with old versions 
of the index. Forcing users to generate the index from scratch is 
undesirable too.


As an alternative to having the index referenced in the .guix-channel, 
we could use git-annex. This would take care of: Fetching the index, 
uploading the new index on push, and updating the hash. No extra steps 
would be /required/ by developers, as it won't be necessary to have the 
index 100% up to date. But developers could choose to regenerate the 
index and call `git annex sync`. I suspect that adding git-annex as a 
dependency would be resisted, but that's the way I think would work 
best. And could apply to existing indexes. It depends on how long it 
takes to generate the index from scratch. There was some talk of 
data.guix.gnu.org using PostgreSQL to index packages. I suppose it'd be 
worth figuring out what they do to see if they have anything sql or code 
that might be portable to sqlite.



Full disclosure: I've got nothing lined up for the summer yet, so I'm on the
prowl for GSoC projects :)

Cool!

In that spirit, one tool that is missing is: search packages in all the
history. Somehow the need is described by this message [1]: how to find
which Guix revision provides which version of Foo?

In addition, “guix search” is slow [2].

Well, I have started the embryo of an extension based on Guile-Xapian
for indexing and improving the search.  Really an embryo. :-)

I think this would fit some GSoC. ;-)


As I said above, [2] is a fairly long thread, but I think I get the 
general idea. It seems that Xapian was implemented but didn't have the 
desired speedup. Am I getting the right impression there?


It's certainly an interesting problem. I'll keep thinking about it.

Kind regards,

Christina


1: Re: List available versions of package.
Philippe Veber 
Tue, 11 Jun 2019 09:43:08 +0200
id:CAOOOohSzUezKvm=ro0bxrgh3m0eo2x0cotvd--varxwoqtc...@mail.gmail.com
https://lists.gnu.org/archive/html/help-guix/2019-06
https://yhetil.org/guix/CAOOOohSzUezKvm=ro0bxrgh3m0eo2x0cotvd--varxwoqtc...@mail.gmail.com

2: https://issues.guix.gnu.org/issue/39258

Re: Golang mudules to follow common grouping

[Christina O'Donnell]

Hi,

Okay that's a better plan :)


Let's use  Occam's razor for now. We have few common groups,
the task is to drop use-module (gnu packages golang) for each
of them by sorting packages into recently introduced modules.
Right, put the dependencies in of golang-web in golang-web (etc.) as 
much as sensibly possible.


How about when a library is used by many packages in more than one 
different modules?


Kind regards
- Christina

On 05/02/2024 14:00, Sharlatan Hellseher wrote:

Hello,

Thank you for your research on that, I was not expected to go
far like you did :-)

My short plan was to follow existing naming model which is in
use for python-*.scm, lisp-*.scm, perl-*scm, java-*.scm.
I see that golang would need some extra modules in the future
and comparing with python-*.scm the package base is not big
yet.

Let's use  Occam's razor for now. We have few common groups,
the task is to drop use-module (gnu packages golang) for each
of them by sorting packages into recently introduced modules.

- golang-web.scm
- golang-check.scm
- golang-xyz.scm
- golang-crypto.scm

Thanks,
Oleg

Re: Golang mudules to follow common grouping

[Christina O'Donnell]

Hi again Oleg,

> > I think you may help! The identification of the group is still human
> > decision making process and I'm not sure it may be automated in any 
point.

>
> I can certainly help with this then. I'll have some free time on 
Friday and

> I can coordinate with you then.
Ah, I lost track of time..

Okay, so I wrote a grotesque shell script that extracted the name, synopsis,
and description from each package. Then I manually chunked the output into a
transformer, asking it to:

  "Given a list of packages and descriptions, please[1] suggest ways of
   categorizing them (by function or theme) in a way that keeps each 
category

   about equal size."

...followed by 5 paragraphs of minutia to make sure it does the right thing.

Then I give it 200 lines of packages ~36 packages. It gives me its output
which is inevitably missing one or more of the constraints I gave it, so I
say "Did you follow my instructions?" [2] After this, it either gives me an
output with one less thing wrong with it and an apology, or goes off in a
huff and wastes all my GPU credit.


Things that are standing out so far are:

 - golang-web.scm could become a broader golang-networking.scm
 - golang-crypto.scm could become a broader golang-security.scm
 - There could be a need for a golang-text.scm that focusses on text 
parsing and

   processing.
 - A golang-utils could exist, however this could easily become the 'other'
   file.

Anyway, point is: I've got a very poor classification of packages in
golang.scm. I want to refine that using a langchain script. However, before
I move anymore forward with this, I wanted to ask: Is it okay to use AI
tools that are non-free in the aid of the GNU project?

> > We may extend handy script accelerating committing process, see
> > "etc/committer.scm"
>
> Okay, cool, I'll have a look at it on Friday.

Great, I haven't had a chance to look at this yet, but I will early next 
week.


Here's a link to my repo: https://github.com/cdo256/guix-package-refactor

Let me know what you think.

Kind regards,
 - Christina

[1] I am always polite to AI.
[2] Welcome to modern programming, people. Feels more like primary school
children.

Re: Issues with issues.guix.gnu.org (502 Bad Gateway)

[Christina O'Donnell]

Hi Felix,


You are welcome to use my Mumi clone at mumi.juix.org.

Bookmarked, thanks!

Looks like issues.guix.gnu.org is back up again.

Kind regards,
 - Christina

On 04/02/2024 05:33, Felix Lechner wrote:

Hi Christina,

On Sat, Feb 03 2024, Christina O'Donnell wrote:


connecting to https://issues.guix.gnu.org/ results in ... a 502 bad
gateway.



Kind regards
Felix

Issues with issues.guix.gnu.org (502 Bad Gateway)

[Christina O'Donnell]

Hi Guix,

From my machine[1] connecting to https://issues.guix.gnu.org/ results 
in, after 130 seconds[2], a 502 bad gateway. It's been having issues for 
over a week, but I only just found a need to test it.


I couldn't see an issue about it on debbugs so I thought it prudent to 
raise an issue.


Reproduced with curl, firefox, and ungoogled-chromium on the main page 
and on specific issues.


It appears to be under a lot of load.

Kind regards,
 - Christina

[1] based in Cambridge, UK, and via my VPS in Dusseldorf, Germany on 
2024-02-03 19:30 UTC+0


[2] Rather consistently 130-140 seconds from both UK and Dusseldorf. 
Apparently the server is in Berlin.

Re: Mechanism for helping in multi-channels configuration

[Christina O'Donnell]

Hi Simon,


The wishlist is: provide a machine-readable description on guix-science
channel side in order to help in finding the good overlap between
commits of different channels.

It could be nice if instead of an hard error, “guix pull” could say:
« the channel ’guix’ needs to be at least at commit 1234abc ».


I was just thinking about these kinds of errors. It would also happen 
between

channels when packages are split from a single file (eg. golang.scm to
golang-xyz.scm). Then channels immediately go out of sync as we're doing
continual releases. So, it wouldn't just be for time-machine. It's all a 
bit too
fragile for my liking. I assume we won't be to frequent versioned 
releases any

time soon..

A sketch of a solution might be:

 1. Have a script that scrapes all the define-public symbols in every 
file in

    every package.
 2. Have a script that determines the symbols needed by each file. (Macros
    make this more difficult, but.)
 3. Have both scripts have an incremental version that runs on diffs (for
    performance).
 4. Run this for every commit on every branch on every channel caching the
    result.
 5. Have a CI script keep this updated for new commits.
 6. Have a server track incompatibilities.

For example, a 'definition-reference' could look like,

(definition-reference (commit-range start-hash end-hash)
  file-path
  identifier)

(definition-reference (commit-range "44b340d..." "06dba3b...")
  "gnu/packages/golang"
  'go-github-com-rs-xid)

Commit ranges makes the size of entries tractable (since package 
probably aren't

getting moved / deleted / added very much).

Then use a hash table, (or trie or B+ Tree, or distributed hash table, 
etc) to

go from identifier to definition-reference.

You would probably would also want to know commit date so you could 
index on it.
That would let you find versions that supplied the identifier that are 
as close

as possible chronologically to a particular version of a different channel

Now this isn't perfect (in case anyone was getting that impression ;):
 - It won't have any idea about version incompatibilities.
 - It couldn't trace renamed variables.
 - And probably more.

Might be useful to additionally track package versions, but that might 
run into

resource issues.

I'm thinking a Guile daemon backed by SQLite.. What do you think?

Full disclosure: I've got nothing lined up for the summer yet, so I'm on the
prowl for GSoC projects :)

Kind regards,
 - Christina

Re: Golang mudules to follow common grouping

[Christina O'Donnell]

Hi Oleg,


I've pushed the split III to master.


Fantastic work!


I think you may help! The identification of the group is still human
decision making process and I'm not sure it may be automated in any point.


I can certainly help with this then. I'll have some free time on Friday and
I can coordinate with you then.


Each of the module header contains a  short annotation which packages it
 expects to have, feel free to improve it to make it even more clear
for  others.

-  golang-check
-  golang-web
-  golang-crypto

TBA:
-  golang-compression :: Anything related to that subject, see
 python-compression, java-compression, perl-compression.

-  golang-build or golang-extension :: Any low level golang add-ons not
 included in core distribution see  or
any 0  dependencies high reference modules.

-  golang-xyz :: As any other *-xyz module would absorb anything else
  left  behind.


This all sounds sensible to me.


1.  Put a magic comment above each package that you would like to move.
2.  Run a simple script that makes a note of all of these into a
 to-move-list.
3.  Then stash the change with the comments you made (in case you need
to  change things)
4.  Run another script that takes the package list and performs the
 move in one's repository.
5.  Sort out the use-package declarations manually and run tests.
6.  When satisfied, stash the change and keep just the use-package
 changes.
7.  Run a final script that loops through all the packages and commits
 each one in turn.
8.  Rebase to suit.


We may  extend handy script accelerating committing process, see
 "etc/committer.scm"


Okay, cool, I'll have a look at it on Friday.


-  I'm not a scheme programmer, but I did use Haskell at university so
 I'm familiar with thinking in a functional style.

Me too  =), but you still can help by just providing some review to
 existing code base and available packages in golagn.scm and trying to
 identify close group for each of them.


 I'm also imagining some the possibility of having a script that can
 remove redundant #:use-module's in the future, though I don't know if
we  care about a few unneeded modules being included.

The  clean up task may be organasied after sort process is completed, having
not  required #:use-module does not hurt too much but for keeping modules
tidy  and fast to load it definitely beneficial.


This makes sense. Not a priority at present but a nice-to

Looking forward to hacking golang.scm to pieces!

Kind regards,
 - Christina

Re: Golang mudules to follow common grouping

[Christina O'Donnell]

Hi Oleg,

> I've added comments in commentary section in the top of the file 
asking to

> keep packages alphabetically sorted seen in julia-xyz.scm as well.
> python-*.scm ordered semi random grouped closer to package purpose which
> require more thinking where to put a new one :-)

Ah, I suppose it isn't that important given you can grep for the right
package.

> Good point her, I did manual split, with Emacs keyboard macros, magit 
history
> scan for copyright lines and manual check where package was used to 
include

> new module name.
>
> The split into golang-crypto is in review now and there would be 2 
more common

> grouping: golang-compression and golang-build (or golang-extension). Rest
> packages which are hard to determine a group wound go to generic 
golang-xyz

> sorted alphabetically.

Hmm, there seems to be a limit in the degree of parallelizability in this
process unfortunately. But if there's anything you can think of that 
could help

(manually) in this effort, then I'd be happy to help!

> Let me know your tooling which you familiar with I might think
> about some sort of automation.

I've had a grep around the web and I can't see any tools or libraries 
that have
been created already to perform functions like this. I know everyone has 
their
own scripts (I've got one written in shell but it won't be of much use 
here. I

would suggest writing it in Scheme so it could be easier to add more complex
features and it's familiar to Guix developers.

 1. Put a magic comment above each package that you would like to move.
 2. Run a simple script that makes a note of all of these into a 
to-move-list.
 3. Then stash the change with the comments you made (in case you need 
to change

    things)
 4. Run another script that takes the package list and performs the move in
    one's repository.
 5. Sort out the use-package declarations manually and run tests.
 6. When satisfied, stash the change and keep just the use-package changes.
 7. Run a final script that loops through all the packages and commits 
each one

    in turn.
 8. Rebase to suit.

That's 3 scripts:
 - xyz-source-transform-read-packages scm-filename package-list (for 
step 2)
 - xyz-source-transform-move-packages package-list from-filename 
to-filename

   (for step 4)
 - xyz-source-transform-stage-package-commit package-list from-filename
   to-filename (for step 7)

(Where 'xyz' is either 'guix' or something else depending on where the 
code ends

up part of Guix or not. That's not important for now.)

Some caveats:
 - I'm not a scheme programmer, but I did use Haskell at university so I'm
   familiar with thinking in a functional style.
 - For this to work, you'd have to update most package files that 
reference (gnu

   package golang) to also reference the new package, so there will be some
   use-module redundancy (which could be resolved in a final clean-up 
commit).


I'm also imagining some the possibility of having a script that can remove
redundant #:use-module's in the future, though I don't know if we care 
about a

few unneeded modules being included.

I'd love to hear what you think before I fire up another emacs instance. 
Does it

sound like a realistic workflow?

All comments welcome!

Kind regards,
 - Christina

Re: Golang mudules to follow common grouping

[Christina O'Donnell]

Hi Oleg,

On 13/01/2024 21:05, Sharlatan Hellseher wrote:
> Hi Guix,
>
> I'm about to prepare split and aggregation of all golag packages
>  related to cryptography. The process would be the same as for
>  golang-check and golang-web.
>
>
> In progress:
> golang-cryptography
>
> Planned:
> golang-compression
> golang-build

I just wanted to offer my labor to this task. I'm very new to guix 
development
(I've only packaged one package so far), but I've got a fair amount of 
free time
and I've been looking for ways to contribute to Guix. (Plus I've already 
got my

head in golang.scm trying to package gitleaks and all its dependencies.)

I know that there's a lot to do already, but I was also wondering whether
they're going in any particular order once they're moved. There's a 
comment at

the end of golang.scm that says,

> ;;; Avoid adding new packages to the end of this file. To reduce the 
chances
> ;;; of a merge conflict, place them above by existing packages with 
similar

> ;;; functionality or similar names.

Would it be more organized if they was just one order: either in 
alphabetical
order or grouped by function? My suggestion would be to use the file 
split to
group by function and then sort each file alphabetically. Do you know 
how it is

arranged for other languages?

Another question I have: Is there any tooling that can help big package
migrations like this go faster? Eg. a script to split one big diff into
individual package moves with appropriate change-log entries.

If you could share any scripts that you're using then that could help speed
up similar tasks in future.

Kind regards,
 - Christina

Re: Kdump

[Christina O'Donnell]

  
  
On 31/07/2023 18:45, Csepp wrote:


  
Christina O'Donnell  writes:


  
Hi guix and guixesses,

I'm still enjoying my guix machine crashing every other week despite changing all the software and half the hardware. So I'm trying to get kdump
working so I can get to the real reason behind it. However I see that kdump-tools haven't been packaged yet. I see this as an opportunity for me to
contribute to Guix, but it'll be my first time.

- How interested would people be in me packaging kdump and related tools?

  
  
More debugging tools (and docs!) are always welcome IMHO.


  
- Is there a reason why it's not there already?

- Has it been tried before?

  
  
Searching the mailing lists doesn't turn up much, so I assume no one has
tried it:
https://yhetil.org/guix/?q=kdump


  
The scope seems to be around 3-4 packages and a system service. Does that sound about right or could there be more I'm missing?

On Debian there's:

crash/testing,now 8.0.2-1 amd64 [installed,automatic]
  kernel debugging utility, allowing gdb like syntax

kdump-tools/testing,now 1:1.8.1 amd64 [installed]
  scripts and tools for automating kdump (Linux crash dumps)

libkdumpfile-dev/testing 0.5.1-1 amd64
  libkdumpfile development libraries and header files

libkdumpfile-doc/testing,testing 0.5.1-1 all
  Kernel coredump file access (documentation)

libkdumpfile10/testing 0.5.1-1 amd64
  Kernel coredump file access

python3-libkdumpfile/testing 0.5.1-1 amd64
  Python bindings for libkdumpfile

I'd want to package all of these except the python bindings. I see that kexec-tools is already in guix which is good!

Is this a sensible direction?

Kind regards,
 - Christina

  
  
I'd say go for it!  If they are mostly written in C then you probably
don't have to package a lot of transitive dependencies.  Looking in
gnu/packages/linux.scm could be a good starting point for packaging
kernel related tools.


Ah I must have missed this in my inbox. Thanks for the feedback!
  I'll have a go at this once some more time appears in my day.

  

Kdump

[Christina O'Donnell]

  
  
Hi guix and guixesses,
  
  I'm still enjoying my guix machine crashing every other week
  despite changing all the software and half the hardware. So I'm
  trying to get kdump working so I can get to the real reason behind
  it. However I see that kdump-tools haven't been packaged yet. I
  see this as an opportunity for me to contribute to Guix, but it'll
  be my first time.

- How interested would people be in me packaging kdump and
  related tools?
- Is there a reason why it's not there already?
- Has it been tried before?
The scope seems to be around 3-4 packages and a system service.
  Does that sound about right or could there be more I'm missing?

  On Debian there's:
  
  crash/testing,now 8.0.2-1 amd64
[installed,automatic]
  kernel debugging utility, allowing gdb like syntax

kdump-tools/testing,now 1:1.8.1 amd64 [installed]
  scripts and tools for automating kdump (Linux crash dumps)

libkdumpfile-dev/testing 0.5.1-1 amd64
  libkdumpfile development libraries and header files

libkdumpfile-doc/testing,testing 0.5.1-1 all
  Kernel coredump file access (documentation)

libkdumpfile10/testing 0.5.1-1 amd64
  Kernel coredump file access

python3-libkdumpfile/testing 0.5.1-1 amd64
  Python bindings for libkdumpfile
I'd want to package all of these except the python bindings. I
  see that kexec-tools is already in guix which is good!
Is this a sensible direction?
  
  Kind regards,
   - Christina

  

python-nbconvert build fails

[Christina O'Donnell]

  
  
Hi Guix! This is my first time posting on a GNU mailing list so
  I'd appreciate feedback on clarity and wording.
  
  I've run `guix pull` and `guix package -u` after a couple weeks
  and I've hit a build failure:
  
  builder for
`/gnu/store/wv5clm6yl9hfghdsdqv0y2j749p1snhy-python-nbconvert-6.5.0.drv'
failed with exit code 1
It seems that some tests have failed:
  
  === short test
summary info 
FAILED exporters/tests/test_pdf.py::TestPDF::test_export -
nbconvert.exporter...
FAILED
tests/test_nbconvertapp.py::TestNbConvertApp::test_filename_accent_pdf
FAILED
tests/test_nbconvertapp.py::TestNbConvertApp::test_filename_spaces
- O...
FAILED
tests/test_nbconvertapp.py::TestNbConvertApp::test_linked_images
- OSE...
FAILED tests/test_nbconvertapp.py::TestNbConvertApp::test_pdf -
OSError: [NbC...
= 5 failed, 282 passed, 52 warnings in 13.77s
==
These all seem to have come from xelatex. In the first it appears
  as xlatex not finding a format file. In the later ones, however,
  the error is unclear.

E  
nbconvert.exporters.pdf.LatexFailed: PDF creating failed,
captured latex output:
E   Failed to run
"['/gnu/store/hyrpg4gnb92wfn5w9yjckfnwq4v50g4g-texlive-bin-20230313/bin/xelatex',
'notebook.tex', '-quiet']" command:
E   This is XeTeX, Version
3.141592653-2.6-0.95 (TeX Live 2023/GNU Guix) (preloaded
format=xelatex)
E    restricted \write18 enabled.
E
E   kpathsea: Running mktexfmt xelatex.fmt
E   mktexfmt: No such file or directory
E   I can't find the format file
`xelatex.fmt'!

  E  
nbconvert.exporters.pdf.LatexFailed: PDF creating failed,
captured latex output:
E   Failed to run
"['/gnu/store/hyrpg4gnb92wfn5w9yjckfnwq4v50g4g-texlive-bin-20230313/bin/xelatex',
'notebook.tex', '-quiet']" command
I couldn't glean anything else from the output. I've attached the
  build log if you're interested.
  
  When I tried checking out the source and used `guix shell -D python-nbconvert` I
  appear to have tripped over something else that might be telling:
cdo@peter
/t/guix-build-python-nbconvert-6.5.0.drv-0> guix shell -D
python-nbconvert --debug=3 -K &| tee
/tmp/nbconvert-shell.out
The following derivation will be built:
  /gnu/store/06di2r17gyji5n0pgyg39i69dpqcc923-profile.drv

building path(s)
`/gnu/store/nywxlfivxzrzvicba102wq7in0jrb06w-texlive-font-maps'
|   setting up chroot environment in
`/gnu/store/b72xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv.chroot'
|   executing builder
`/gnu/store/g8p09w6r78hhkl2rv1747pcp9zbk6fxv-guile-3.0.9/bin/guile'
|   @ build-started
/gnu/store/b72xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv
- x86_64-linux
/var/log/guix/drvs/b7//2xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv.gz
5379
note: keeping build directory
`/tmp/guix-build-texlive-font-maps.drv-0'
builder for
`/gnu/store/b72xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv'
failed with exit code 1
build of
/gnu/store/b72xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv
failed
View build log at
'/var/log/guix/drvs/b7/2xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv.gz'.
cannot build derivation
`/gnu/store/06di2r17gyji5n0pgyg39i69dpqcc923-profile.drv': 1
dependencies couldn't be built
guix shell: error: build of
`/gnu/store/06di2r17gyji5n0pgyg39i69dpqcc923-profile.drv' failed
  
That ends up with a log with no clear error that ends in:
  
ERROR: In procedure copy-file:
In procedure copy-file: Permission denied
On ci.guix.gnu.org, it's failing for tex-team, python-team and
  rust-team since 11th July. I reproduced it on the latest staging
  and master. I'm happy to investigate the matter further. Should I
  raise a bug first?
Kind regards,
   - Christina

  


2xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv.gz
Description: application/gzip


5clm6yl9hfghdsdqv0y2j749p1snhy-python-nbconvert-6.5.0.drv.gz
Description: application/gzip

Re: python-nbconvert build fails

[Christina O'Donnell]

  
  
Sorry, I've just seen this is a duplicate of
  https://issues.guix.gnu.org/64729. I should have checked there
  first!

On 23/07/2023 20:15, Christina
  O'Donnell wrote:


  
  Hi Guix! This is my first time posting on a GNU mailing list so
I'd appreciate feedback on clarity and wording.

I've run `guix pull` and `guix package -u` after a couple weeks
and I've hit a build failure:

builder for
  `/gnu/store/wv5clm6yl9hfghdsdqv0y2j749p1snhy-python-nbconvert-6.5.0.drv'
  failed with exit code 1
  It seems that some tests have failed:

=== short test
  summary info 
  FAILED exporters/tests/test_pdf.py::TestPDF::test_export -
  nbconvert.exporter...
  FAILED
  tests/test_nbconvertapp.py::TestNbConvertApp::test_filename_accent_pdf
  FAILED
  tests/test_nbconvertapp.py::TestNbConvertApp::test_filename_spaces
  - O...
  FAILED
  tests/test_nbconvertapp.py::TestNbConvertApp::test_linked_images
  - OSE...
  FAILED tests/test_nbconvertapp.py::TestNbConvertApp::test_pdf
  - OSError: [NbC...
  = 5 failed, 282 passed, 52 warnings in 13.77s
  ==
  These all seem to have come from xelatex. In the first it
appears as xlatex not finding a format file. In the later ones,
however, the error is unclear.
  
  E  
  nbconvert.exporters.pdf.LatexFailed: PDF creating failed,
  captured latex output:
  E   Failed to run
"['/gnu/store/hyrpg4gnb92wfn5w9yjckfnwq4v50g4g-texlive-bin-20230313/bin/xelatex',
  'notebook.tex', '-quiet']" command:
  E   This is XeTeX, Version
  3.141592653-2.6-0.95 (TeX Live 2023/GNU Guix) (preloaded
  format=xelatex)
  E    restricted \write18 enabled.
  E
  E   kpathsea: Running mktexfmt xelatex.fmt
  E   mktexfmt: No such file or directory
  E   I can't find the format file
  `xelatex.fmt'!
  
E  
  nbconvert.exporters.pdf.LatexFailed: PDF creating failed,
  captured latex output:
  E   Failed to run
"['/gnu/store/hyrpg4gnb92wfn5w9yjckfnwq4v50g4g-texlive-bin-20230313/bin/xelatex',
  'notebook.tex', '-quiet']" command
  I couldn't glean anything else from the output. I've attached
the build log if you're interested.

When I tried checking out the source and used `guix shell -D python-nbconvert` I
appear to have tripped over something else that might be
telling:
  cdo@peter
  /t/guix-build-python-nbconvert-6.5.0.drv-0> guix shell -D
  python-nbconvert --debug=3 -K &| tee
  /tmp/nbconvert-shell.out
  The following derivation will be built:
    /gnu/store/06di2r17gyji5n0pgyg39i69dpqcc923-profile.drv
  
  building path(s)
  `/gnu/store/nywxlfivxzrzvicba102wq7in0jrb06w-texlive-font-maps'
  |   setting up chroot environment in
`/gnu/store/b72xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv.chroot'
  |   executing builder
  `/gnu/store/g8p09w6r78hhkl2rv1747pcp9zbk6fxv-guile-3.0.9/bin/guile'
  |   @ build-started
  /gnu/store/b72xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv
  - x86_64-linux
/var/log/guix/drvs/b7//2xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv.gz
  5379
  note: keeping build directory
  `/tmp/guix-build-texlive-font-maps.drv-0'
  builder for
  `/gnu/store/b72xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv'
  failed with exit code 1
  build of
  /gnu/store/b72xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv
  failed
  View build log at
'/var/log/guix/drvs/b7/2xsvb29yr6rx29ca0mz5937vaz24ny-texlive-font-maps.drv.gz'.
  cannot build derivation
  `/gnu/store/06di2r17gyji5n0pgyg39i69dpqcc923-profile.drv': 1
  dependencies couldn't be built
  guix shell: error: build of
  `/gnu/store/06di2r17gyji5n0pgyg39i69dpqcc923-profile.drv'
  failed

  That ends up with a log with no clear error that ends in:

  ERROR: In procedure copy-file:
  In procedure copy-file: Permission denied
  On ci.guix.gnu.org, it's failing for tex-team, python-team and
rust-team since 11th July. I reproduced it on the latest staging
and master. I'm happy to investigate the matter further. Should
I raise a bug first?
  Kind regards,
 - Christina