Re: Losing signing keys for custom Guix channel

[elaexuotee]

Jake  wrote:
> If commit i adds a new signing key to the channel’s authorisations file and
> commit i+1 is signed with that signing key, then commit i+1 can be used in
> channel intro.
> 
> You can’t add a signing key to the authorisations in a commit and sign that
> same commit with the new key.  Is that issue here?

I don't think that's completely accurate. My original channel introduction
commit is precisely the one creating a .guix-authorizations file with my old
key info.

I can certainly add an extra signing key to the authorizations; I just can't
sign that commit with the old key, since the old key has been lost.

Re: Losing signing keys for custom Guix channel

[elaexuotee]

> > from reading about guix authentication I think the new signing key
> > must be first added to the .guix-authoriations file and that commit
> > must signed with the current signing keys before the new signing
> > key can be used.
> 
> Yes, it’s likely the problem; the rest of the description you gave
> elaexuotee looks fine to me.
> 
> (No need to rewrite the history; changing the introduction is enough.)
> 
> Ludo’.

Well, the catch 22 is that I've lost the original key and so can only sign
.guix-authorizations with the new one.

> (No need to rewrite the history; changing the introduction is enough.)

Without the old key, I'm gathering that a history rewrite is the only way right
now. Seems like a fresh channel introduction should be enough, but our current
authorization check appears to look at earlier commits even in that case, IIUC.

Maybe forcing history rewrites on key loss is the desired behavior? I'm not
sure. From a client perspective, the only difference is whether or not you have
to specify --allow-downgrades on the next pull. In either case a channel intro
update is necessary.

Re: guix --container is RAM hungry

[Ludovic Courtès]

Hi Edouard,

Edouard Klein  skribis:

> I'm a huge fan of guix --container, and I created a system to use those
> by default for network services. But the VPS these services run on has
> only 2GB of RAM, and I just realized that a container, by default,
> requires at least 200MB.

Ouch, confirmed:

--8<---cut here---start->8---
$ \time -v guix shell -C coreutils -- uname 2>&1 |grep 'Maximum resident'
Maximum resident set size (kbytes): 283048
$ \time -v guix shell coreutils -- uname 2>&1 |grep 'Maximum resident'
Maximum resident set size (kbytes): 56588
$ guix describe
Generation 297  Mar 24 2024 23:12:25(current)
  guix 28bc0e8
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 28bc0e870b4d48b8e3e773382bb0e999df2e3611
--8<---cut here---end--->8---

As raingloom and Ricardo wrote, there’s a Guile process that keeps
waiting.

The ‘-C’ variant has to load more modules and do more work (compute
derivations), so it’s not surprising that it takes more memory than the
other variant (on a cache hit it has nothing to do).

Merely loading (guix scripts shell) consumes a lot, but GC says it’s
“only” using 7 MiB out of this:

--8<---cut here---start->8---
$ \time -f %M guile  -c '(use-modules (guix scripts shell)) (pk (quotient 
(assoc-ref (gc-stats) (quote heap-size)) 1024))'

;;; (7204)
38272
--8<---cut here---end--->8---

So what we’re seeing here is probably the mappings made by the loader
(libguile/loader.c).

Needs more investigation…

Ludo’.

Re: Losing signing keys for custom Guix channel

[Ludovic Courtès]

Hello,

Markku Korkeala  skribis:

> On Mon, Mar 25, 2024 at 02:41:26PM +0900, elaexuo...@wilsonb.com wrote:

[...]

>> Here are the changes I've made:
>> - New public key added to keyring branch
>> - Appended new key fingerprint to .guix-authorizations (at commit X)
>> - Update introduction in .config/guix/channels.scm
>>   - Point to commit X
>>   - Update openpgp-fingerprint
>> 
>> As a sanity check, I've confirmed that the fingerprint on commit X, the
>> fingerprint in .guix-authorizations, and the openpgp-fingerprint in my
>> channels.scm are all the same.
>> 
>> What am I missing?
>
> Hi all,
>
> from reading about guix authentication I think the new signing key
> must be first added to the .guix-authoriations file and that commit
> must signed with the current signing keys before the new signing
> key can be used.

Yes, it’s likely the problem; the rest of the description you gave
elaexuotee looks fine to me.

(No need to rewrite the history; changing the introduction is enough.)

Ludo’.

Re: Shepherd timers

[Ludovic Courtès]

Hello!

Felix Lechner  skribis:

> The simple example worked great, but my more complex attempt to convert
> an mcron job seems to have failed.  Do you know why?
>
> (define (rsync-debbugs-shepherd-service config)
>   (shepherd-service
>(provision '(rsync-debbugs))

Nice use case.  :-)

>> herd restart rsync-debbugs
> Service user-homes has been started.
> Starting service rsync-debbugs...
> Service rsync-debbugs started.
> Service rsync-debbugs running with value #< channel: #< getq: 
> # getq-gc-counter: # 7fb467c1c980 value: 42> putq: # 
> putq-gc-counter: #> event: 
> #< seconds: (0) minutes: (iota 12 3 5) hours: (0 1 2 3 4 5 6 
> 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23) days-of-month: (1 2 3 4 5 6 
> 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31) 
> months: (1 2 3 4 5 6 7 8 9 10 11 12) days-of-week: #f> action: #< 
> arguments: 
> ("/gnu/store/1g0gk9pcymgc0bb1dh115swakmh438p6-rsync-3.2.7/bin/rsync" "-4" 
> "--delete" "-av" "--password-file=/root/secrets/var-lib-debbugs.rsync" 
> "debbugs-...@debbugs.gnu.org::var-lib-debbugs" "/var/lib/debbugs/") user: #f 
> group: #f environment-variables: ("HOME=/" "TERM=linux" 
> "BOOT_IMAGE=/gnu/store/psn2jwyrvr8jxb1nrndxf3h7kg8vvfwa-linux-6.6.2/bzImage" 
> "PATH=/gnu/store/2jbvnib5ar70q1vlhhngish1svqfry0j-e2fsck-static-1.46.4/sbin:/gnu/store/jlbzcgxxzhynv74ss2510b0x978srnf9-loadkeys-static-2.5.1/bin"
>  
> "GUIX_LOCPATH=/gnu/store/5fmqijrs5f7vx8mc2q2pmq26yvhb74sm-glibc-utf8-locales-2.35/lib/locale")
>  directory: "/" resource-limits: ()>>.
> Service rsync-debbugs has been started.
> #[STATUS] End time 2024-03-25 14:04:40, duration 1.081s
> (root@wallace-server.local)[/s/srv/patchwise]
>> herd status rsync-debbugs
> #[STATUS] End time 2024-03-25 14:07:41, duration 159.524s

So ‘herd status rsync-debbugs’ exited after 160s without printing
anything?  What does /var/log/messages say?

I tried something similar on my laptop but it seems to work.

Namely, I have:

--8<---cut here---start->8---
(use-modules (shepherd service timer))

(define gc-timer
  (service
   '(gc)
   #:start (make-timer-constructor
(calendar-event #:minutes '(0))
(command '("/run/current-system/profile/bin/guix" "gc" "-F2G")
 #:user "ludo"))
   #:stop (make-timer-destructor)
   #:actions (list timer-trigger-action)))

(register-services (list gc-timer))
--8<---cut here---end--->8---

And then I do:

  sudo herd load root that-file.scm

I start, restart, reload, restart, etc. and it seems fine.

Thanks for trying it out!

Ludo’.

Re: Shepherd timers

[Ludovic Courtès]

Hi,

raingl...@riseup.net skribis:

> Does it support RTC wakeup?

No.  I don’t have plans but it’d be nice to have.

Ludo’.

Re: PyTorch with ROCm

[Ludovic Courtès]

Hello!

David Elsing  skribis:

> after seeing that ROCm packages [1] are available in the Guix-HPC
> channel, I decided to try and package PyTorch 2.2.1 with ROCm 6.0.2.

Nice!

> The changes for the ROCm packages are here [4] as a modification of
> Guix-HPC. There, the python-pytorch-rocm package in
> amd/machine-learning.scm depends on the python-pytorch-avx package in
> [2,3]. Both python-pytorch and python-pytorch-avx support AVX2 / AVX-512
> instructions, but the latter also has support for fbgemm and nnpack. I
> used it over python-pytorch because AVX2 or AVX-512 instructions should
> be available on a CPU with PCIe atomics anyway, which ROCm requires.

I’m happy to merge your changes in the ‘guix-hpc’ channel for the time
being (I can create you an account there if you wish so you can create
merge requests etc.).  Let me know!

I agree with Ricardo that this should be merged into Guix proper
eventually.  This is still in flux and we’d need to check what Kjetil
and Thomas at AMD think, in particular wrt. versions, so no ETA so far.

> For some packages, such as composable-kernel, the build time and
> memory requirement is already very high when building only for one GPU
> architecture, so maybe it would be best to make a separate package for
> each architecture?

Is PyTorch able to build code for several GPU architectures and pick the
right one at run time?  If it does, that would seem like the better
option for me, unless that is indeed so computationally expensive that
it’s not affordable.

> I'm not sure they can be combined however, as the GPU code is included
> in the shared libraries. Thus all dependent packages like
> python-pytorch-rocm would need to be built for each architecture as
> well, which is a large duplication for the non-GPU parts.

Yeah, but maybe that’s OK if we keep the number of supported GPU
architectures to a minimum?

> There were a few other issues as well, some of them should probably be
> addressed upstream:
> - Many tests assume a GPU to be present, so they need to be disabled.

Yes.  I/we’d like to eventually support that.  (There’d need to be some
annotation in derivations or packages specifying what hardware is
required, and ‘cuirass remote-worker’, ‘guix offload’, etc. would need
to honor that.)

> - For several packages (e.g. rocfft), I had to disable the
>   validate-runpath? phase, as there was an error when reading ELF
>   files. It is however possible that I also disabled it for packages
>   where it was not necessary, but it was the case for rocblas at
>   least. Here, kernels generated are contained in ELF files, which are
>   detected by elf-file? in guix/build/utils.scm, but rejected by
>   has-elf-header? in guix/elf.scm, which leads to an error.

Weird.  We’d need to look more closely into the errors you got.

[...]

> - There were a few errors due to using the GCC 11 system headers with
>   rocm-toolchain (which is based on Clang+LLVM). For roctracer,
>   replacing std::experimental::filesystem by std::filesystem suffices,
>   but for rocthrust, the placement new operator is not found. I
>   applied the patch from Gentoo [5], where it is replaced by a simple
>   assignment. It looks like UB to me though, even if it happens to
>   work. The question is whether this is a bug in libstdc++, clang or
>   amdclang++...
> - rocMLIR also contains a fork of the LLVM source tree and it is not
>   clear at a first glance how exactly it differs from the main ROCm
>   fork of LLVM or upstream LLVM.

Oh, just noticed your patch bring a lot of things beyond PyTorch itself!
I think there’s some overlap with
, we
should synchronize.

Thanks!

Ludo’.

Google Summer of Code Inquiry

[Zachary Liebl]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Hello Guix Developers,

I am interested in taking on one of your Google Summer of Code projects. I have 
been a long time NixOS user, and I need an excuse to finally get involved with 
Guix, and I think this is it.

I am particularly interested in the project "Add support for AppImage in guix 
pack." What should my next steps be in my application for your GSoC project?

PGP Fingerprint: 3FC8 9016 9BDC 77DF D203 A5DA 0D3F 1B1D 08A0 AE32
-BEGIN PGP SIGNATURE-

iHUEARYIAB0WIQSIuDZ1H+XGeTNlKEB8aUFvaMxlKQUCZgSk5QAKCRB8aUFvaMxl
KcybAQDK/7Jhc0WiN7VjFvidAQqDZZH25EZ6x2Mu2RSdRO7uYQD9GYFku31tBzFp
D5h5Aa0ZP9bJcBLLm71VGK+uGsPTggc=
=3xSh
-END PGP SIGNATURE-