Re: Building and caching old Guix derivations for a faster time machine
Hi Simon, Simon Tournier writes: > Hi Maxim, > > On Thu, 30 Nov 2023 at 08:28, Maxim Cournoyer > wrote: > >> I'd like to have a single archive type as well in the future, but I'd >> settle on Zstd, not lzip, because it's faster to compress and >> decompress, and its compression ratio is not that different when using >> its highest level (19). > > When running an inferior (past revision), some past Guile code as it was > in this past revision is launched. Hum, I have never checked: the > substitution mechanism depends on present revision code (Guile and > daemon) or on past revision? > > Other said, what are the requirements for the backward compatibility? > Being able to run past Guix from a recent Guix, somehow. We're only impacting the future, not the past, I think. The inferior mechanism still relies on the same daemon, as far as I know, and the currently available gzipped nars would remain available according to their current retention policy (6 months when unused). >>> 1. Keep for as longer as we can all the requirements for running Guix >>> itself, e.g., “guix time-machine”. Keep all the dependencies and all >>> the outputs of derivations. At least, for all the ones the build farms >>> are already building. >>> >>> 2. Keep for 3-5 years all the outputs for specific Guix revision, as >>> v1.0, v1.1, v1.2, v1.3, v1.4. And some few others. >> >> That'd be nice, but not presently doable as we can't fine tune retention >> for a particular 'derivation' and its inputs in the Cuirass >> configuration, unless I've missed it. > > That’s an implementation detail, a bug or a feature request, pick the > one you prefer. ;-) I'd say it's a feature request :-). > We could imagine various paths for these next steps, IMHO. For > instance, we could move these outputs to some specific stores > independent of the current ones (ci.guix and bordeaux.guix). For > instance, we could have “cold” storage with some cooking bakery for > making hot again, instead of keeping all hot. For instance, we could > imagine etc. :-) > > Well, I do not have think much and I just speak loud: Cuirass (and Build > Coordinator) are the builders, and I would not rely on them for some NAR > “archiving“, instead maybe “we” could put some love into the tool > nar-herder. Somehow, extract specific NAR that the project would like > to keep longer than the unpredictable current mechanism. It seems the nar-herder would perhaps be well suited for this, if someone is inclined to implement it, given it keeps each nars in a database, which should make it fast to query for all the 'guix' packages substitutes. Perhaps it even has (or could have) hooks when registering a new nars which could define what is done to it (send to another server). Otherwise good old 'find' could be used to rsync the 'guix' named nars and their .narinfo metadata files to a different location, but that'd probably be less efficient (IO-intensive) on the huge multi-terabytes collection of nars we carry. -- Thanks, Maxim
Re: When is check-system run?
Hi Tomas, Tomas Volf <~@wolfsden.cz> writes: > Hello, > > I would like to inquire regarding system tests. There is check-system make > target, however it seems like at least some of them are broken. Can someone > check whether that is case even on their machine? Do they all pass for you? The system tests are run for new Guix revisions periodically by this job set: https://ci.guix.gnu.org/jobset/tests Otherwise we depend on people touching e.g. services running them manually to ensure they pass. Accumulated test failures that have slipped in (we do not have a CI infrastructure gating merges on test failures yet) should ideally be reported and fixed (thanks for working on it!). There is typically an effort to ensure all (or most) tests pass before a new release is made. > I already have patches fixing 4 of them, and debugging 5th, but I would like > to > ask when exactly are the those tests run and where are the failures > reported to? Thank you! You can report the failures to bug-g...@gnu.org, if they weren't already; these are Guix bugs. > Since the breakage sneaked into the master, I assume they are not run as part > of > the regular patches? Are there any measures the project is planning to take > to > prevent this from happening again? It'd be nice to have our CI infrastructure act more like a CI (currently it's more of a build farm than a CI), but I don't know of anyone working on it (though Christopher may have long term plans toward that goal with QA [0]). [0] qa.guix.gnu.org/ -- Thanks, Maxim
%base-packages and default grub theme depend on rust
So, I stumbled a bit with a fairly recently installed aarch64/arm64
system. The install went fine late December, but then I tried "guix
system reconfigure" a couple days ago, and even though it is a very
simple configuration (based on bare-bones.tmpl with grub-efi)... it
pretty much needed to rebuild about 12 rust versions, as no substitutes
were available on aarch64-linux. *sigh*
I thought I tracked this down to guix-icons depending on librsvg, which
depends on rust... and guix-icons is in %base-packages.
So I dropped use of %base-packages-artwork, which pulls in guix-icons
and used all the various other %base-packages-* stuff explicitly... but
it still wanted to pull in guix-icons/librsvg/rust-* etc ... it just did
so later in the build process... foiled again!
This is because the default grub theme generates a .png from an .svg
... using guile-rsvg, which uses librsvg, which uses rust ...
But this machine just has a serial console, and has no need of a
background image in the grub configuration...
So eventually I figured out how to get a grub theme without a background
image and drop guix-icons from the configuration by avoiding use of
%base-packages-artwork:
(bootloader (bootloader-configuration
+(theme (grub-theme (image #f)))
(bootloader grub-efi-bootloader)
(targets '("/boot/efi"
- (packages (cons screen %base-packages))
+ (packages (append (list screen nss-certs)
+ %base-packages-interactive
+ %base-packages-linux
+ %base-packages-networking
+ %base-packages-utils))
Maybe these is a more elegant way of doing this.
So, the example bare-bones system configuration does seems to have heavy
layers of rust obscuring the bones.
Do we want a really-quite-bare-bones configuration example?
Should guix-icons not actually be in %base-packages?
Can the grub theme be implemented differently without requiring rust?
This seems like possibly a discussion so far, but I would also be happy
to turn this into a bug report if desired!
Thanks everyone!
live well,
vagrant
signature.asc
Description: PGP signature
Re: Module unavailable on build side
this may help: https://github.com/attila-lendvai/guix-crypto/blob/main/src/guix-crypto/service-utils.scm#L56 and you should grep for its use in that repo. w-i-m ensures that the GEXP's that are instantiated in its dynamic extent will "capture" these modules as their dependencies, and i think it also inserts the appropriate use-modules forms at the head of the GEXP's. but take this all with a grain of salt, because what i understand i decoded on my own from the guix codebase, and the names of these abstractions are rather misleading. also, i'm using these in the service code that gets compiled for shepherd to be loaded. the environment surrounding the building of packages may behave differently. HTH, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “There is no difference between living and learning […] it is impossible and misleading and harmful to think of them as being separate.” — John Holt (1923–1985)
Re: When is check-system run?
Hi Tomas, On Fri, Jan 12 2024, Tomas Volf wrote: > it seems like at least some of them are broken. Can someone check > whether that is case even on their machine? Yes, some are broken. > Do they all pass for you? No. > when exactly are the those tests run and where are the failures > reported to? I believe they run before 'guix pull' is finalized, but I cannot point to how they are invoked. So far I have written four of them. (Cachefilesd should pass; the other three are pending and have not been accepted yet.) Sorry, that's all I have. Kind regards Felix
Re: TTY auto-login broken
Christian Miller writes: > Hello, > > this is how I configured my system: > > (modify-services %desktop-services > (delete gdm-service-type) > (mingetty-service-type config => > (mingetty-configuration > (inherit config) > (auto-login "cm") > ;; TODO: Work around to fix "Error in service module" > (login-pause? #t > > Without the (login-pause? #t) I would get "Error in service module" as > an error message in my TTY and can't use the system. > > There is also someone else with that issue[0] > > I asked on the IRC[1] some time ago (somenickname was me in that case) > and nckx had an assumption why this is happening. > > [0]: https://issues.guix.gnu.org/68384 > [1]: https://logs.guix.gnu.org/guix/2023-09-28.log#211350 Also the cookbook describes something similar and it is apparently not working anymore either. I've reported it some time ago. https://issues.guix.gnu.org/56958 Thanks, Joshua -- Joshua Branson Sent from the Hurd
TTY auto-login broken
Hello, this is how I configured my system: (modify-services %desktop-services (delete gdm-service-type) (mingetty-service-type config => (mingetty-configuration (inherit config) (auto-login "cm") ;; TODO: Work around to fix "Error in service module" (login-pause? #t Without the (login-pause? #t) I would get "Error in service module" as an error message in my TTY and can't use the system. There is also someone else with that issue[0] I asked on the IRC[1] some time ago (somenickname was me in that case) and nckx had an assumption why this is happening. [0]: https://issues.guix.gnu.org/68384 [1]: https://logs.guix.gnu.org/guix/2023-09-28.log#211350 -- Christian Miller
Problems with Emacs vterm
Hello, I have two problems with vterm. 1: It does have directory-tracking according to it's manuals. I also looked at the package and the required scripts are present. But this feature does not work for me. 2: I don't know why but TRAMP is not working with vterm, too. It opens a new buffer that is killed instantly. This also breaks the prompt of newly created vterm buffers for me. Those features work for Eshell though. -- Christian Miller
Feedback of the GNU Guix cookbook manual
Hello,
I read the cookbook on revision
ee7c9d254117fa470686210ad2ef5e7f1ba4fefc using Emacs in TTY mode.
It seems kinda messy. Some descriptions are read like a temporarily
note that should have been rewritten for the cookbook manual. There
are even lines that exceed the 80 column limit.
Though, it was still very useful for me as a beginner. It made the
entrance way easier than Emacs since I had actually code I could use
(this is also true for the normal manual).
Here are some things I noticed:
*** 2.1.3.2 Snippets
"the code in only" should probably be "the code is only"
Snippets are quoted (i.e. non-evaluated) Scheme code that are a means
of patching the source. They are a Guix-y alternative to the
traditional ‘.patch’ files. Because of the quote, the code in only
evaluated when passed to the Guix daemon for building. There can be as
many snippets as needed.
*** 2.1.3.3 Inputs
Seems like a copy paste error. It says "using the style above
instead" but there is none since it is missing. In the guix manual
both code snippets are present.
This is the “old style”, where each input in the list is explicitly
given a label (a string). It is still supported but we recommend
using the style above instead. *Note (guix)package Reference::,
for more info.
*** 2.1.3.5 Build system arguments
Use the new style with gexpr.
L216
#:make-flags (list (string-append "prefix=" (assoc-ref %outputs "out"))
"CC=gcc")
L315
(lambda* (#:key inputs outputs #:allow-other-keys)
(let ((bash-directory (assoc-ref inputs "bash"))
(output-directory (assoc-ref outputs "out"))
(doc-directory (assoc-ref outputs "doc")))
;; ...
#true))
*** 3.1 Auto-Login to a Specific TTY
Whitespace ( see
While the Guix manual explains auto-login one user to _all_ TTYs ( *note
(guix)auto-login to TTY::), some might prefer a situation, in which one
*** 3.2 Customizing the kernel
6 is current. Maybe just say current as of the time of writing
The current ‘linux-libre’ package is for the 5.15.x series, and is
declared like this:
Also it does use the old style syntax for code.
*** 3.7.1 StumpWM
Guix system instead of Guix System and uses old style syntax for code.
*** Running Guix on a Kimsufi server
Maybe use local file to be consistent with Linode guide.
(authorized-keys
`(("guix" ,(plain-file "SSH-KEY-NAME.PUB"
"SSH-PUBLIC-KEY-CONTENT"))
Just give the actual commands and don't assume user knows it.
Transfer your operating system declaration ‘os.scm’ file on the
server via the ‘scp’ or ‘sftp’ commands.
Now all that is left is to install Guix with a ‘guix system init’ and
restart.
*** Guix Containers
What?
;; Common command line tools lest the container is too empty.
*** The benefits of manifest
guix environment on the first step and on the third step it is guix
shell. guix environment is superseded by guix shell and should
therefore not be mentioned.
1. We have an environment for hacking on a project for which there
isn’t a Guix package yet. We build the environment using a
manifest, and then run ‘guix environment -m manifest.scm’. So far
so good.
2. Many weeks pass and we have run a couple of ‘guix pull’ in the mean
time. Maybe a dependency from our manifest has been updated; or we
may have run ‘guix gc’ and some packages needed by our manifest
have been garbage-collected.
3. Eventually, we set to work on that project again, so we run ‘guix
shell -m manifest.scm’. But now we have to wait for Guix to build
and install stuff!
*** Guix environment via direnv
guix environment should be replaced with guix shell. Also exceeding 80 columns
eval "$(guix environment --search-paths --root="$gcroot" --pure guix
--ad-hoc ${PACKAGES[@]} ${PACKAGES_MAINTENANCE[@]} "$@")"
*** 7.3 Network Access
Replace white list with allow list?
--
Christian Miller
Feedback of the GNU Guix manual
Hello, I read through the GNU Guix manual on revision ee7c9d254117fa470686210ad2ef5e7f1ba4fefc with Emacs in TTY mode. As a beginner, the manual was helpful and I learned lot's of stuff. Though it felt heartless. No consistency and sometimes the structure is confusing (especially the contributing part, since it tries to get GNU Guix System and GNU Guix under one section and it was kinda confusing for me). Overall it was a better experience than reading the Emacs manual. The Emacs manual may have a better consistent style but it was hard to understand. Felt like it assumed people would know stuff which I did not do. It seems that there is no style guide. Sometimes default values are mentioned and sometimes not. Also some use uppercase and other use lowercase for type of variable. There is also Scheme code which is most of the time in the old style notation. For me as a beginner, this was confusing. I will mention lot's of style issues since at that time I did not know it is in such a bad state. Here are some things I noticed: ** Binary installation L73 Guix system should be Guix System ** Binary Installation L88 and L95 ~root is weird. Should be root or simply ~ for the user profile which would be root as root user. ** 2.4 Setting up the deamon Seems like an issue with info. Have seen this in the Emacs manual as well. There is also sometimes see see (doc) L15 See also see Substitutes. ** 3.7 After System Installation L27 Libera_Chat, why the underscore. It is colored as well for me so I guess this is a special char. Should be anways just Libera Chat ** 3.8 Installing Guix in a Virtual Machine L25 in an vm should be in a vm ** 6.4 L60 packages should be package The outputs of a packages are listed in the third column of the output of ‘guix package --list-available’ (*note Invoking guix package::). ** 6.7 L37 true for Guix System as well? The result of running ‘guix pull’ is a “profile” available under ‘~/.config/guix/current’ containing the latest Guix. Thus, make sure to add it to the beginning of your search path so that you use the latest version, and similarly for the Info manual (*note Documentation::): export PATH="$HOME/.config/guix/current/bin:$PATH" export INFOPATH="$HOME/.config/guix/current/share/info:$INFOPATH" ** 6.8 Maybe just mention the actual movie for people that never have seen or heard of it (1) If you don’t know what a DeLorean is, consider traveling back to the 1980’s. (Back to the Future (1985)) ** 7.1 should be variant-packages (not sure anymore, those notices I made are several weeks old) The output of ‘guix describe’ above shows that we’re now running Generation 19 and that it includes both Guix and packages from the ‘variant-personal-packages’ channel (*note Invoking guix describe::). ** 8.1 See https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.7=6eb3c3d0a52dca337e327ae8868ca1f44a712e02 You may pass any ‘guix shell’ option, but there’s one caveat: the Linux kernel has a limit of 127 bytes on shebang length. ** 8.1 this is done automatically on Guix System. Should be mentioned? ‘guix shell’ defines the ‘GUIX_ENVIRONMENT’ variable in the shell it spawns; its value is the file name of the profile of this environment. This allows users to, say, define a specific prompt for development environments in their ‘.bashrc’ (*note (bash)Bash Startup Files::): if [ -n "$GUIX_ENVIRONMENT" ] then export PS1="\u@\h \w [dev]\$ " fi ... or to browse the profile: $ ls "$GUIX_ENVIRONMENT/bin" ** 8.2 Time to remove? Being deprecated, ‘guix environment’ is slated for eventual removal, but the Guix project is committed to keeping it until May 1st, 2023. Please get in touch with us at if you would like to discuss it. ** 8.3 L47 they they What if the recipient of your pack does not have root privileges on their machine, and thus cannot unpack it in the root file system? In that case, you will want to use the ‘--relocatable’ option (see below). This option produces “relocatable binaries”, meaning they they can be placed anywhere in the file system hierarchy: in the example above, users can unpack your tarball in their home directory and directly run ** 9 missing application word for apis. GNU Guix provides several Scheme programming interfaces (APIs) to define, build, and query packages. The first interface allows users to write high-level package definitions. These definitions refer to familiar packaging concepts, such as the name and version of a package, its build system, and its dependencies. These definitions can then be turned into concrete build actions. ** 9.5 build system texlive-build-system located in located in It also generates font metrics (i.e., ‘.tfm’ files) out of Metafont files whenever possible. Likewise, it can also create TeX formats (i.e., ‘.fmt’ files) listed in the ‘#:create-formats’
Re: How to answer all people in a thread?
Let's see if this works. Does everyone get an email? -- Christian Miller
Re: Module unavailable on build side
Hi, On 2024-01-13 20:12:38 -0800, Felix Lechner via Development of GNU Guix and the GNU System distribution. wrote: > Hi, > > How may I make code from a channel available inside 'modify-phases', > please? > > I tried #:modules, but it says "no code for (x y z)." Thanks! Based on some quick experimentation `source-module-closure' seems to be the way. Package definition importing a (nonguix licenses) module follows: (use-modules (guix build-system gnu) (guix gexp) (guix modules) (guix packages) (nongnu packages linux)) (package (name "test") (version "1") (source #f) (build-system gnu-build-system) (arguments (list #:imported-modules (source-module-closure `((nonguix licenses) ,@%gnu-build-system-modules) #:select? (λ (m) (or (guix-module-name? m) (eq? (car m) 'nonguix #:phases #~(modify-phases %standard-phases (add-before 'unpack 'foo (λ _ (use-modules (nonguix licenses)) (error nonfree)) (home-page #f) (synopsis #f) (description #f) (license #f)) This does "build" resulting in the expected error: ice-9/boot-9.scm:1685:16: In procedure raise-exception: # However it took me a while to find a module that actually works. Due to the web of dependencies, it can get complex quickly and you would need to properly manage the #:select? value. Just putting (const #t) in there sadly does not work. Not sure, maybe there is a better way. If there is, I hope someone will tell me (us). In the mean time, I hope this helps to some degree. Have a nice day, Tomas Volf PS: My understanding it that #:modules is equivalent to (use-modules), that is why I used #:imported-modules to add them into the build environment first. -- There are only two hard things in Computer Science: cache invalidation, naming things and off-by-one errors. signature.asc Description: PGP signature
