Re: Garbage Collector - for your Information
Hi Guixers, thanks for Your advice Ludo and all other Guixers, for Your information (first time of using gc): After using Guix System for 6 month I did a: "guix gc" It collected about 50 GB and deleted that. (I wanted to do that to get to know how it collects without deleting generations) Than I deleted all generations I did a "guix gc" again and it deleted approx 40 GB again. After that I did a guix pull, guix package -u, sudo guix system reconfigure /etc/config.scm and it downloaded approx 2,2 GB of packages. The result of that is, that in future I will do a "guix gc -F20G" in order not to delete too much and later to redownload and rebuild them. Gottfried Am 09.08.22 um 22:18 schrieb Ludovic Courtès: Hi, Felix Lechner via skribis: On Mon, Jul 25, 2022 at 2:52 AM Gottfried wrote: The manual says that it is dangerous to use: "guix gc" because it can delete too much. I agree that 'guix gc' deletes too much, but it's probably not dangerous. You will just see some downloads and builds repeated when reconfiguring later. Indeed, it’s not dangerous, and I don’t think the manual suggests that. To avoid collecting things just to redownload/rebuild them later, I usually ask ‘guix gc’ to free up some amount of space, as in: guix gc -F20G That ensures 20G (more or less) are available on my disk and doesn’t try to collect more than that. Thanks, Ludo’. OpenPGP_0x61FAF349C9FB7F94.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: Garbage Collector
Hi, Felix Lechner via skribis: > On Mon, Jul 25, 2022 at 2:52 AM Gottfried wrote: >> >> The manual says that it is dangerous to use: "guix gc" >> because it can delete too much. > > I agree that 'guix gc' deletes too much, but it's probably not > dangerous. You will just see some downloads and builds repeated when > reconfiguring later. Indeed, it’s not dangerous, and I don’t think the manual suggests that. To avoid collecting things just to redownload/rebuild them later, I usually ask ‘guix gc’ to free up some amount of space, as in: guix gc -F20G That ensures 20G (more or less) are available on my disk and doesn’t try to collect more than that. Thanks, Ludo’.
Re: Garbage Collector
Hi, Guixers,
I think I managed to add the proposal of Felix Lechner and to set up my
brackets.
Behind:
(sane-service-type _ => sane-backends
I remove one bracket (because the 4. brackets closes "services")
and
behind:
(guix-service-type
config => (guix-configuration
(inherit config)
(extra-options (list "--gc-keep-derivations=yes"
"--gc-keep-outputs=yes"))
I add 3 brackets, to have 5 brackets (the 5. bracket closes "services").
This is how I understand it.
Am I right?
kind regards
Gottfried
My confi.scm before changing the brackets.
;; This is an operating system configuration generated
;; by the graphical installer.
(use-modules (gnu))
(use-package-modules cups scanner)
(use-service-modules cups desktop networking ssh xorg)
(operating-system
(locale "de_DE.utf8")
(timezone "Europe/Berlin")
(keyboard-layout (keyboard-layout "de"))
(host-name "Tuxedo")
(users (cons* (user-account
(name "gfp")
(comment "Gfp")
(group "users")
(home-directory "/home/gfp")
(supplementary-groups
'("wheel" "netdev" "audio" "video")))
%base-user-accounts))
(packages
(append
(list (specification->package "nss-certs"))
%base-packages))
(services
(append
(list (service mate-desktop-service-type)
(service enlightenment-desktop-service-type)
(service cups-service-type
(cups-configuration
(web-interface? #t)
(extensions (list cups-filters
hplip
(service openssh-service-type)
(service tor-service-type)
(set-xorg-configuration
(xorg-configuration
(keyboard-layout keyboard-layout
(modify-services %desktop-services
(sane-service-type _ => sane-backends
(guix-service-type
config => (guix-configuration
(inherit config)
(extra-options (list "--gc-keep-derivations=yes"
"--gc-keep-outputs=yes"))
(bootloader
(bootloader-configuration
(bootloader grub-efi-bootloader)
(target "/boot/efi")
(keyboard-layout keyboard-layout)))
(swap-devices
(list (uuid "51d5cd20-4513-4a02-9e35-df4338eccaa0")))
(file-systems
(cons* (file-system
(mount-point "/boot/efi")
(device (uuid "BB77-FE3B" 'fat32))
(type "vfat"))
(file-system
(mount-point "/")
(device
(uuid "4fb0ed7c-61ab-45eb-be0b-ff527b320e6d"
'ext4))
(type "ext4"))
%base-file-systems)))
Gottfried
Am 25.07.22 um 17:20 schrieb Felix Lechner:
Hi,
On Mon, Jul 25, 2022 at 2:52 AM Gottfried wrote:
The manual says that it is dangerous to use: "guix gc"
because it can delete too much.
I agree that 'guix gc' deletes too much, but it's probably not
dangerous. You will just see some downloads and builds repeated when
reconfiguring later.
I believe 'guix gc' deletes items that are not garbage to me (but I am
new to Guix). Based on friendly advice I received, I currently use
these settings:
(guix-service-type
config => (guix-configuration
(inherit config)
(extra-options (list "--gc-keep-derivations=yes" "--gc-keep-outputs=yes"))
...
Kind regards
Felix Lechner
OpenPGP_0x61FAF349C9FB7F94.asc
Description: OpenPGP public key
OpenPGP_signature
Description: OpenPGP digital signature
Re: Garbage Collector
Hi again,
in my config.scm if I add this part
(guix-service-type
>config => (guix-configuration
> (inherit config)
> (extra-options (list "--gc-keep-derivations=yes"
"--gc-keep-outputs=yes"))
at the end of my guix-service-types,
it seems to me there I have to make 4 brackets:
Or am I wrong?
My config.scm:
;; This is an operating system configuration generated
;; by the graphical installer.
(use-modules (gnu))
(use-package-modules cups scanner)
(use-service-modules cups desktop networking ssh xorg)
(operating-system
(locale "de_DE.utf8")
(timezone "Europe/Berlin")
(keyboard-layout (keyboard-layout "de"))
(host-name "Tuxedo")
(users (cons* (user-account
(name "gfp")
(comment "Gfp")
(group "users")
(home-directory "/home/gfp")
(supplementary-groups
'("wheel" "netdev" "audio" "video")))
%base-user-accounts))
(packages
(append
(list (specification->package "nss-certs"))
%base-packages))
(services
(append
(list (service mate-desktop-service-type)
(service enlightenment-desktop-service-type)
(service cups-service-type
(cups-configuration
(web-interface? #t)
(extensions (list cups-filters
hplip
(service openssh-service-type)
(service tor-service-type)
(set-xorg-configuration
(xorg-configuration
(keyboard-layout keyboard-layout
(modify-services %desktop-services
(sane-service-type _ => sane-backends
(guix-service-type
config => (guix-configuration
(inherit config)
(extra-options (list "--gc-keep-derivations=yes"
"--gc-keep-outputs=yes"))
(bootloader
(bootloader-configuration
(bootloader grub-efi-bootloader)
(target "/boot/efi")
(keyboard-layout keyboard-layout)))
(swap-devices
(list (uuid "51d5cd20-4513-4a02-9e35-df4338eccaa0")))
(file-systems
(cons* (file-system
(mount-point "/boot/efi")
(device (uuid "BB77-FE3B" 'fat32))
(type "vfat"))
(file-system
(mount-point "/")
(device
(uuid "4fb0ed7c-61ab-45eb-be0b-ff527b320e6d"
'ext4))
(type "ext4"))
%base-file-systems)))
Gottfried
Am 25.07.22 um 17:20 schrieb Felix Lechner:
Hi,
On Mon, Jul 25, 2022 at 2:52 AM Gottfried wrote:
The manual says that it is dangerous to use: "guix gc"
because it can delete too much.
I agree that 'guix gc' deletes too much, but it's probably not
dangerous. You will just see some downloads and builds repeated when
reconfiguring later.
I believe 'guix gc' deletes items that are not garbage to me (but I am
new to Guix). Based on friendly advice I received, I currently use
these settings:
(guix-service-type
config => (guix-configuration
(inherit config)
(extra-options (list "--gc-keep-derivations=yes" "--gc-keep-outputs=yes"))
...
Kind regards
Felix Lechner
OpenPGP_0x61FAF349C9FB7F94.asc
Description: OpenPGP public key
OpenPGP_signature
Description: OpenPGP digital signature
Re: Garbage Collector
Hi, when I look at your: (guix-service-type config => (guix-configuration (inherit config) (extra-options (list "--gc-keep-derivations=yes" "--gc-keep-outputs=yes")) Isn't there at the end one bracket too little, to close up the whole thing? Because when I add that to my config.scm file the first bracket before guix-service-type appears violett in Emacs, but there is no violet bracket at the end which closes it. Gottfried Am 25.07.22 um 17:20 schrieb Felix Lechner: Hi, On Mon, Jul 25, 2022 at 2:52 AM Gottfried wrote: The manual says that it is dangerous to use: "guix gc" because it can delete too much. I agree that 'guix gc' deletes too much, but it's probably not dangerous. You will just see some downloads and builds repeated when reconfiguring later. I believe 'guix gc' deletes items that are not garbage to me (but I am new to Guix). Based on friendly advice I received, I currently use these settings: > (guix-service-type config => (guix-configuration (inherit config) (extra-options (list "--gc-keep-derivations=yes" "--gc-keep-outputs=yes")) ... Kind regards Felix Lechner OpenPGP_0x61FAF349C9FB7F94.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: Garbage Collector
Hi, On Mon, Jul 25, 2022 at 2:52 AM Gottfried wrote: > > The manual says that it is dangerous to use: "guix gc" > because it can delete too much. I agree that 'guix gc' deletes too much, but it's probably not dangerous. You will just see some downloads and builds repeated when reconfiguring later. I believe 'guix gc' deletes items that are not garbage to me (but I am new to Guix). Based on friendly advice I received, I currently use these settings: (guix-service-type config => (guix-configuration (inherit config) (extra-options (list "--gc-keep-derivations=yes" "--gc-keep-outputs=yes")) ... Kind regards Felix Lechner
Garbage Collector
Hi Guixers, I would like to use guix gc. The manual says that it is dangerous to use: "guix gc" because it can delete too much. So would it be a good idea to use: "guix gc -d 2m" It would delete older than 2 month generations and also collect garbage. Than the danger to delete something that the system needs is minimized. Or is there still a better idea around? Thanks for your answers Gottfried OpenPGP_0x61FAF349C9FB7F94.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: Avoiding the garbage collector for the Guix Data Service
Hi, Christopher Baines skribis: > I think so, it at least seems to greatly reduce the chance the > derivations get removed. I made the Guix Data Service change here [1], > and I haven't seen the issue since. > > 1: > https://git.savannah.gnu.org/cgit/guix/data-service.git/commit/?id=f59354ed23a9c50e43b7aaaeaddb5365feda4e29 Good! It looks like the right thing to do, to me. Ludo’.
Re: Avoiding the garbage collector for the Guix Data Service
Ludovic Courtès writes: > Hi Christopher, > > Christopher Baines skribis: > >> The Guix Data Service has Guix compute lots of derivations, and then it >> loads the data in to a database. I'm having some issues with the garbage >> collector though, as sometimes it removes data from the disk before I've >> had a chance to read it (at least I think this is what's happening, I >> think this job failed for this reason [1]). >> >> 1: http://data.guix.gnu.org/job/10638 >> >> I'm not quite sure what the best strategy is to combat this? I had a >> look, and I couldn't see how to acquire a lock to prevent the garbage >> collector from running. I thought about registering thousands of gc >> roots, but that seems to just move the race a bit, as there's still a >> period between generating the derivation and registering the root where >> it can be deleted. >> >> Any ideas? > > At the Guix Days we discussed using ‘add-temp-root’ and similar to > ensure that .drv files are at least GC-protected for the duration of the > session (connection to the daemon). > > Does that address your problem? I think so, it at least seems to greatly reduce the chance the derivations get removed. I made the Guix Data Service change here [1], and I haven't seen the issue since. 1: https://git.savannah.gnu.org/cgit/guix/data-service.git/commit/?id=f59354ed23a9c50e43b7aaaeaddb5365feda4e29 Thanks, Chris signature.asc Description: PGP signature
Re: Avoiding the garbage collector for the Guix Data Service
Hi Christopher, Christopher Baines skribis: > The Guix Data Service has Guix compute lots of derivations, and then it > loads the data in to a database. I'm having some issues with the garbage > collector though, as sometimes it removes data from the disk before I've > had a chance to read it (at least I think this is what's happening, I > think this job failed for this reason [1]). > > 1: http://data.guix.gnu.org/job/10638 > > I'm not quite sure what the best strategy is to combat this? I had a > look, and I couldn't see how to acquire a lock to prevent the garbage > collector from running. I thought about registering thousands of gc > roots, but that seems to just move the race a bit, as there's still a > period between generating the derivation and registering the root where > it can be deleted. > > Any ideas? At the Guix Days we discussed using ‘add-temp-root’ and similar to ensure that .drv files are at least GC-protected for the duration of the session (connection to the daemon). Does that address your problem? Thanks, Ludo’.
Avoiding the garbage collector for the Guix Data Service
Hey, The Guix Data Service has Guix compute lots of derivations, and then it loads the data in to a database. I'm having some issues with the garbage collector though, as sometimes it removes data from the disk before I've had a chance to read it (at least I think this is what's happening, I think this job failed for this reason [1]). 1: http://data.guix.gnu.org/job/10638 I'm not quite sure what the best strategy is to combat this? I had a look, and I couldn't see how to acquire a lock to prevent the garbage collector from running. I thought about registering thousands of gc roots, but that seems to just move the race a bit, as there's still a period between generating the derivation and registering the root where it can be deleted. Any ideas? Thanks, Chris signature.asc Description: PGP signature
Re: What is Guix garbage collector root aka gc root
Hello Leo and Ricardo, Leo Famulari <l...@famulari.name> writes: … > You can see the GC root symlinks in /var/guix/gcroots. You can make a > new GC root manually by adding a symlink to the store item you want to > preserve. For example: > > $ ln --symbolic $(guix build hello) /var/guix/gcroots/my-hello Just for the record and other readers. If I create “GC root” with “guix build --root=$HOME/PACKAGE PACKAGE”, I will have two symbolic links --8<---cut here---start->8--- $HOME/PACKAGE -> /gnu/store/…-PACKAGE /var/guix/gcroots/auto/HASH -> $HOME/PACKAGE --8<---cut here---end--->8--- To remove this package in next garbage collector invocation just remove “$HOME/PACKAGE” symbolic link. First invocation of GC like “guix gc --list-live” will remove “/var/guix/gcroots/auto/HASH” symbolic link. Thank you Leo and Ricardo! -- Best regards, Oleg.
Re: What is Guix garbage collector root aka gc root
Hi Oleg, > 2. I need to build a package, but don't install it. Just preserve it > for offloading. Is “guix build --root=$HOME/.guix-profile PACKAGE” (run > as non root user) what I want? Or if not, what FILE in “--root=FILE” do > I need to pass? You can pass any file to “--root”. It will *create* a GC root for you. The manual says this: --8<---cut here---start->8--- ‘--root=FILE’ ‘-r FILE’ Make FILE a symlink to the profile for this environment, and register it as a garbage collector root. This is useful if you want to protect your environment from garbage collection, to make it “persistent”. When this option is omitted, the environment is protected from garbage collection only for the duration of the ‘guix environment’ session. This means that next time you recreate the same environment, you could have to rebuild or re-download packages. --8<---cut here---end--->8--- Since it will create a new symlink, do *not* give it the path to an existing profile. -- Ricardo GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC https://elephly.net
Re: What is Guix garbage collector root aka gc root
On Sun, Aug 13, 2017 at 04:19:54PM +0300, Oleg Pykhalov wrote: > I'm trying to understand what is “garbage collector root” aka “gc root”. Garbage-collector roots (GC roots) are items in /gnu/store that are registered to protect them from the garbage collector: https://www.gnu.org/software/guix/manual/html_node/Invoking-guix-gc.html You can see the GC root symlinks in /var/guix/gcroots. You can make a new GC root manually by adding a symlink to the store item you want to preserve. For example: $ ln --symbolic $(guix build hello) /var/guix/gcroots/my-hello > 1. Is “$HOME/.guix-profile” a “gc root”? Yes, this is mentioned in the manual, section Invoking guix package: https://www.gnu.org/software/guix/manual/html_node/Invoking-guix-package.html > 2. I need to build a package, but don't install it. Just preserve it > for offloading. Is “guix build --root=$HOME/.guix-profile PACKAGE” (run > as non root user) what I want? Or if not, what FILE in “--root=FILE” do > I need to pass? I've never used this feature, so I'm not sure if the value of FILE can be anywhere on the filesystem, or if it has to be in /var/guix/gcroots. Hopefully someone else can clarify this. Or you can read the code or experiment :) signature.asc Description: PGP signature
What is Guix garbage collector root aka gc root
Hello Guix, I'm trying to understand what is “garbage collector root” aka “gc root”. 1. Is “$HOME/.guix-profile” a “gc root”? 2. I need to build a package, but don't install it. Just preserve it for offloading. Is “guix build --root=$HOME/.guix-profile PACKAGE” (run as non root user) what I want? Or if not, what FILE in “--root=FILE” do I need to pass? Thanks. -- Best regards, Oleg.
