Re: Questions about guix handling emacs packages
Hi, zimoun writes: > Re-source the profile seems enough. :-) True, although I usually launch my programs from rofi and I haven't figured out how to re-source the environment for that. > Feel free to give a try and report on help-guix or guix-devel or #guix > your progress or you are stuck. Thanks, I just sent a patch for adding emacs-buffer-move. Figured I start with one and see how that goes. Then follow with the rest. Really digging the git send-email workflow. > I am using emacs@26.3, for comparison. > And Guix 2c8305d14 which is one month older. > > I am using emacs-lua-mode@20191204-1.1f596a9 and I do not have such > issue. But I remember something similar and it was coming from > init.el file; at the time my Emacs packages was loaded with > 'use-package' and now they are loaded with 'with-eval-after-load'. > Well, I do not remember exactly, sorry. Yeah, my errors are probably due to I'm using emacs-next (27.0.91) and `use-package`. And I still need the ability for emacs to download all my packages as I don't have access to guix on my work machine. Although I'm toying with the idea of using `guix pack' to deploy emacs on my work machine. >> - emacs-org-super-agenda: > > Thank you for the report. As you can see with the Data Service, it > seems that a regression has been introduced. > > https://data.guix.gnu.org/repository/1/branch/master/package/emacs-ts/output-history > https://data.guix.gnu.org/repository/1/branch/master/package/emacs-ts/derivation-history > > The log corresponding is: > > http://ci.guix.gnu.org/log/k45l3rcrr81qwkh6w9l0b1gsn6xbm2jd-emacs-ts-0.1-2.395649a > > and as you have noticed, the test 'ts-update' fails. > > > Hum? The regression is probably introduced by commit 4ef89d884. Oh, did not know about the data service. Thanks for pointing that out. I did a `guix import elpa -a melpa-stable -r ts` and that built fine. So maybe just update emacs-ts from 0.1 to 0.2 will fix the issue. >> - emacs-magit: > I am using it and I not see what you are seeing. But I am not using > an older Guix than b2d35dd9a. Probably an issue that I'm using emacs 27 instead of emacs 26. >> - emacs-ledger-mode: > > I did not try. No worries. Anyways thanks for all the help! -- s/Fred[re]+i[ck]+/Fredrik/g
Re: Guix Docker image inflation
Hi Chris, On Sat, 30 May 2020 at 01:31, Chris Marusich wrote: > Could it be that you are accumulating layers without bound? > > https://developers.redhat.com/blog/2016/03/09/more-about-docker-images-size/ > > Since Docker images are built up of immutable layers, if you build your > image from an existing base image, I'm not sure that it's possible to > produce a new image that is smaller than the base image. Basically, > even if you run "guix gc" to remove dead store items, they will still > exist on a prior layer, so the size of the new image won't decrease. > And since you're installing new things, the size will actually increase. > If you repeat this process by using the new image as an input for yet > another build, I think you will accumulate layers and storage space > without bound. Thank you for the explanation. The issue is these layers. When I wrote [1], it was not clear for me because I am not enough familiar with Docker, but with your explanations, it is clear now. :-) [1] http://issues.guix.gnu.org/41607#1 > FYI, Guix itself can build Docker images from scratch - no base image > required! It can even build a Docker image of a full-blown Guix System > from scratch. Sorry if you already knew that - I just wanted to point > it out in case you didn't! I think the idea is to use GitlabCI to build the Docker images containing Guix materials. And AFAIK, GitlabCI does not provide Guix related tools, isn't it? I mean there is no gitlab-runner able to run guix-daemon. If I remember well, we discussed about this topic at FOSDEM, it should be awesome. :-) Cheers, simon
Re: Guix Docker image inflation
Stephen Scheck writes: > Hello, > > As an exercise, I set up daily Guix System Docker image builds using GitLab > and Docker Hub, here: > https://hub.docker.com/repository/registry-1.docker.io/singularsyntax/guix/tags?page=1 > > The build process works as follows: if an existing `latest` image does not > exist for a given branch (master, 1.1.0, etc.), then bootstrap an image by > running `guix system docker-image` inside an Alpine Linux Docker container > with a fresh Guix installation. Using this image as a seed, `guix pull` is > run for the desired branch, and the resulting image is committed to the > Docker repository. If a "latest" image does exist, it is used instead as > the base from which to run `guix pull`. Daily images are thus built > incrementally from the previous day's build. For anybody curious about the > process, the build script can be browsed here: > https://gitlab.com/singularsyntax-docker-hub/guix/-/blob/master/.gitlab-ci.yml > > It works pretty well, except that I'm observing substantial image size > inflation day-over-day, starting at ~197 MB from the seed image, now up to > 1.71 GB eleven days later despite running `guix gc --delete-generations`, > `guix gc --collect-garbage`, and `guix gc --optimize` after pulling prior > to committing each new image. > > I'm wondering if there is some other Guix GC operation or option I'm > missing, or any other suggestions which could stop this unsustainable image > bloat from occurring. I really do doubt that the Guix System itself is > growing this quickly. Could it be that you are accumulating layers without bound? https://developers.redhat.com/blog/2016/03/09/more-about-docker-images-size/ Since Docker images are built up of immutable layers, if you build your image from an existing base image, I'm not sure that it's possible to produce a new image that is smaller than the base image. Basically, even if you run "guix gc" to remove dead store items, they will still exist on a prior layer, so the size of the new image won't decrease. And since you're installing new things, the size will actually increase. If you repeat this process by using the new image as an input for yet another build, I think you will accumulate layers and storage space without bound. If this is what's happening, you might consider always building starting from the same base image every time. You could then update the base image (e.g., by changing the FROM line of a Dockerfile, if that's what you're using) periodically as new versions of it are released. This would probably allow you to avoid accumulating layers without bound. FYI, Guix itself can build Docker images from scratch - no base image required! It can even build a Docker image of a full-blown Guix System from scratch. Sorry if you already knew that - I just wanted to point it out in case you didn't! See: https://guix.gnu.org/manual/en/html_node/Invoking-guix-pack.html https://guix.gnu.org/manual/en/html_node/Invoking-guix-system.html Hope that helps, -- Chris signature.asc Description: PGP signature
Re: Guix Docker image inflation
On Fri, May 29, 2020 at 5:54 PM zimoun wrote: > Do you have '/var/' in your Docker image? Because it looks like the same > than: > Yes: root@guix ~# ls -la /var/guix total 44 drwxr-xr-x 1 root root 4096 May 16 19:36 ./ drwxr-xr-x 1 root root 4096 May 29 22:02 ../ drwxr-xr-x 1 root root 4096 May 29 22:02 daemon-socket/ drwxr-xr-x 1 root root 4096 May 27 00:34 db/ -rw--- 1 root root0 May 16 19:35 gc.lock drwxr-xr-x 1 root root 4096 May 16 19:57 gcroots/ drwxr-xr-x 1 root root 4096 Jan 1 1970 profiles/ drwxr-xr-x 1 root root 4096 May 16 19:35 substitute/ drwxr-xr-x 1 root root 4096 May 27 00:34 temproots/ drwxr-xr-x 1 root root 4096 May 16 19:36 userpool/ If you'd like, you can fetch the exact same image and look around yourself: docker pull singularsyntax/guix:master-a5374cd # same as singularsyntax/guix:latest CONTAINER=`docker run --detach --tty --privileged singularsyntax/guix:master-a5374cd` docker exec --interactive --tty $CONTAINER /run/current-system/profile/bin/bash --login
Re: Guix Docker image inflation
On Fri, 29 May 2020 at 23:04, Stephen Scheck wrote: > root@guix /# guix system list-generations > guix system: error: open-file: No such file or directory: > "/var/guix/profiles/system-1-link/parameters" Do you have '/var/' in your Docker image? Because it looks like the same than: > root@localhost /gnu/store# guix package --list-profiles > /root/.config/guix/current > root@localhost /gnu/store# guix package -d > guix package: error: profile '/var/guix/profiles/per-user/root/guix-profile' > does not exist In addition, you have that: > root@localhost /gnu/store# guix package --list-profiles > /root/.config/guix/current and it is really weird because you are doing: guix package --install --fallback jq /root/.config/guix/current/bin/guix describe --format=json | /root/.guix-profile/bin/jq therefore, somehow, the profile '/root/.guix-profile' should appears with '--list-profiles' too. I do not know if it is a bug -- as Leo suggests -- or if something is not configured as expected. Well, I asked you about the initial Docker images because it should come from this one. The fact that "guix gc --list-dead" outputs a lot of items and the fact that 'you cannot garbage collect with "guix gc" leads me to think that something is wrong with '/var/guix/'. I do not know... Well, does "guix gc --list-dead | grep guix-cli-modules.drv | wc -l" return the same number than you have ran "guix pull"? All the best, simon
Re: Guix Docker image inflation
On Fri, May 29, 2020 at 2:44 PM zimoun wrote: > On Fri, 29 May 2020 at 20:37, Leo Famulari wrote: > > > > On Fri, May 29, 2020 at 08:21:08PM +0200, Marius Bakke wrote: > > > Leo Famulari writes: > > > > --branch and --commit would be passed to `guix pull`, and then you'd > run > > > > `guix system docker-image` based on that. > > > > > > There is also 'guix time-machine --commit=abc123 -- system > docker-image'. > > > > Right, that's probably more efficient than creating lots of `guix pull` > > generations. > > Yes, but it is hard to apriori know the forward commit. > Yes, and also, does a Docker image created by `guix pull` followed by `guix system docker-image [...]` in fact really inherit the Guix snapshot from the system that creates it? Here's what I get on a freshly minted image made that way: root@guix /# guix pull --list-generations guix pull: error: profile '/var/guix/profiles/per-user/root/current-guix' does not exist root@guix /# guix describe guix describe: error: failed to determine origin hint: Perhaps this `guix' command was not obtained with `guix pull'? Its version string is 1.1.0-4.bdc801e. root@guix /# guix package --list-generations guix package: error: profile '/var/guix/profiles/per-user/root/guix-profile' does not exist But here's `guix describe` output from the parent system: root@localhost /# guix describe Generation 13 May 29 2020 19:28:11 (current) guix 41a2d6a repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 41a2d6a8b9294a6eb8e97aaefd569e755f5f461e Until a fresh `guix pull` is run on the new image, it isn't functional and there's no apparent way to confirm its actual commit hash, so I don't really see what advantage it offers over the incremental method I'm using (and it's unfeasibly slow, about 10-15 minutes for an incremental pull compared to over an hour to finish `guix system docker-image`).
Re: Guix Docker image inflation
On Fri, May 29, 2020 at 4:02 PM zimoun wrote: > Well, could you try > >guix system delete-generations >guix gc > root@guix /# guix system list-generations guix system: error: open-file: No such file or directory: "/var/guix/profiles/system-1-link/parameters" root@guix /# guix system delete-generations Backtrace: 1 (primitive-load "/root/.config/guix/current/bin/guix") In guix/ui.scm: 1936:12 0 (run-guix-command _ . _) guix/ui.scm:1936:12: In procedure run-guix-command: In procedure struct-vtable: Wrong type argument in position 1 (expecting struct): #f
Re: Guix Docker image inflation
On Fri, 29 May 2020 at 20:47, Stephen Scheck wrote: > Not the point, no, but how else do I obtain a seed Guix Docker image, which I > can use to birth clean, pristine > "baby" images of Guix's own making? It would be really nice if the Guix > project itself provided such an image! Help welcome! :-) Well, this > root@localhost /gnu/store# guix gc > finding garbage collector roots... > deleting garbage... > [0 MiB] deleting > '/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' > [0 MiB] deleting '/gnu/store/slwkzcmg6r1lr9a16x3krd2ax384p8wr-guix-system' > [0 MiB] deleting '/gnu/store/dzifisbdk1gwy2fw2hwzgvdnjak22awl-guix-extra' > deleting `/gnu/store/trash' > deleting unused links... > note: currently hard linking saves 1181.82 MiB > guix gc: freed 0.636 MiBs and this > root@localhost /gnu/store# guix gc --list-dead > finding garbage collector roots... > determining live/dead paths... > /gnu/store/0bm8h4ns6bymc7q24vhfr0dnb7qab729-guix-cli > /gnu/store/0hjjj9dppc5xvq3bfjwbsygrfyqn0rlv-guix-cli > /gnu/store/0m0xx2958fgyz8kk093afik5cn4rhrc1-guix-cli-modules > /gnu/store/0pi2jhn3a778gc3fm1l31sh07fik4zwa-guix-system-tests-modules > /gnu/store/0vwg9aqzs5xrk10vcs4dl105s3f42ilf-guix-b1affd477-modules > # Lots more listed... is weird. Something wrong happens here. Well, could you try guix system delete-generations guix gc ?
Re: Guix Docker image inflation
On Fri, May 29, 2020 at 2:08 PM zimoun wrote: > How the initial Docker image > singularsyntax/guix-bootstrap:1.1.0-alpine-3.11 is built? > To understand, you use the Docker image > singularsyntax/guix-bootstrap:1.1.0-alpine-3.11 to build another > Docker image namely guix-docker-image.tar using Guix, right? > Well, that is not the point neither the issue. :-) > You can look at the Dockerfile here: https://gitlab.com/singularsyntax-docker-hub/guix-bootstrap It's pretty close to exactly the manual instructions for installing Guix on a "foreign" distro on top of Alpine Linux. Not the point, no, but how else do I obtain a seed Guix Docker image, which I can use to birth clean, pristine "baby" images of Guix's own making? It would be really nice if the Guix project itself provided such an image! > could you try that > > --8<---cut here---start->8--- > GUIX_PATH=/root/.config/guix/current/bin > $GUIX_PATH/guix pull --branch=$CI_COMMIT_REF_NAME--fallback > /root/.config/guix/current/bin/guix pull -d > /root/.config/guix/current/bin/guix package -d > /root/.config/guix/current/bin/guix gc > docker commit > /root/.config/guix/current/bin/guix package --install --fallback jq > --8<---cut here---end--->8--- > > Last, you could try to see what "guix package --list-profiles" says > and then "guix gc --list-dead". root@localhost /gnu/store# guix pull -d root@localhost /gnu/store# guix package --list-profiles /root/.config/guix/current root@localhost /gnu/store# guix package -d guix package: error: profile '/var/guix/profiles/per-user/root/guix-profile' does not exist root@localhost /gnu/store# guix package --list-profiles /root/.config/guix/current root@localhost /gnu/store# du -hs . 4.3G . root@localhost /gnu/store# guix gc finding garbage collector roots... deleting garbage... [0 MiB] deleting '/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' [0 MiB] deleting '/gnu/store/slwkzcmg6r1lr9a16x3krd2ax384p8wr-guix-system' [0 MiB] deleting '/gnu/store/dzifisbdk1gwy2fw2hwzgvdnjak22awl-guix-extra' deleting `/gnu/store/trash' deleting unused links... note: currently hard linking saves 1181.82 MiB guix gc: freed 0.636 MiBs root@localhost /gnu/store# du -hs . 4.3G . root@localhost /gnu/store# guix gc --list-dead finding garbage collector roots... determining live/dead paths... /gnu/store/0bm8h4ns6bymc7q24vhfr0dnb7qab729-guix-cli /gnu/store/0hjjj9dppc5xvq3bfjwbsygrfyqn0rlv-guix-cli /gnu/store/0m0xx2958fgyz8kk093afik5cn4rhrc1-guix-cli-modules /gnu/store/0pi2jhn3a778gc3fm1l31sh07fik4zwa-guix-system-tests-modules /gnu/store/0vwg9aqzs5xrk10vcs4dl105s3f42ilf-guix-b1affd477-modules # Lots more listed...
Re: Guix Docker image inflation
Leo Famulari writes: >> How else would you suggest that it be done? It would be nice if `guix >> system docker-image` >> took `--branch` and `--commit` options to build a container from a >> well-defined Guix check-in >> state, but that doesn't seem to be the case. And in any case - too slow. >> The point here is to >> leverage daily incremental pulls to keep data transfer and build times down. > > --branch and --commit would be passed to `guix pull`, and then you'd run > `guix system docker-image` based on that. There is also 'guix time-machine --commit=abc123 -- system docker-image'. signature.asc Description: PGP signature
Re: Guix Docker image inflation
On Fri, 29 May 2020 at 20:37, Leo Famulari wrote: > > On Fri, May 29, 2020 at 08:21:08PM +0200, Marius Bakke wrote: > > Leo Famulari writes: > > > --branch and --commit would be passed to `guix pull`, and then you'd run > > > `guix system docker-image` based on that. > > > > There is also 'guix time-machine --commit=abc123 -- system docker-image'. > > Right, that's probably more efficient than creating lots of `guix pull` > generations. Yes, but it is hard to apriori know the forward commit.
Re: Guix Docker image inflation
On Fri, May 29, 2020 at 08:21:08PM +0200, Marius Bakke wrote: > Leo Famulari writes: > > --branch and --commit would be passed to `guix pull`, and then you'd run > > `guix system docker-image` based on that. > > There is also 'guix time-machine --commit=abc123 -- system docker-image'. Right, that's probably more efficient than creating lots of `guix pull` generations.
Re: Guix Docker image inflation
On Fri, May 29, 2020 at 2:02 PM Leo Famulari wrote: > Okay. For debugging, can you try garbage collecting those modules > directories? And if the garbage collector refuses, you can investigate > why with the 3 R's of Guix garbage collection, --referrers, > --references, and --requisites. > # Hmm... root@localhost /gnu/store# guix gc --references /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules guix gc: error: path `/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' is not valid # Hmm... root@localhost /gnu/store# guix gc --requisites /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules guix gc: error: path `/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' is not valid # Hmm... this one is different - no output root@localhost /gnu/store# guix gc --referrers /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules # Now try to delete it... root@localhost /gnu/store# guix gc --delete /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules finding garbage collector roots... [0 MiB] deleting '/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' deleting `/gnu/store/trash' deleting unused links... note: currently hard linking saves 1181.36 MiB # Still there... root@localhost /gnu/store# du -hs /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules 210M /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules
Re: Guix Docker image inflation
Dear, On Wed, 27 May 2020 at 21:42, Stephen Scheck wrote: > https://gitlab.com/singularsyntax-docker-hub/guix/-/blob/master/.gitlab-ci.yml How the initial Docker image singularsyntax/guix-bootstrap:1.1.0-alpine-3.11 is built? To understand, you use the Docker image singularsyntax/guix-bootstrap:1.1.0-alpine-3.11 to build another Docker image namely guix-docker-image.tar using Guix, right? Well, that is not the point neither the issue. :-) Well, instead of that --8<---cut here---start->8--- GUIX_PATH=/root/.config/guix/current/bin $GUIX_PATH/guix pull --branch=$CI_COMMIT_REF_NAME--fallback /root/.config/guix/current/bin/guix gc --delete-generations /root/.config/guix/current/bin/guix gc --collect-garbage /root/.config/guix/current/bin/guix gc --optimize docker commit /root/.config/guix/current/bin/guix package --install --fallback jq --8<---cut here---end--->8--- could you try that --8<---cut here---start->8--- GUIX_PATH=/root/.config/guix/current/bin $GUIX_PATH/guix pull --branch=$CI_COMMIT_REF_NAME--fallback /root/.config/guix/current/bin/guix pull -d /root/.config/guix/current/bin/guix package -d /root/.config/guix/current/bin/guix gc docker commit /root/.config/guix/current/bin/guix package --install --fallback jq --8<---cut here---end--->8--- ? Last, you could try to see what "guix package --list-profiles" says and then "guix gc --list-dead". Hope that helps, simon
Re: Guix Docker image inflation
On Fri, May 29, 2020 at 01:56:28PM -0400, Stephen Scheck wrote: > > > "guix-system$|guix-packages-base$|guix-[0-9a-f]*-modules$" > > [...] > > > 191M > > /gnu/store/l3amdz5xyhflg5wdzlxr2685dq5glic2-guix-527ab3125-modules > > > 201M > > /gnu/store/5mhn1ynxvy7jihsknsnv3yspkkvc0r5s-guix-2e59ae238-modules > > > > If I understand correctly, you should not need both of these directories > > in a Guix VM image. The latter hashes are truncated guix.git commit > > hashes and a VM image would only be based on a single one. > > > > Exactly, I agree (to the extent that I understand Guix). > > I recommend looking into why all these directories are being copied into > > your images. > > > > Whatever is in /gnu/store (as managed by Guix) goes into the image, nothing > more and nothing less. Okay. For debugging, can you try garbage collecting those modules directories? And if the garbage collector refuses, you can investigate why with the 3 R's of Guix garbage collection, --referrers, --references, and --requisites. > How else would you suggest that it be done? It would be nice if `guix > system docker-image` > took `--branch` and `--commit` options to build a container from a > well-defined Guix check-in > state, but that doesn't seem to be the case. And in any case - too slow. > The point here is to > leverage daily incremental pulls to keep data transfer and build times down. --branch and --commit would be passed to `guix pull`, and then you'd run `guix system docker-image` based on that.
Re: Guix Docker image inflation
On Fri, May 29, 2020 at 1:08 PM Leo Famulari wrote: > I'm still not quite sure what you are doing (or what Docker does) so > please bear with me. > > > root@localhost /# du -h --max-depth=1 /gnu/store | egrep > > "guix-system$|guix-packages-base$|guix-[0-9a-f]*-modules$" > [...] > > 191M > /gnu/store/l3amdz5xyhflg5wdzlxr2685dq5glic2-guix-527ab3125-modules > > 201M > /gnu/store/5mhn1ynxvy7jihsknsnv3yspkkvc0r5s-guix-2e59ae238-modules > > If I understand correctly, you should not need both of these directories > in a Guix VM image. The latter hashes are truncated guix.git commit > hashes and a VM image would only be based on a single one. > Exactly, I agree (to the extent that I understand Guix). I recommend looking into why all these directories are being copied into > your images. > Whatever is in /gnu/store (as managed by Guix) goes into the image, nothing more and nothing less. > > I figure you'd want to create each image with *only* the things > corresponding to the Git commit it's based on, but it sounds like they > are being created by copying the entire host image, which doesn't seem > right. > > If the Docker images are being created by simply snapshotting the file > system of a non-ephemeral Guix system, that's probably not the right way > to do it. Is that what's going on? > Yes, as I said, the image is created from a file system snapshot, after Guix is brought up to date via `guix pull` and those various Guix garbage collection operations are run. However, it's not quite "non-ephmeral" as each Guix operation is run as an atomic command inside the Docker container, with nothing else running (except for guix-daemon, which has to always be running for Guix to operate to the best of my understanding, and a couple other Guix System daemons which anyway would be equivalent to the situation to any Guix installation running outside of a Docker container). How else would you suggest that it be done? It would be nice if `guix system docker-image` took `--branch` and `--commit` options to build a container from a well-defined Guix check-in state, but that doesn't seem to be the case. And in any case - too slow. The point here is to leverage daily incremental pulls to keep data transfer and build times down.
Re: Guix Docker image inflation
On Fri, May 29, 2020 at 12:19:46PM -0400, Stephen Scheck wrote: > The previous day's Docker image is used as the base for the new one being > built - the image is pulled from Docker Hub, `guix pull` is run inside it, > and a new > image is "committed" (Docker terminology for creating a new image from a > file system snapshot). I'm still not quite sure what you are doing (or what Docker does) so please bear with me. > root@localhost /# du -h --max-depth=1 /gnu/store | egrep > "guix-system$|guix-packages-base$|guix-[0-9a-f]*-modules$" [...] > 191M /gnu/store/l3amdz5xyhflg5wdzlxr2685dq5glic2-guix-527ab3125-modules > 201M /gnu/store/5mhn1ynxvy7jihsknsnv3yspkkvc0r5s-guix-2e59ae238-modules If I understand correctly, you should not need both of these directories in a Guix VM image. The latter hashes are truncated guix.git commit hashes and a VM image would only be based on a single one. I recommend looking into why all these directories are being copied into your images. I figure you'd want to create each image with *only* the things corresponding to the Git commit it's based on, but it sounds like they are being created by copying the entire host image, which doesn't seem right. If the Docker images are being created by simply snapshotting the file system of a non-ephemeral Guix system, that's probably not the right way to do it. Is that what's going on?
Re: Guix Docker image inflation
On Thu, May 28, 2020 at 3:33 PM Leo Famulari wrote: > I'm not familiar with Docker so I'm not sure exactly what you are doing. > Specifically, I can't tell if you are creating new Docker images from > scratch each day, or if you are continuing to use the same one from day > to day. The previous day's Docker image is used as the base for the new one being built - the image is pulled from Docker Hub, `guix pull` is run inside it, and a new image is "committed" (Docker terminology for creating a new image from a file system snapshot). BTW, I posted an incorrect internal link - the actual Docker images are available here if you'd like to try them out: https://hub.docker.com/r/singularsyntax/guix/tags > I'm also not sure which image is growing each day... The daily Docker images described above. > In general, the parameters --delete-generations and --collect-garbage > are supposed to be passed values like a reference to a profile or an > amount of data to delete, respectively. Are you doing that? `guix gc --delete-generations` without a parameter causes all preceding pull and package generations to be deleted. > Are you removing / invalidating old generations before attempting to > garbage collect them? The store items they refer to cannot be deleted > until the generations themselves are no longer registered. Yes, `guix gc --delete-generations`, `guix gc --collect-garbage`, and `guix gc --optimize` are run in the order given. Note that passing a specific amount parameter to `--collect-garbage` makes no difference. > Usually, these old profiles are responsible for most of the disk usage > in /gnu/store. Indeed. It's clear what's taking up the space, but I don't understand why it does not get garbage collected: root@localhost /# guix pull --list-generations Generation 12 May 28 2020 20:45:30 (current) guix a5374cd repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: a5374cde918cfeae5c16b43b9f2dd2b24bc3564d root@localhost /# guix package --list-generations guix package: error: profile '/var/guix/profiles/per-user/root/guix-profile' does not exist root@localhost /# du -h --max-depth=1 /gnu/store | egrep "guix-system$|guix-packages-base$|guix-[0-9a-f]*-modules$" 44M /gnu/store/slwkzcmg6r1lr9a16x3krd2ax384p8wr-guix-system 44M /gnu/store/zf67wb6c0s97vwmywjq09hy9jq0w5mmi-guix-system 107M /gnu/store/plaay02w581vx9ilyiv93sl1lw54n7h5-guix-packages-base 44M /gnu/store/qhbk7g8z97m37iak1s1yn2my82gv0lj5-guix-system 103M /gnu/store/2qcfl7h10dynjlifyvqwh9iiic52q5x6-guix-packages-base 107M /gnu/store/m0fv2xmfif5pxnfb1bscfvgyfx0x6xdc-guix-packages-base 90M /gnu/store/hz2rn2l0jixg91q4rsdcwc489y71ll29-guix-05e1edf22-modules 41M /gnu/store/w47fgv8p2hvaqdwywymwvm0qlh4gw0ih-guix-system 191M /gnu/store/l3amdz5xyhflg5wdzlxr2685dq5glic2-guix-527ab3125-modules 201M /gnu/store/5mhn1ynxvy7jihsknsnv3yspkkvc0r5s-guix-2e59ae238-modules 44M /gnu/store/dzc16sv8jv831m0jkk5llc2ws1a3mk0z-guix-system 44M /gnu/store/9a2hr5lh15vxqa7bjih8w47wr6hr11nv-guix-system 103M /gnu/store/1lwdys51wi08r5an2rr6sqk9kbgr7qip-guix-packages-base 44M /gnu/store/c3spiv1c0fg83j7d99mjwk0s6fw77wl5-guix-system 44M /gnu/store/vwzk618h1wxy6z9i06xnhnxj4gvhkiss-guix-system 6.7M /gnu/store/a5xsqxr04pwnyni5x2gqjnishzq80cbw-guix-packages-base 14M /gnu/store/mych9fchln22pbhpc5syxyymx4hz496y-guix-8bd0b533b-modules 35M /gnu/store/brbwlbnx56ms50kklyqk9fsf0xkwjjf9-guix-498e2e669-modules 3.2M /gnu/store/dirpwhdr7h4nyphy4ncxqi4f2njv3rsh-guix-packages-base 35M /gnu/store/d3h4b7nvnms8d03ddi9b481dlxpykl7l-guix-5e3d16994-modules 5.8M /gnu/store/n339sr8c63f0nzja6yl8zfwy1jklj19j-guix-packages-base 25M /gnu/store/0vwg9aqzs5xrk10vcs4dl105s3f42ilf-guix-b1affd477-modules 41M /gnu/store/pwr8ab20xa1whxag689lsz82l2na08x0-guix-system 6.5M /gnu/store/6sggbpgg0zkbgxwf3wa2j15dis8z7cr1-guix-packages-base 57M /gnu/store/8z9qc2bvq8azc08p4miq77yf2agk07aq-guix-843e77205-modules 71M /gnu/store/ibgjq1ampj8bldrabbsnwik2sr0gg3as-guix-a43fe7acd-modules 37M /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules 18M /gnu/store/i72b4biraw6bhy1v7ly46kwyaacvfa28-guix-system 178M /gnu/store/47aack48aczpzm635axsy4jf2pvmwrv0-guix-ef1d475b0-modules 15M /gnu/store/77sxajrwigsdnyr4l4jq4pk6v5kwbm59-guix-system
Re: ffmpeg and alsa/pulseaudio
‐‐‐ Original Message ‐‐‐ On Friday, May 29, 2020 3:34 AM, Leo Famulari wrote: > On Tue, Apr 07, 2020 at 11:54:35AM +, Ekaitz Zarraga wrote: > > > Hi, > > I'm trying to make a screencast with ffmpeg and I'm having issues. > > I had a screen recording script on the past, based on alsa, but it's not > > working in guix because it looks like ffmpeg wasn't compiled with alsa. Is > > that true? Is there something I missing to activate alsa on ffmpeg? > > This is the command I was running: > > ffmpeg \ > > -y -f x11grab -draw_mouse 1 -framerate 25 -video_size 1920x1080 -i $DISPLAY > > \ > > -f alsa -ac 2 -i default \ > > -pix_fmt yuv420p -c:v libx264 -preset veryfast \ > > -c:a libmp3lame -q:v 1 -s 1920x1080 -f matroska \ > > output.mkv > > I tried this but it failed with ":0: Invalid argument". That's the > $DISPLAY variable. Do you know what it should say? Hi Leo, Thanks for the answer. I have :1 in DISPLAY and it's not failing because of that. More on the issue: I had to use a different microphone for this just because of this issue. Looks like when I use a USB microphone and use pulseaudio engine it breaks somehow and records audio with wrong timestamps. An analog microphone works well. With alsa I found a working configuration but it doesn't work :( Thanks!
