Re: ZFS on Guix

[raid5atemyhomework]

Hi Joshua,

> raid5atemyhomework raid5atemyhomew...@protonmail.com writes:
>
> > The patchset currently dying on issues.guix.gnu.org would provide a nice 
> > simple single-step way to enable very basic ZFS support on your Guix 
> > system. Until it gets merged, however, you can still enable very very basic 
> > ZFS support on your Guix system by following the below minimal guide.
>
> If this patch does not get merged into guix, then you could always add
> it to the cookbook. :)

It includes changes to how Guix builds the system, and requires code changes to 
Guix itself, not just to the `configuration.scm`.

The code changes enable:

* `/home` on ZFS.
* Allowing you to put any filesystem inside a `zvol` and declare it in 
`file-system`.
* Assurance that by the time `file-systems` is started, even ZFS automounts 
have been mounted; important if you are starting a Shepherd service that has 
been configured to access filesystems (i.e. all the important ones).

The modifications in the previous email work only in `configuration.scm` but do 
not allow the above.  I think `/home` on ZFS is an important target to have 
(it's what I wanted for my server) at least, but requires changes to how other 
services in Guix System are built anyway.

Thanks
raid5atemyhomework

Re: ZFS on Guix

[Joshua Branson]

raid5atemyhomework  writes:

> The patchset currently dying on issues.guix.gnu.org would provide a nice 
> simple single-step way to enable *very basic* ZFS support on your Guix 
> system.  Until it gets merged, however, you can still enable *very very 
> basic* ZFS support on your Guix system by following the below minimal guide.

If this patch does not get merged into guix, then you could always add
it to the cookbook.  :)

-- 
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar
  

Re: ZFS on Guix

[Danny Milosavljevic]

I just wanted to say that I'm not ignoring your patch, I'm just not qualified
to review it.  I hope someone steps up to it--otherwise I can't really tell
whether (mbegin %state-monad...) inside a random service procedure is a good
idea.

Then again, provenance-service-type does it and there it seems to be fine...


pgpqoMwICNujI.pgp
Description: OpenPGP digital signature

Re: ZFS on Guix

[raid5atemyhomework]

Greetings guix-developers and other people trying to use Guix,

The patchset currently dying on issues.guix.gnu.org would provide a nice simple 
single-step way to enable *very basic* ZFS support on your Guix system.  Until 
it gets merged, however, you can still enable *very very basic* ZFS support on 
your Guix system by following the below minimal guide.

First, select a Linux kernel that the latest ZFS package on Guix supports.  
Current on Guix is ZFS 2.0.1.  That supports up to Linux 5.10.  So I suggest 
using a `linux-libre-5.10` kernel.  Don't go with the default `linux-libre` or 
`linux-libre-lts` since those could be updated to past what the ZFS on Guix 
supports, be explicit with what kernel version you have.  Just remember to 
update to a later LTS version if ZFS on Guix ever gets updated to a later 
version.

Then, you need to create a ZFS package, by adding some declarations before the 
`operating-system` form:

```scheme
(use-modules #;...
 (guix packages))
(use-package-modules #;... linux file-systems)

(define my-kernel linux-libre-5.10)
(define my-zfs
  (package
(inherit zfs)
(arguments (cons* #:linux my-kernel (package-arguments zfs)
```

Then in the `operating-system` form, you need to add the following bits:

* Add ZFS to the kernel-loadable modules so that the installed kernel knows how 
to read and write ZFS disks.
* Add ZFS to the system profile packages so that you can easily manage ZFS 
disks.

So you have to modify a number of places:

```scheme
(operating-system
  (kernel my-kernel)
  (kernel-loadable-modules (list (list my-zfs "module")))
  #;...
  (packages
(append (map specification->package (list "nss-certs"))
(list my-zfs)
%base-packages))
  #;...)
```

With the above you get a ***ridiculously minimal*** ZFS support.

* You have to `sudo modprobe zfs` explicitly at startup.
* You have to `sudo zpool import -a` explicitly at startup.

To do the above automatically at startup you need to add a Shepherd service to 
do the `zpool import`, and add a `kernel-module-loader` extension.  This 
requires a bit more code to be added to your `configuration.scm`.

Here's what I got in my `configuration.scm`:

```scheme
(define zfs-shepherd-services
  (let ((zpool(file-append my-zfs "/sbin/zpool"))
(zfs  (file-append my-zfs "/sbin/zfs"))
(scheme-modules   `((srfi srfi-1)
(srfi srfi-34)
(srfi srfi-35)
(rnrs io ports)
,@%default-modules)))
(define zfs-scan
  (shepherd-service
(provision '(zfs-scan))
(documentation "Scans for ZFS pools.")
(requirement '(kernel-module-loader udev))
(modules scheme-modules)
(start #~(lambda _
   (invoke/quiet #$zpool "import" "-a" "-N")))
(stop #~(const #f
(define zfs-automount
  (shepherd-service
(provision '(zfs-automount))
(documentation "Automounts ZFS data sets.")
(requirement '(zfs-scan))
(modules scheme-modules)
(start #~(lambda _
   (with-output-to-port
 (current-error-port)
 (lambda ()
   (invoke #$zfs "mount" "-a" "-l")
(stop #~(lambda _
  (chdir "/")
  (invoke/quiet #$zfs "unmount" "-a" "-f")
  #f
(list zfs-scan
  zfs-automount)))
```

Then, add some `simple-service`s to your `operating-system`:

```scheme
(use-modules
  #;...
  (gnu services))
(use-service-modules linux shepherd #;...)

#;...

(operating-system
  #;...
  (services
(append
  (list
#;...
(simple-service 'zfs-loader
kernel-module-loader-service-type
'("zfs"))
(simple-service 'zfs-shepherd-services
shepherd-root-service-type
zfs-shepherd-services)
(simple-service 'zfs-shepherd-services-user-processes
user-processes-service-type
'(zfs-automount))
#;...)
  #;...
  %desktop-services))
  #;...)
```

The above lets you mount ZFS pools automatically at startup.  Encrypted pools 
with `keylocation=prompt` will prompt at the console on bootup.

Caveats:

* You can't have a `/home` on ZFS mount.  ZFS has to be mounted before 
`file-systems` target starts, otherwise Guix will fill up the root-mounr's 
`/home` directory and ZFS will refuse to mount over that.  No `/` or `/boot` on 
ZFS either.  Probably no good way to put `/gnu/store` on ZFS either, because 
Guix inherently assumes it's on the `/` mount.
* The above setup does not maintain a `/etc/zfs/zpool.cache` file, because I'm 
not really certain whether it's kosher in Guix to have a file maintained by ZFS 
in the `/etc` dir hierarchy.  This has a number of consequences:
  

Re: Would a Guix QA page be helpful?

[zimoun]

Hi Bengt,

On Sun, 07 Feb 2021 at 22:13, Bengt Richter  wrote:
> On +2021-02-06 20:20:04 +, Christopher Baines wrote:

>> The Guix Data Service has been getting better at finding problems, but
>> getting that data requires knowing it exists, and where to find it, and
>> clicking all the relevant links.
>>
>> I've been wondering if it would be good to have a QA page that just
>> summarises and links to this information, things like:
>>
>>  - Broken packages
>>  - Broken system tests
>>  - Broken fixed output package derivations
>>  - Lint warnings
>>  - ...

[...]

> Mainly, please represent the info so that a person can get it with wget
> and extract items with grep or simple tool and inspect with any editor.
>
> I.e., some form of structured text. Someone will make a shiny browser 
> interface,
> but don't please don't make that a dependency for access to the data.

AFAIK, once you know where the data is, it is JSON.  For example:



The issue is to know what is the API.  Once you know it, you do not need
“shiny browser interface” to get them.  For a simple example, see:




Having the API documented with plain words would be nice.  Help welcome! :-)


> Good tags for searching are helpful, of course.

I have always mixed feeling about tags.  Well, it depends on what you
name ’tags’ and what you want to search for.


All the best,
simon

Re: Would a Guix QA page be helpful?

[zimoun]

Hi Chris,

On Sat, 06 Feb 2021 at 20:20, Christopher Baines  wrote:

> I've been wondering if it would be good to have a QA page that just
> summarises and links to this information, things like:
>
>  - Broken packages
>  - Broken system tests
>  - Broken fixed output package derivations
>  - Lint warnings
>  - ...

This would be really nice.

BTW, where is the API documented?


Cheers,
simon

Re: Would a Guix QA page be helpful?

[Björn Höfling]

Hi Chris,

On Sat, 06 Feb 2021 20:20:04 +
Christopher Baines  wrote:

> Hey,
> 
> The Guix Data Service has been getting better at finding problems, but
> getting that data requires knowing it exists, and where to find it,
> and clicking all the relevant links.

Absolutely yes! I already wanted to ask you for this.

You are working on the data service and from time to time you post very
good improvements on the mailing list. Yet when I go to
data.guix.gnu.org I only get to a fraction of what is already
established. This is a unlucky situation: We have it implemented, but
nobody knows about it.

Thanks,

Björn


pgpaZEOr1CNCS.pgp
Description: OpenPGP digital signature

Re: Would a Guix QA page be helpful?

[Bengt Richter]

Hi Christopher,

tl;dr: +1 :)

On +2021-02-06 20:20:04 +, Christopher Baines wrote:
> Hey,
> 
> The Guix Data Service has been getting better at finding problems, but
> getting that data requires knowing it exists, and where to find it, and
> clicking all the relevant links.
> 
> I've been wondering if it would be good to have a QA page that just
> summarises and links to this information, things like:
> 
>  - Broken packages
>  - Broken system tests
>  - Broken fixed output package derivations
>  - Lint warnings
>  - ...
> 
> While I'd really like to get to a place where less packages and system
> tests are unintentionally broken, and less lint warnings are
> unintentionally introduced, at the moment, there are plenty of these
> problems. There's also things like the broken package sources, where
> there will probably always be new breakages being introduced.
> 
> Given the Guix Data Service can look at what's changed within a time
> period on a branch, recent breakages could also be displayed.
> 
> This could potentially sit at qa.guix.gnu.org.
> 
> Any thoughts?
>

Mainly, please represent the info so that a person can get it with wget
and extract items with grep or simple tool and inspect with any editor.

I.e., some form of structured text. Someone will make a shiny browser interface,
but don't please don't make that a dependency for access to the data.

And please use '+%F %T ' date format for time stamps, UTC, so one doesn't
have to wonder what year it was when looking at an old copy/paste or screen 
capture ;-)

That'd be enough to please me, anyway :)
Good tags for searching are helpful, of course.

> Thanks,
> 
> Chris
-- 
Regards,
Bengt Richter

Re: Building the core-updates branch.

[Leo Famulari]

On Sun, Feb 07, 2021 at 07:43:50PM +0100, Mathieu Othacehe wrote:
> 
> > Turns out "core-updates" is still configured to build the "core" subset,
> > I don't really understand why the evaluation 74281 triggered the build
> > of so many packages.
> 
> Just fixed it with 2c30440f89244202bddebe76f90c6b0b8145ba5d on
> maintenance repository.
> 
> We will now be building 87 jobs instead of 44290 on core-updates.

Awesome!

Re: Building the core-updates branch.

[Mathieu Othacehe]

> Turns out "core-updates" is still configured to build the "core" subset,
> I don't really understand why the evaluation 74281 triggered the build
> of so many packages.

Just fixed it with 2c30440f89244202bddebe76f90c6b0b8145ba5d on
maintenance repository.

We will now be building 87 jobs instead of 44290 on core-updates.

Mathieu

Re: Building the core-updates branch.

[Mathieu Othacehe]

Hey Ludo,

> Agreed, I think that was the intent.  Can we make it focus on the ‘core’
> subset?  How does one do that nowadays?  Looks like my sqlite3/Cuirass
> foo is no longer helpful.  :-)

Turns out "core-updates" is still configured to build the "core" subset,
I don't really understand why the evaluation 74281 triggered the build
of so many packages.

Regarding the sqlite3 -> psql transition, there are not that much
differences. I fire up psql this way:

--8<---cut here---start->8---
sudo -u cuirass -s /bin/sh -c 'psql cuirass -h /var/run/postgresql'
--8<---cut here---end--->8---

The commands that are listed here[1] are still (mostly) relevant, even
if this document would really need to be updated. I plan to improve the
Cuirass documentation for a future 1.0 release anyways.

Thanks,

Mathieu

[1]: 
https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/cuirass-howto.org

Re: Blog post about the upcoming FOSDEM + Guix Day

[Pjotr Prins]

Just a reminder: tomorrow we have a GNU Guix day online! Free for all.

See 

https://libreplanet.org/wiki/Group:Guix/FOSDEM2021

Starts at 5am UTC with a Greek breakfast and runs all day until we are
done! See the link in above page.

Pj.