bug#30760: guix system init broken on non GuixSD

[Ludovic Courtès]

Hi!

l...@gnu.org (Ludovic Courtès) skribis:

> Nevertheless, the risk of false-positives obviously exists, hence the
> need for an escape hatch.
>
> What about the attached patch?

Pushed as 61b1dbbdcd87f6b37d6b87715a9a4da4e63485ab.

Thanks,
Ludo’.

bug#30760: guix system init broken on non GuixSD

[Ludovic Courtès]

Hello,

Danny Milosavljevic  skribis:

>> I'm afraid this is still not correct.
>> 
>> # guix system init config.scm /mnt/mnt/
>> ...
>> config.scm:64:9: error: you may need these modules in the initrd for 
>> /dev/nvme0n1p2: shpchp
>> hint: Try adding them to the `initrd-modules' field of your 
>> `operating-system' declaration, along these lines:
>> 
>>   (operating-system
>> ;; ...
>> (initrd-modules (append (list "shpchp")
>> %base-initrd-modules)))
>> 
>> I don't have `shpchp` as a module as I have it compiled into kernel
>> directly. Can I somehow disable the check?

Exactly what I feared.  ;-)

> I think it's a good idea to add a command-line switch that disables the check.
>
> But then people will just disable the check always and it won't improve until
> it's correct.  It's still a good idea to give people the choice.
>
> @Ludo: It would also be great to have a command-line switch to check the slow,
> correct, way.  We'd also have to check modules.builtin of the new system's 
> initrd
> - but we'd do it only when the option is passed :)
>
> I suggest to change it to:
>
>> # guix system init config.scm /mnt/mnt/
>> ...
>> config.scm:64:9: WARNING: you may need these modules in the initrd for 
>> /dev/nvme0n1p2: shpchp
>^^^ not error

I thought about making it a warning rather than an error back then, but
thought that it wouldn’t work well: the warning would immediately go
off-screen as build logs start scrolling by.

Thus I took the optimistic view that false positives like the one Tomáš
experienced should be rare because usually init/reconfigure are used on
GuixSD, with a kernel config very close to the target config.

Nevertheless, the risk of false-positives obviously exists, hence the
need for an escape hatch.

What about the attached patch?

Thanks,
Ludo’.

diff --git a/doc/guix.texi b/doc/guix.texi
index d3a7908f9..bcea89e07 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -20458,6 +20458,16 @@ of the image size as a function of the size of the system declared in
 Make @var{file} a symlink to the result, and register it as a garbage
 collector root.
 
+@item --skip-checks
+Skip pre-installation safety checks.
+
+By default, @command{guix system init} and @command{guix system
+reconfigure} perform safety checks: they make sure the file systems that
+appear in the @code{operating-system} declaration actually exist
+(@pxref{File Systems}), and that any Linux kernel modules that may be
+needed at boot time are listed in @code{initrd-modules} (@pxref{Initial
+RAM Disk}).  Passing this option skips these tests altogether.
+
 @item --on-error=@var{strategy}
 Apply @var{strategy} when an error occurs when reading @var{file}.
 @var{strategy} may be one of the following:
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index acfccce96..f0c4a2ba1 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -733,7 +733,8 @@ and TARGET arguments."
   (#$installer #$bootloader #$device #$target))
 
 (define* (perform-action action os
- #:key install-bootloader?
+ #:key skip-safety-checks?
+ install-bootloader?
  dry-run? derivations-only?
  use-substitutes? bootloader-target target
  image-size file-system-type full-boot?
@@ -750,7 +751,10 @@ When DERIVATIONS-ONLY? is true, print the derivation file name(s) without
 building anything.
 
 When GC-ROOT is a path, also make that path an indirect root of the build
-output when building a system derivation, such as a disk image."
+output when building a system derivation, such as a disk image.
+
+When SKIP-SAFETY-CHECKS? is true, skip the file system and initrd module
+static checks."
   (define println
 (cut format #t "~a~%" <>))
 
@@ -760,7 +764,8 @@ output when building a system derivation, such as a disk image."
   ;; Check whether the declared file systems exist.  This is better than
   ;; instantiating a broken configuration.  Assume that we can only check if
   ;; running as root.
-  (when (memq action '(init reconfigure))
+  (when (and (not skip-safety-checks?)
+ (memq action '(init reconfigure)))
 (check-mapped-devices os)
 (when (zero? (getuid))
   (check-file-system-availability (operating-system-file-systems os))
@@ -933,6 +938,8 @@ Some ACTIONS support additional ARGS.\n"))
   --expose=SPEC  for 'vm', expose host file system according to SPEC"))
   (display (G_ "
   --full-bootfor 'vm', make a full boot sequence"))
+  (display (G_ "
+  --skip-checks  skip file system and initrd module safety checks"))
   (newline)
   (display (G_ "
   -h, --help display this help and exit"))
@@ -974,6 +981,9 @@ Some ACTIONS support additional ARGS.\n"))
  (option '("full-boot") #f #f
  

bug#30760: guix system init broken on non GuixSD

[Danny Milosavljevic]

Or maybe to this:

> > # guix system init config.scm /mnt/mnt/
> > ...
> > config.scm:64:9: WARNING: you may need these modules in the initrd for 
> > /dev/nvme0n1p2: shpchp  
>^^^ not error
> > hint: Try adding them to the `initrd-modules' field of your 
> > `operating-system' declaration, along these lines:
> > 
> >   (operating-system
> > ;; ...
> > (initrd-modules (append (list "shpchp")
> > %base-initrd-modules)))
> >
> > (sleeps 5 s)
> > (builds entire system)
> > (checks the slow, correct way at the end - when everything was built 
> > already anyway)


pgpKyUIGlKaWv.pgp
Description: OpenPGP digital signature

bug#30760: guix system init broken on non GuixSD

[Danny Milosavljevic]

> I'm afraid this is still not correct.
> 
> # guix system init config.scm /mnt/mnt/
> ...
> config.scm:64:9: error: you may need these modules in the initrd for 
> /dev/nvme0n1p2: shpchp
> hint: Try adding them to the `initrd-modules' field of your 
> `operating-system' declaration, along these lines:
> 
>   (operating-system
> ;; ...
> (initrd-modules (append (list "shpchp")
> %base-initrd-modules)))
> 
> I don't have `shpchp` as a module as I have it compiled into kernel
> directly. Can I somehow disable the check?

I think it's a good idea to add a command-line switch that disables the check.

But then people will just disable the check always and it won't improve until
it's correct.  It's still a good idea to give people the choice.

@Ludo: It would also be great to have a command-line switch to check the slow,
correct, way.  We'd also have to check modules.builtin of the new system's 
initrd
- but we'd do it only when the option is passed :)

I suggest to change it to:

> # guix system init config.scm /mnt/mnt/
> ...
> config.scm:64:9: WARNING: you may need these modules in the initrd for 
> /dev/nvme0n1p2: shpchp
   ^^^ not error
> hint: Try adding them to the `initrd-modules' field of your 
> `operating-system' declaration, along these lines:
> 
>   (operating-system
> ;; ...
> (initrd-modules (append (list "shpchp")
> %base-initrd-modules)))
>
> If you think this warning is mistaken, invoke guix again with the option
> --enable-paranoid-initrd-checks to be on the safe side, or with the option
> --skip-initrd-checks to continue regardless.


pgpgt7F21OzMY.pgp
Description: OpenPGP digital signature

bug#30760: guix system init broken on non GuixSD

[Tomáš Čech]

On Sun, Mar 11, 2018 at 10:38:18PM +0100, Ludovic Courtès wrote:

Tomáš Čech  skribis:


In ice-9/boot-9.scm:
   829:9  1 (catch system-error # …)
In gnu/system/linux-initrd.scm:
   361:6  0 (_)

gnu/system/linux-initrd.scm:361:6: known-module-aliases: unbound variable


My bad!  Danny eventually fixed it in
0803ddf2677ead5e9d8ef698316125e0c8b9c998.


I'm afraid this is still not correct.

# guix system init config.scm /mnt/mnt/
...
config.scm:64:9: error: you may need these modules in the initrd for 
/dev/nvme0n1p2: shpchp
hint: Try adding them to the `initrd-modules' field of your `operating-system' 
declaration, along these lines:

 (operating-system
   ;; ...
   (initrd-modules (append (list "shpchp")
   %base-initrd-modules)))

I don't have `shpchp` as a module as I have it compiled into kernel
directly. Can I somehow disable the check?

Thanks.

S_W


signature.asc
Description: Digital signature

bug#30760: guix system init broken on non GuixSD

[Danny Milosavljevic]

Hi Ludo,

> BTW, we should add a ‘--skip-checks’ option to ‘guix system’ so that
> users can skip those checks.  That’d provide an escape hatch in case
> ‘check-device-initrd-modules’ makes the wrong diagnostic.

Yeah, good idea!


pgpBUFlRA1b_T.pgp
Description: OpenPGP digital signature

bug#30760: guix system init broken on non GuixSD

[Tomáš Čech]

On Sat, Mar 10, 2018 at 12:19:52AM +0100, Ludovic Courtès wrote:

Danny Milosavljevic  skribis:


[huge build]

The current tradeoff is to make that diagnostic based on the running
kernel, even if it’s an approximation.


Ah, good point.


If that’s fine with you I’d like to fix this bug with the conservative
patch below.


Sure, looks good.


Pushed as 8d5c14edf5a6d01f859b1aa00c836ffdb5ddecf4.


I'm afraid that now it leads to:

Backtrace:
12 (primitive-load "/usr/bin/guix")
In guix/ui.scm:
1501:12 11 (run-guix-command _ . _)
In ice-9/boot-9.scm:
  829:9 10 (catch _ _ # …)
  829:9  9 (catch _ _ # …)
In guix/scripts/system.scm:
 1180:8  8 (_)
 1052:6  7 (process-action _ _ _)
In guix/store.scm:
1443:24  6 (run-with-store _ _ #:guile-for-build _ #:system _ # _)
In guix/scripts/system.scm:
1065:13  5 (_ _)
  764:4  4 (perform-action init #< kernel: # …)
In srfi/srfi-1.scm:
  640:9  3 (for-each # …)
In gnu/system/linux-initrd.scm:
  360:4  2 (check-device-initrd-modules "/dev/nvme0n1p2" ("ahci" …) …)
In ice-9/boot-9.scm:
  829:9  1 (catch system-error # …)
In gnu/system/linux-initrd.scm:
  361:6  0 (_)

gnu/system/linux-initrd.scm:361:6: known-module-aliases: unbound variable


This is part of my config:

(initrd (lambda (file-system . rest)
 (raw-initrd file-systems
 #:linux linux-x1-sw1
 #:linux-modules '()
 #:helper-packages '(linux-firmware-initrd-x1-sw1)
 #:mapped-devices mapped-devices)))


I don't have any modules to be loaded in initrd, kernel is compiled
using my configuration which fits my needs and follows the HW it will run on.

S_W


signature.asc
Description: Digital signature

bug#30760: guix system init broken on non GuixSD

[Ludovic Courtès]

Danny Milosavljevic  skribis:

> [huge build]
>> The current tradeoff is to make that diagnostic based on the running
>> kernel, even if it’s an approximation.
>
> Ah, good point.
>
>> If that’s fine with you I’d like to fix this bug with the conservative
>> patch below.
>
> Sure, looks good.

Pushed as 8d5c14edf5a6d01f859b1aa00c836ffdb5ddecf4.

> While we are approximating we could also in a later version fall back to
> (the host system's) "`cat /proc/sys/kernel/modprobe` --showconfig" - it
> could be used to find aliases.
>
> But maybe that would make it brittle.  Hmm...

Yeah, I don’t think it’d make a big difference.

BTW, we should add a ‘--skip-checks’ option to ‘guix system’ so that
users can skip those checks.  That’d provide an escape hatch in case
‘check-device-initrd-modules’ makes the wrong diagnostic.

Thoughts?

Ludo’.

bug#30760: guix system init broken on non GuixSD

[Danny Milosavljevic]

[huge build]
> The current tradeoff is to make that diagnostic based on the running
> kernel, even if it’s an approximation.

Ah, good point.

> If that’s fine with you I’d like to fix this bug with the conservative
> patch below.

Sure, looks good.

While we are approximating we could also in a later version fall back to
(the host system's) "`cat /proc/sys/kernel/modprobe` --showconfig" - it
could be used to find aliases.

But maybe that would make it brittle.  Hmm...




pgp3NttpMp91T.pgp
Description: OpenPGP digital signature

bug#30760: guix system init broken on non GuixSD

[Ludovic Courtès]

Hello,

Danny Milosavljevic  skribis:

> there's a problem with check-device-initrd-modules: on "guix system init"
> it doesn't get linux-module-directory and doesn't pass it on to 
> matching-modules.
> matching-modules then eventually defaults to (current-alias-file) - which is 
> not
> found on a non-GuixSD system.

Yeah.

> Would it be possible to get rid of the defaults in 
> gnu/build/linux-modules.scm ?
> I don't think those are safe or useful for our requirements.  I've had to work
> around those before.
>
> check-initrd-modules could use the initrd's new kernel modules to find
> out which modules to include (after all).
>
> Then we could also check the dependencies directly in the new Linux kernel
> modules and all in all it would be safer.

It would be safer indeed, but we’d have to build the kernel and
everything before we can make a diagnostic.  That would lead to a weird
user experience, similar to what we currently see with grafts (things
are built/downloader, and later on you get a message about what’s going
to be built.)

The current tradeoff is to make that diagnostic based on the running
kernel, even if it’s an approximation.

If that’s fine with you I’d like to fix this bug with the conservative
patch below.

Thoughts?

Ludo’.

diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm
index 4fe673cca..8cae4fb63 100644
--- a/gnu/build/linux-modules.scm
+++ b/gnu/build/linux-modules.scm
@@ -40,6 +40,7 @@
 current-module-debugging-port
 
 device-module-aliases
+current-alias-file
 known-module-aliases
 matching-modules))
 
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 1eb5f5130..16a8c4375 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -353,17 +353,27 @@ loaded at boot time in the order in which they appear."
 (define (check-device-initrd-modules device linux-modules location)
   "Raise an error if DEVICE needs modules beyond LINUX-MODULES to operate.
 DEVICE must be a \"/dev\" file name."
-  (let ((modules (delete-duplicates
-  (append-map matching-modules
-  (device-module-aliases device)
-(unless (every (cute member <> linux-modules) modules)
-  (raise (condition
-  (
-   (message (format #f (G_ "you may need these modules \
+  (define aliases
+;; Attempt to load 'modules.alias' from the current kernel, assuming we're
+;; on GuixSD, and assuming that corresponds to the kernel we'll be
+;; installing.  Skip the whole thing if that file cannot be read.
+(catch 'system-error
+  (lambda ()
+(known-module-aliases))
+  (const #f)))
+
+  (when aliases
+(let ((modules (delete-duplicates
+(append-map (cut matching-modules <> aliases)
+(device-module-aliases device)
+  (unless (every (cute member <> linux-modules) modules)
+(raise (condition
+(
+ (message (format #f (G_ "you may need these modules \
 in the initrd for ~a:~{ ~a~}")
-device modules)))
-  (
-   (hint (format #f (G_ "Try adding them to the
+  device modules)))
+(
+ (hint (format #f (G_ "Try adding them to the
 @code{initrd-modules} field of your @code{operating-system} declaration, along
 these lines:
 
@@ -373,8 +383,8 @@ these lines:
(initrd-modules (append (list~{ ~s~})
%base-initrd-modules)))
 @end example\n")
- modules)))
-  (
-   (location (source-properties->location location
+   modules)))
+(
+ (location (source-properties->location location)
 
 ;;; linux-initrd.scm ends here

bug#30760: guix system init broken on non GuixSD

[Danny Milosavljevic]

Hi Ludo,

there's a problem with check-device-initrd-modules: on "guix system init"
it doesn't get linux-module-directory and doesn't pass it on to 
matching-modules.
matching-modules then eventually defaults to (current-alias-file) - which is not
found on a non-GuixSD system.

Would it be possible to get rid of the defaults in gnu/build/linux-modules.scm ?
I don't think those are safe or useful for our requirements.  I've had to work
around those before.

check-initrd-modules could use the initrd's new kernel modules to find
out which modules to include (after all).

Then we could also check the dependencies directly in the new Linux kernel
modules and all in all it would be safer.


pgpqWWdQXmq_G.pgp
Description: OpenPGP digital signature

bug#30760: guix system init broken on non GuixSD

[Tomáš Čech]

`guix system init` seems to be broken for non GuixSD distirbutions:
When I tried it on openSUSE:

# guix system --no-bootloader init /Devel/git/guix-config/config.scm /mnt/mnt/
;;; note: source file /Devel/extra/gnu/packages/connman.scm
;;;   newer than compiled /root/.config/guix/latest/gnu/packages/connman.go
;;; note: source file /Devel/extra/gnu/packages/connman.scm
;;;   newer than compiled 
/usr/lib64/guile/2.2/site-ccache/gnu/packages/connman.go
;;; note: source file /Devel/extra/gnu/packages/connman.scm
;;;   newer than compiled 
/usr/lib64/guile/2.2/site-ccache/gnu/packages/connman.go
guix system: error: open-file: No such file or directory: 
"/run/booted-system/kernel/lib/modules/4.15.6-1-default/modules.alias"

4.15.6-1-default is version of my running kernel, but not defined as package - 
it is not expected to be used for guix call.

/run/booted-system/ is specific for GuixSD.


signature.asc
Description: Digital signature