Re: [gentoo-user] Re: Question about genkernel's default kernel config

[Raymond Jennings]

On Thu, Aug 18, 2016 at 3:50 PM, Rich Freeman  wrote:

> On Thu, Aug 18, 2016 at 6:41 PM, Raymond Jennings 
> wrote:
> > On Mon, Aug 15, 2016 at 1:32 AM, Rich Freeman  wrote:
> >>
> >> And since it uses udev it is fairly robust against things like adding
> >> a drive and now the kernel re-letters everything.
> >
> > Did you seriously just post that on a gentoo list?
> >
>
> Yes
>
> > I assume you mean /dev/sd? and not A: B: C:
> >
>
> Yes.  What else would you call it?
>

A bad attempt at humor on my part.

Re: [gentoo-user] Re: Question about genkernel's default kernel config

[Rich Freeman]

On Thu, Aug 18, 2016 at 6:41 PM, Raymond Jennings  wrote:
> On Mon, Aug 15, 2016 at 1:32 AM, Rich Freeman  wrote:
>>
>> And since it uses udev it is fairly robust against things like adding
>> a drive and now the kernel re-letters everything.
>
> Did you seriously just post that on a gentoo list?
>

Yes

> I assume you mean /dev/sd? and not A: B: C:
>

Yes.  What else would you call it?

-- 
Rich

Re: [gentoo-user] Re: Question about genkernel's default kernel config

[Raymond Jennings]

On Mon, Aug 15, 2016 at 1:32 AM, Rich Freeman  wrote:

> And since it uses udev it is fairly robust against things like adding
> a drive and now the kernel re-letters everything.
>

Did you seriously just post that on a gentoo list?

I assume you mean /dev/sd? and not A: B: C:

:P

Re: [gentoo-user] Re: Question about genkernel's default kernel config

[J. Roeleveld]

On Monday, August 15, 2016 08:41:31 AM Rich Freeman wrote:
> On Mon, Aug 15, 2016 at 5:06 AM, J. Roeleveld  wrote:
> > On Monday, August 15, 2016 04:32:29 AM Rich Freeman wrote:
> >> It is also somewhat dependent on a correct fstab.  Don't take that for
> >> granted: the kernel doesn't look at fstab at all when mounting root,
> >> and neither do most of the other tools, so if your root partition
> >> isn't correctly defined in fstab you might never know it and dracut
> >> will get confused.  If nothing else once it does have it correctly
> >> mounted it will read fstab and then mess it up when it re-mounts root
> >> per "your" instructions.
> > 
> > I understand what you're saying. Except in my case, that wasn't the cause.
> 
> Oh, I agree.  I just said that in lieu of replying to every single
> other email in this thread.  :)
> 
> >> If you just need to tweak dracut behavior you may be better off with a
> >> dracut module.  They're just shell scripts and pretty simple to write.
> >> That lets you tweak something at some point during boot without having
> >> to build the entire thing yourself.
> > 
> > This actually had (or has, not bothered to check current status) a
> > distinct
> > lack of usable documentation. Looking for clear howto's on creating my own
> > (embedded into the kernel) initramfs was a lot quicker.
> 
> Yeah, I found this frustrating as well.
> 
> This may or may not be helpful:
> https://rich0gentoo.wordpress.com/2012/01/21/a-quick-dracut-module/

It shows where to find it and add custom steps.
For my scenario, I'd need to first find where the code is that currently 
handles 
it. Disable that and insert my own in its place.
The majority works, except for it asking for my passphrase multiple times, 
which, in Dracut, might actually be handled by "cryptsetup" itself. In which 
case I'm stuck.

In my script it's handled via:

# Obtain key through loop device
/sbin/losetup /dev/loop0 /key.iso || rescue_shell
/sbin/cryptsetup -T 5 luksOpen /dev/loop0 key || rescue_shell

# Unlock the root partition
/sbin/cryptsetup --key-file /dev/mapper/key luksOpen --allow-discards /dev/sda2 
sda2 || rescue_shell
/sbin/cryptsetup --key-file /dev/mapper/key luksOpen --allow-discards /dev/sdb2 
sdb2 || rescue_shell

# Clean up the loop device
/sbin/cryptsetup luksClose key
/sbin/losetup -d /dev/loop0


> In general the benefits of using dracut are the benefits of using
> anything that somebody else maintains.  You could replace openrc with
> a single shell script as well, or a fair bit of systemd.  That doesn't
> mean that this is really the optimal approach.

If I'd need the level of complexity Dracut allows, I'd look into it. But all I 
need to do is maintain a few small and simple files and the kernel-build 
handles all the heavy lifting.

> >> And since it uses udev it is fairly robust against things like adding
> >> a drive and now the kernel re-letters everything.
> > 
> > If I were using normal partitioning, I wouldn't need an initramfs.
> 
> An initramfs is beneficial even if you don't "need" one.  To start
> with it allows you to build a more modular kernel, which is especially
> beneficial if you aren't customizing your kernel for every host.  It
> also tends to be more robust when something goes wrong.  You end up
> having a rescue shell even if root doesn't mount, more robust fsck/etc
> during early boot, and it is going to be a lot smarter when you
> add/remove a drive (since root can be identified by UUID or label).
> 
> Dracut is becoming common enough that I think it is worth learning...

For binary distributions, I tend to stick with the supplied kernel and 
initramfs. Those are usually for running software from big vendors like Oracle 
or IBM.

For the rest, it's quicker to make the few changes needed to the init-script.
I don't have a large serverfarm requiring that level of standardisation.

--
Joost

Re: [gentoo-user] Re: Question about genkernel's default kernel config

[Rich Freeman]

On Mon, Aug 15, 2016 at 5:06 AM, J. Roeleveld  wrote:
> On Monday, August 15, 2016 04:32:29 AM Rich Freeman wrote:
>>
>> It is also somewhat dependent on a correct fstab.  Don't take that for
>> granted: the kernel doesn't look at fstab at all when mounting root,
>> and neither do most of the other tools, so if your root partition
>> isn't correctly defined in fstab you might never know it and dracut
>> will get confused.  If nothing else once it does have it correctly
>> mounted it will read fstab and then mess it up when it re-mounts root
>> per "your" instructions.
>
> I understand what you're saying. Except in my case, that wasn't the cause.
>

Oh, I agree.  I just said that in lieu of replying to every single
other email in this thread.  :)

>> If you just need to tweak dracut behavior you may be better off with a
>> dracut module.  They're just shell scripts and pretty simple to write.
>> That lets you tweak something at some point during boot without having
>> to build the entire thing yourself.
>
> This actually had (or has, not bothered to check current status) a distinct
> lack of usable documentation. Looking for clear howto's on creating my own
> (embedded into the kernel) initramfs was a lot quicker.

Yeah, I found this frustrating as well.

This may or may not be helpful:
https://rich0gentoo.wordpress.com/2012/01/21/a-quick-dracut-module/

In general the benefits of using dracut are the benefits of using
anything that somebody else maintains.  You could replace openrc with
a single shell script as well, or a fair bit of systemd.  That doesn't
mean that this is really the optimal approach.


>> And since it uses udev it is fairly robust against things like adding
>> a drive and now the kernel re-letters everything.

> If I were using normal partitioning, I wouldn't need an initramfs.

An initramfs is beneficial even if you don't "need" one.  To start
with it allows you to build a more modular kernel, which is especially
beneficial if you aren't customizing your kernel for every host.  It
also tends to be more robust when something goes wrong.  You end up
having a rescue shell even if root doesn't mount, more robust fsck/etc
during early boot, and it is going to be a lot smarter when you
add/remove a drive (since root can be identified by UUID or label).

Dracut is becoming common enough that I think it is worth learning...

-- 
Rich

Re: [gentoo-user] Re: Question about genkernel's default kernel config

[J. Roeleveld]

On Monday, August 15, 2016 04:32:29 AM Rich Freeman wrote:
> On Mon, Aug 15, 2016 at 4:10 AM, J. Roeleveld  wrote:
> > The ones created by genkernel or dracut always need a few iterations
> > before
> > they work semi-reliably and are not flexible enough.
> > I have 2 disks in my laptop. Both are encrypted using LUKS and the same
> > passphrase. Neither genkernel nor dracut have the intelligence to ask me
> > once and try the key on both, only asking for a 2nd key when the provided
> > one doesn't work for both.
> > 
> > I ended up writing my own, which has proven more reliable and stable. The
> > reduced size also makes maintenance less of an issue.
> 
> Nothing wrong with this if you know what you're doing.  Dracut is
> designed to be one-size-fits-all with a lot of logic to figure out
> what your system needs to boot.
> 
> It is also somewhat dependent on a correct fstab.  Don't take that for
> granted: the kernel doesn't look at fstab at all when mounting root,
> and neither do most of the other tools, so if your root partition
> isn't correctly defined in fstab you might never know it and dracut
> will get confused.  If nothing else once it does have it correctly
> mounted it will read fstab and then mess it up when it re-mounts root
> per "your" instructions.

I understand what you're saying. Except in my case, that wasn't the cause.

> If you just need to tweak dracut behavior you may be better off with a
> dracut module.  They're just shell scripts and pretty simple to write.
> That lets you tweak something at some point during boot without having
> to build the entire thing yourself.

This actually had (or has, not bothered to check current status) a distinct 
lack of usable documentation. Looking for clear howto's on creating my own 
(embedded into the kernel) initramfs was a lot quicker.

> And since it uses udev it is fairly robust against things like adding
> a drive and now the kernel re-letters everything.

If I were using normal partitioning, I wouldn't need an initramfs.

> If you tweak it to use the same password for all drives I wouldn't be
> surprised if upstream accepts the patch...

If it were properly documented where it does this, I would be willing to try.
Right now, my init-script is 45 lines (including white-space)

My laptop has 2 disks. Both are fully encrypted (apart from a small /boot).
On top of the encrypted parts I have LVM, which contains my partitions.

It asks for a password, decrypts both drives, enables LVM, checks if it needs 
to resume from "suspend to disk", if not, mounts the partitions and boots. If 
it does need to resume, it simply resumes.
If it fails anywhere, it drops me into a busybox shell.

I can then simply "cat" the init script to see what commands I need to run to 
boot, replacing anything that is no longer valid.

With dracut and genkernel, the init-script is too convoluted to simply follow. 
Which means it is not allowed to fail.

--
Joost

Re: [gentoo-user] Re: Question about genkernel's default kernel config

[Neil Bothwick]

On Mon, 15 Aug 2016 10:10:22 +0200, J. Roeleveld wrote:

> > A homegrown initramfs created by a novice is going to be more reliable
> > than one created by dracut or genkernel? Seems unlikely.  
> 
> The ones created by genkernel or dracut always need a few iterations
> before they work semi-reliably and are not flexible enough.

Yes, I usually have to rerun dracut a few times after tweaking its
options, because I don't use the full-fat defaults. However, when I
rolled my own initramfs, there were several more iterations after making
changes. In my experience, once you have a sensible config, dracut just
works - your comments about multiple LUKS partitions notwithstanding.


-- 
Neil Bothwick

Biology is the only science in which multiplication means the same thing
as division.


pgpYg7sYNYLnD.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Re: Question about genkernel's default kernel config

[Rich Freeman]

On Mon, Aug 15, 2016 at 4:10 AM, J. Roeleveld  wrote:
>
> The ones created by genkernel or dracut always need a few iterations before
> they work semi-reliably and are not flexible enough.
> I have 2 disks in my laptop. Both are encrypted using LUKS and the same
> passphrase. Neither genkernel nor dracut have the intelligence to ask me once
> and try the key on both, only asking for a 2nd key when the provided one
> doesn't work for both.
>
> I ended up writing my own, which has proven more reliable and stable. The
> reduced size also makes maintenance less of an issue.
>

Nothing wrong with this if you know what you're doing.  Dracut is
designed to be one-size-fits-all with a lot of logic to figure out
what your system needs to boot.

It is also somewhat dependent on a correct fstab.  Don't take that for
granted: the kernel doesn't look at fstab at all when mounting root,
and neither do most of the other tools, so if your root partition
isn't correctly defined in fstab you might never know it and dracut
will get confused.  If nothing else once it does have it correctly
mounted it will read fstab and then mess it up when it re-mounts root
per "your" instructions.

If you just need to tweak dracut behavior you may be better off with a
dracut module.  They're just shell scripts and pretty simple to write.
That lets you tweak something at some point during boot without having
to build the entire thing yourself.

And since it uses udev it is fairly robust against things like adding
a drive and now the kernel re-letters everything.

If you tweak it to use the same password for all drives I wouldn't be
surprised if upstream accepts the patch...

-- 
Rich

Re: [gentoo-user] Re: Question about genkernel's default kernel config

[J. Roeleveld]

On Monday, August 15, 2016 09:48:14 AM Jeroen Mathon wrote:
> I have used genkernel for my initramfs.
> 
> The dracut one seems to hang on a kernel panic(Implying that it did not
> load the correct modules for luks and lvm).

In my experience, normal for both... 

> Besides the size of your initramfs should not matter since its only used
> to mount the partitions and do the pre-kernel loading.

Agreed, except for the next part:

> The only thing that you will win by making your own initramfs would be
> size and boot speed.

If done sanely, the reduced complexity makes debugging and maintenance a lot 
easier.

> Correct me if im wrong btw.
> 
> On 15-08-16 02:45, Rich Freeman wrote:
> > On Sun, Aug 14, 2016 at 7:38 PM, Mike Gilbert  wrote:
> >> On Sun, Aug 14, 2016 at 4:29 PM, J. Roeleveld  wrote:
> >>> On August 14, 2016 8:55:32 PM GMT+02:00, Neil Bothwick 
 wrote:
>  On Sun, 14 Aug 2016 11:48:08 -0700, Ian Zimmerman wrote:
> > Don't you still need genkernel if you want to build an initramfs?
>  
>  No, dracut.
>  
> > The handbook (amd64) seems to imply you do, and I don't know of an
>  
>  easy
>  
> > way to build an initramfs just with the bare kernel source.
>  
>  Dracut.
> >>> 
> >>> Dracut if you just want something quick and easy.
> >>> If you want something small and reliable, build your own.
> >> 
> >> A homegrown initramfs created by a novice is going to be more reliable
> >> than one created by dracut or genkernel? Seems unlikely.
> > 
> > This is a silly argument.  Everybody knows that anybody who wants
> > something reliable just writes their own kernel in the first place.

Re: [gentoo-user] Re: Question about genkernel's default kernel config

[J. Roeleveld]

On Sunday, August 14, 2016 08:38:42 PM Mike Gilbert wrote:
> On Sun, Aug 14, 2016 at 4:29 PM, J. Roeleveld  wrote:
> > On August 14, 2016 8:55:32 PM GMT+02:00, Neil Bothwick 
 wrote:
> >>On Sun, 14 Aug 2016 11:48:08 -0700, Ian Zimmerman wrote:
> >>> Don't you still need genkernel if you want to build an initramfs?
> >>
> >>No, dracut.
> >>
> >>> The handbook (amd64) seems to imply you do, and I don't know of an
> >>
> >>easy
> >>
> >>> way to build an initramfs just with the bare kernel source.
> >>
> >>Dracut.
> >>
> > Dracut if you just want something quick and easy.
> > If you want something small and reliable, build your own.
> 
> A homegrown initramfs created by a novice is going to be more reliable
> than one created by dracut or genkernel? Seems unlikely.

The ones created by genkernel or dracut always need a few iterations before 
they work semi-reliably and are not flexible enough.
I have 2 disks in my laptop. Both are encrypted using LUKS and the same 
passphrase. Neither genkernel nor dracut have the intelligence to ask me once 
and try the key on both, only asking for a 2nd key when the provided one 
doesn't work for both.

I ended up writing my own, which has proven more reliable and stable. The 
reduced size also makes maintenance less of an issue.

--
Joost