Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Raymond Jennings]

I think this might be one reason that /etc/mtab was deprecated in favor of
a symlink to /proc/mounts :P

On Thu, Feb 18, 2016 at 9:07 PM, Duncan <1i5t5.dun...@cox.net> wrote:

> Rich Freeman posted on Thu, 18 Feb 2016 07:22:36 -0500 as excerpted:
>
> > 4.  In the runlevel paradigm you usually think of services running
> > inside a runlevel (perhaps this isn't strictly true, but most people
> > think this way, in part because runlevels don't change much).  In
> > systemd this really isn't the case.  Services run before targets, or
> > after them.  A target won't be considered running if anything it depends
> > on isn't running.
>
> Some minor additional notes, with the first one being here.
>
> Systemd target units are analogous to edge-triggered interrupts, which
> they resemble in that they are simply "synchronization points" (the term
> used in the systemd.target (5) manpage itself).  Level-triggered
> interrupts can be held on or held off (high or low), but edge-triggered a
> re simply events that occur and then are passed as time moves on.  As
> such, targets can be started, but not normally (while the job queue is
> idle) stopped, as they de-assert as soon as they are actually reached,
> tho many of their requirements generally continue to run until stopped by
> some other event, often conflicts= against some other target or general
> unit being started, or specific admin systemctl stop.
>
> Tho the systemd FAQ suggests this wasn't always so, as it suggests using
> systemctl list-units --type=target in answer to a question about how to
> find what "runlevel" you're in.  That command seems to return nothing,
> here, tho, at least while no target is actively starting, so it would
> seem that answer's a bit dated.
>
> It can be noted, however, that certain units, most often specific targets
> intended to be specifically invokable by users, can be "isolated", as
> they have the AllowIsolate unit setting.  Systemctl isolate  will
> then cause it to be started exclusively, stopping anything that's not a
> dependency of that unit.  The systemctl emergency, rescue, reboot,
> shutdown, etc, commands, then become effectively shortcuts to the longer
> systemctl isolate  command form.
>
> > 5.  I'd have to check, but I wouldn't be surprised if systemd doesn't
> > actually require specifying a target at all.  Your default "runlevel"
> > could be apache2.service, which means the system would boot and launch
> > everything necessary to get apache working, and it probably wouldn't
> > even spawn a getty.  This is NOT analogous to just putting only apache2
> > in /etc/runlevels/default, because in that example openrc is running the
> > default runlevel, and it only pulls in apache2.  Systemd is purely
> > dependency driven and when you tell it to make graphical.target the
> > default runlevel it is like running emerge kde-meta.  If all you wanted
> > was kde-runtime you wouldn't redefine kde-meta to pull in only
> > kde-runtime, you'd just run emerge kde-runtime.  Again, I haven't tested
> > this, but I'd be shocked if it didn't work.  Of course, specifying a
> > service as a default instead of a target is very limiting, but it would
> > probably work.  Heck, you could probably specify a mount as the default
> > and the system would just boot and mount something and sit there.  Or
> > you could make it a socket and all it would do is sit and listen for a
> > connection inetd-style.
>
> As mentioned both in the systemd FAQ and in the systemd.special (7)
> manpage, under default.target, this is the default unit started at
> bootup.  Normally, it'll be a symlink to either multi-user.target
> (analogous to sysvinit semi-standard runlevel 3, CLI, no X), or
> graphical.target (analogous to sysvinit semi-standard runlevel 5,
> launching X and and a graphical *DM login).
>
> I don't see specific documentation of whether symlinking to a non-target
> unit is allowed, but systemd does have a commandline option --unit=,
> which is explicitly documented to take a _unit_, default.target being the
> default, but other non-target units being possible as well.  Presumably
> systemd would examine said unit, looking for DefaultDependencies=no, and
> if not specifically set, would start the early "system level" targets,
> before starting the named unit in place of the normal default.target.
>
> So it's definitely possible to do via systemd commandline, but I'm not
> sure if default.target is followed if it doesn't symlink a target unit,
> or not.  I'd guess yes, but have neither seen it specifically documented
> nor tested it myself, nor read of anyone else actually testing it.
>
> >
> > I find it more helpful to think of targets as just units that don't do
> > anything.  We don't use them in openrc but I suspect it would work out
> > of the box, and maybe we should even consider doing it in at least some
> > cases.  For example, right now /etc/init.d/samba uses some scripting to
> > launch both nmbd/smbd and fancy config file parsing 

[gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Duncan]

Rich Freeman posted on Thu, 18 Feb 2016 07:22:36 -0500 as excerpted:

> 4.  In the runlevel paradigm you usually think of services running
> inside a runlevel (perhaps this isn't strictly true, but most people
> think this way, in part because runlevels don't change much).  In
> systemd this really isn't the case.  Services run before targets, or
> after them.  A target won't be considered running if anything it depends
> on isn't running.

Some minor additional notes, with the first one being here.

Systemd target units are analogous to edge-triggered interrupts, which 
they resemble in that they are simply "synchronization points" (the term 
used in the systemd.target (5) manpage itself).  Level-triggered 
interrupts can be held on or held off (high or low), but edge-triggered a 
re simply events that occur and then are passed as time moves on.  As 
such, targets can be started, but not normally (while the job queue is 
idle) stopped, as they de-assert as soon as they are actually reached, 
tho many of their requirements generally continue to run until stopped by 
some other event, often conflicts= against some other target or general 
unit being started, or specific admin systemctl stop.

Tho the systemd FAQ suggests this wasn't always so, as it suggests using 
systemctl list-units --type=target in answer to a question about how to 
find what "runlevel" you're in.  That command seems to return nothing, 
here, tho, at least while no target is actively starting, so it would 
seem that answer's a bit dated.

It can be noted, however, that certain units, most often specific targets 
intended to be specifically invokable by users, can be "isolated", as 
they have the AllowIsolate unit setting.  Systemctl isolate  will 
then cause it to be started exclusively, stopping anything that's not a 
dependency of that unit.  The systemctl emergency, rescue, reboot, 
shutdown, etc, commands, then become effectively shortcuts to the longer 
systemctl isolate  command form.

> 5.  I'd have to check, but I wouldn't be surprised if systemd doesn't
> actually require specifying a target at all.  Your default "runlevel"
> could be apache2.service, which means the system would boot and launch
> everything necessary to get apache working, and it probably wouldn't
> even spawn a getty.  This is NOT analogous to just putting only apache2
> in /etc/runlevels/default, because in that example openrc is running the
> default runlevel, and it only pulls in apache2.  Systemd is purely
> dependency driven and when you tell it to make graphical.target the
> default runlevel it is like running emerge kde-meta.  If all you wanted
> was kde-runtime you wouldn't redefine kde-meta to pull in only
> kde-runtime, you'd just run emerge kde-runtime.  Again, I haven't tested
> this, but I'd be shocked if it didn't work.  Of course, specifying a
> service as a default instead of a target is very limiting, but it would
> probably work.  Heck, you could probably specify a mount as the default
> and the system would just boot and mount something and sit there.  Or
> you could make it a socket and all it would do is sit and listen for a
> connection inetd-style.

As mentioned both in the systemd FAQ and in the systemd.special (7) 
manpage, under default.target, this is the default unit started at 
bootup.  Normally, it'll be a symlink to either multi-user.target 
(analogous to sysvinit semi-standard runlevel 3, CLI, no X), or 
graphical.target (analogous to sysvinit semi-standard runlevel 5, 
launching X and and a graphical *DM login).

I don't see specific documentation of whether symlinking to a non-target 
unit is allowed, but systemd does have a commandline option --unit=, 
which is explicitly documented to take a _unit_, default.target being the 
default, but other non-target units being possible as well.  Presumably 
systemd would examine said unit, looking for DefaultDependencies=no, and 
if not specifically set, would start the early "system level" targets, 
before starting the named unit in place of the normal default.target.

So it's definitely possible to do via systemd commandline, but I'm not 
sure if default.target is followed if it doesn't symlink a target unit, 
or not.  I'd guess yes, but have neither seen it specifically documented 
nor tested it myself, nor read of anyone else actually testing it.

> 
> I find it more helpful to think of targets as just units that don't do
> anything.  We don't use them in openrc but I suspect it would work out
> of the box, and maybe we should even consider doing it in at least some
> cases.  For example, right now /etc/init.d/samba uses some scripting to
> launch both nmbd/smbd and fancy config file parsing to let the users
> control which ones launch.  You could instead break that into three
> files - smbd, nmbd, and samba.  The first two would launch one daemon
> each, and the samba init.d script wouldn't actually launch anything, but
> would just depend on the others.  That would be the 

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Rich Freeman]

On Thu, Feb 18, 2016 at 3:57 AM, Duncan <1i5t5.dun...@cox.net> wrote:
> Rich Freeman posted on Wed, 17 Feb 2016 08:46:34 -0500 as excerpted:
>
>> When systemd says "target" - think "virtual service."  The equivalent in
>> openrc would be an init.d script that has dependencies but which doesn't
>> actually launch any processes.
>>
>> Targets also take the place of runlevels in systemd.
>
> The systemd official comparison of targets is to runlevels, except much
> more flexible as it's actually possible for multiple targets to be in the
> process of being reached at once, not services or "virtual services", and
> indeed, my immediate internal reaction at seeing the "virtual services"
> definition was "no, they're like runlevels", before I even reached the
> next paragraph, where you add that.
>
> Basically, I'd put the runlevel comparison first and primary, as systemd
> documentation does, tho now that I've seen the usage, "virtual services"
> /does/ add some richness to the definition, helping to accent the fact
> that multiple targets can be processed at once.  So it's a difference in
> emphasis, while agreeing in general.

So, perhaps this is getting a bit off-topic, and a bit theoretical.

I've found that in general you aren't going to be effective with
systemd until you grok some of its basic concepts, and this is one of
them.  And, frankly, I've never found the "target = runlevel" analogy
very helpful when it comes to understanding just what systemd is
doing.

It is true that systemd uses targets to accomplish what other
solutions use runlevels for.  However, runlevels are used for things
that have nothing to do with traditional runlevels.

Here are a few cases where the runlevel analogy fails:

1.  In most service manager / rc implementations the system is at only
one runlevel at any time (of course it might be transitioning between
two).  In systemd targets can be started independent of the overall
"runlevel" and targets that are dependencies of the main runlevel can
be reached in parallel at any order semi-independently.

2.  Targets are used for many intermediate states that most sane rc
implementations would never make a separate runlevel, such as gettys
running, local filesystems mounted, remote filesystems mounted,
network interfaces exist, network is online, name resolution working,
swap running, etc.  These really don't work like runlevels, and there
is no strict sequential order that these get loaded in.

3.  Targets can be used for convenience just as virtual packages are
in most distros.  For example, I have an nfs-client target (which just
runs whatever daemons are needed to mount remote nfs shares).  While
upstream doesn't do it this way you could define a samba target that
starts both nmbd and smbd.  I use targets to group services that get
launched together by cron jobs, and so on.

4.  In the runlevel paradigm you usually think of services running
inside a runlevel (perhaps this isn't strictly true, but most people
think this way, in part because runlevels don't change much).  In
systemd this really isn't the case.  Services run before targets, or
after them.  A target won't be considered running if anything it
depends on isn't running.

5.  I'd have to check, but I wouldn't be surprised if systemd doesn't
actually require specifying a target at all.  Your default "runlevel"
could be apache2.service, which means the system would boot and launch
everything necessary to get apache working, and it probably wouldn't
even spawn a getty.  This is NOT analogous to just putting only
apache2 in /etc/runlevels/default, because in that example openrc is
running the default runlevel, and it only pulls in apache2.  Systemd
is purely dependency driven and when you tell it to make
graphical.target the default runlevel it is like running emerge
kde-meta.  If all you wanted was kde-runtime you wouldn't redefine
kde-meta to pull in only kde-runtime, you'd just run emerge
kde-runtime.  Again, I haven't tested this, but I'd be shocked if it
didn't work.  Of course, specifying a service as a default instead of
a target is very limiting, but it would probably work.  Heck, you
could probably specify a mount as the default and the system would
just boot and mount something and sit there.  Or you could make it a
socket and all it would do is sit and listen for a connection
inetd-style.


I find it more helpful to think of targets as just units that don't do
anything.  We don't use them in openrc but I suspect it would work out
of the box, and maybe we should even consider doing it in at least
some cases.  For example, right now /etc/init.d/samba uses some
scripting to launch both nmbd/smbd and fancy config file parsing to
let the users control which ones launch.  You could instead break that
into three files - smbd, nmbd, and samba.  The first two would launch
one daemon each, and the samba init.d script wouldn't actually launch
anything, but would just depend on the others.  That would be the
systemd target 

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Rich Freeman]

On Wed, Feb 17, 2016 at 10:11 PM, Richard Yao  wrote:
>
> dracut does not assist those who do not want generic kernel
> configurations. Unfortunately, the handbook does not do a good job in
> saying that the initramfs generation and generic kernel configurations
> are optional.
>

No argument that this section of the handbook couldn't be improved.
Generic kernel configs are actually not a bad starting point, though
the reality is that you don't really need a fancy tool to have them.
Just stick some config files on a webpage and tell users to download
them to .config and run make oldconfig.

Heck, you can just zcat /proc/config.gz from the install CD most
likely, though you'll want to edit out the initramfs filename in the
config if you do this or it won't build.

Dracut works fine with just about any kernel configuration.  It
doesn't really have anything to do with configuring the kernel.  You
run it after you've built and installed your kernel.  I didn't want to
split this reply up too much but I'm not sure what you meant in your
other email about it not configuring the bootloader - with grub2 the
bootloader generally configures itself.

>
> There is no default and system boot without an initramfs not only works,
> but is advisable for faster boot unless something fancy is being done
> that needs it.
>

Booting without an initramfs is supported on Gentoo as long as you
don't have /usr on a separate partition.  If you do have /usr on a
separate partition this is considered an unsupported configuration
(unless you're using some other mechanism to get /usr mounted early in
boot).

However, even if it is optional I'd still recommend it.  I'll grant
that it will add a few seconds to the boot process in many cases, but
it is far more robust than booting without one.  In general the kernel
developers have been moving more towards depending more on userspace
for initialization where in the past the kernel did more to autodetect
and configure userspace.  I think there is recognition that putting
this functionality in userspace gives everybody more flexibility in
initializing the system.

One big benefit of using an initramfs is more robust root detection.
They typically support labels and uuids for identifying a root
filesystem, while the kernel itself only supports device names.
Before I was using an initramfs I can't tell you how many times I'd
add a hard drive and have the kernel detect them in a different order,
causing my boot to break.  With just one drive this is of course a
lower risk.

Of course once you start making your environment more complex (RAID,
LUKS, LVM, zfs/btrfs with multiple disks, network filesystems, etc) an
initramfs can easily become required.  I tend to just use them
routinely - it is trivial to do and they rarely cause problems.  I'll
admit that on a VM with one partition they're probably overkill.

I guess one area where they add a bit of complexity is if you're using
UEFI without some kind of intermediate bootloader, since a kernel can
be directly booted with EFI and using an initramfs in this
configuration is a bit of a pain in the rear.  It certainly can be
done but it becomes a bit of a chicken/egg putting it all together
(you want the modules in the initramfs, but the initramfs has to be
built before the final kernel itself).

> Claiming to pick a default between genkernel and dracut when both are
> optional makes no sense, especially since dracut's capabilities
> (initramfs generation) are a subset of genkernel's (initramfs generation
> and kernel builds). dracut could replace genkernel's initramfs
> generation capabilities, but it simply cannot replace genkernel for
> building a generic kernel. It was never intended to do that.

So, again the page could probably be cleaned up in how it presents all
of this.  I'll agree that both should stay optional, but I'd suggest
listing dracut first for initramfs generation.  That is the only sense
that I'd make it a "default."

> By the way, pver the course of time, there have been things genkernel
> did better and things dracut did better.

That doesn't surprise me - dracut is newew, and as something new I'm
sure there was a point in time where it didn't do everything more
established solutions already did.

> It is unlikely one will ever be superior to the other.

This is pretty speculative, and any opinion I offer is only going to
be more speculation.  However, I'm not aware of anything a genkernel
initramfs does that dracut doesn't already handle.  It has the
advantage of being a more generic tool, and thus likely to have a
broader development community.

I guess I'm a fan of cross-distro solutions in general, but it would
be fair to say that the fact that something is broadly used doesn't
automatically make it better.  The reality is that Gentoo's approach
to package management is completely different from how almost
everybody does things but we wouldn't be here if we didn't like it.

> However, some feedback on what 

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Rich Freeman]

On Wed, Feb 17, 2016 at 10:02 PM, Richard Yao  wrote:
>
> Dracut handling it well is not up for dispute. When I checked last year,
> dracut simply did not tell systemd to use this functionality because it
> was unnecessary functionality that only served to slowed down the
> shutdown process. It only enables it when a driver indicates an actual
> need, which is the way that it should be.

You might want to check again.  As far as I can tell systemd has been
pivoting back to dracut for a while now by default.

-- 
Rich

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[James Le Cuirot]

On Wed, 17 Feb 2016 22:11:42 -0500
Richard Yao  wrote:

> dracut does not assist those who do not want generic kernel
> configurations. Unfortunately, the handbook does not do a good job in
> saying that the initramfs generation and generic kernel configurations
> are optional.

Does Dracut's hostonly mode not count? I think this is even the
default as I don't specify it here and I had to temporarily force
non-hostonly mode on Fedora in order to fix a broken system. I
certainly keep my Gentoo kernel configuration to a minimum.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer


pgpHzJLhLKgbr.pgp
Description: OpenPGP digital signature

[gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Duncan]

Rich Freeman posted on Wed, 17 Feb 2016 08:46:34 -0500 as excerpted:

> On Tue, Feb 16, 2016 at 9:20 PM, Duncan <1i5t5.dun...@cox.net> wrote:
>>
>> Initial shutdown is via two targets (as opposed to specific services),
> 
> Since not everybody in this thread may be familiar with systemd, I'll
> just add a quick definition.

Thanks for this followup. =:^)

> When systemd says "target" - think "virtual service."  The equivalent in
> openrc would be an init.d script that has dependencies but which doesn't
> actually launch any processes.
> 
> Targets also take the place of runlevels in systemd.

The systemd official comparison of targets is to runlevels, except much 
more flexible as it's actually possible for multiple targets to be in the 
process of being reached at once, not services or "virtual services", and 
indeed, my immediate internal reaction at seeing the "virtual services" 
definition was "no, they're like runlevels", before I even reached the 
next paragraph, where you add that.

Basically, I'd put the runlevel comparison first and primary, as systemd 
documentation does, tho now that I've seen the usage, "virtual services" 
/does/ add some richness to the definition, helping to accent the fact 
that multiple targets can be processed at once.  So it's a difference in 
emphasis, while agreeing in general.

> Just as with runlevels in openrc they are used to synchronize milestones
> during bootup, but the design is much more generic. Presumably openrc
> hard-codes runlevels like sysinit, boot, and default.  In systemd I
> believe it just looks at the config file and directly launches the
> desired target, and then the dependency chain for that pulls in all the
> targets that precede it.  Targets can depend on other targets, and
> services can depend on previous targets.
> 
> The other dimension is that unit files describe what target they are
> typically associated with if it isn't the default.  So, you don't run
> into the sorts of situations you have with openrc that if you want to
> enable mdraid support you need to remember to add it to the boot
> runlevel.  That might be a relatively-easy thing to add support for in
> openrc actually.
> 
> Hopefully that makes Dunan's summary easier to read for anybody who
> doesn't speak systemdese.

Again, thanks. =:^)

> Another bit of background that might be helpful is that systemd also
> manages mounts directly.

After actually looking into how systemd processes mounts as I wrote the 
earlier post, I believe it's safe to say this is a much bigger difference 
than it might seem on the surface, and it was certainly one of the 
biggest challenges I faced in attempting to explain systemd's mount 
processing in sysvinit/openrc terms.

Even with a reasonable level of personal admin-level systemd familiarity, 
I kept expecting and looking for systemd to hand off the unmounting to a 
service, which I expected would simply call one of the separate from 
systemd itself executables (for example journald, networkd, etc, I was 
expecting a mount helper or mountd as well), only after seeing the 
shutdown flowchart and looking into each individual component, I still 
couldn't find it, which is why I ultimately had to go diving directly 
into the systemd sources themselves, which is when I found what I was 
looking for! =:^)

But in hindsight, while there are various non-systemd-binary executables 
that ship with systemd, systemd itself directly processes all unit files, 
including mount units, so I should have known that it'd handle umounting 
directly.  But I simply didn't make that connection, until it became 
obvious after the fact, and indeed, it didn't actually solidify in my 
mind until just now as I wrote about it, tho certainly, after writing the 
earlier post, all the pieces were there to be assembled, and this simply 
triggered it. =:^)

> It parses fstab and creates a network of mount
> units with appropriate dependencies.  Whether you unmount /var/tmp using
> "umount /var/tmp" or "systemctl stop var-tmp.mount" systemd updates the
> status of the var-tmp.mount unit to a stopped status.  I believe if you
> add a noauto line to fstab it will create a mount unit automatically and
> not start it, and if you made it mount on boot I think it would actually
> be mounted as soon as you save the file in your editor (systemd can
> monitor config files for changes - all of this is governed by
> scripts/software called generators).

I'm not sure of the last -- systemd does let you mount and umount at will
[1], and there are enough cases where an admin might setup the fstab 
entry before he's actually prepared for the mount to go ahead, that I 
don't believe it would be wise for systemd to try to jump the gun.

OTOH, it's possible to setup corresponding *.automount units as well, 
with the purpose there being to trigger mounting (using the kernel's 
automount services, which of course would have to be enabled -- they 
aren't here) as soon as something tries to stat/open 

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Richard Yao]

On 02/17/2016 01:32 PM, Rich Freeman wrote:
> On Wed, Feb 17, 2016 at 1:06 PM, Ian Stakenvicius  wrote:
>>
>> Genkernel's initramfs generation was what we endorsed for the most
>> part, until dracut came around.  it's hard to say what "most" are
>> doing but i expect dracut and genkernel based initramfs's make up
>> the vast majority in use by gentoo users, with a small minority
>> rolling their own through other means.
>>
> 
> While I personally endorse dracut over genkernel, the reality is that
> only genkernel is actually documented in the handbook.  This is due at
> least in part to laziness on my part as I've been meaning to add it
> since forever.
> 
> Likewise I intend to update the handbook to make selection of
> openrc/systemd less convoluted as well.  The current handbook does
> offer systemd as an option but then basically refers you out to
> another page that doesn't follow the same flow as the handbook.
> 
> In my notes I've found that it is a pretty trivial change to pick one
> or the other actually if you do it at the right time, so this could be
> added to the handbook with very little disruption to the flow for
> non-systemd users.  I imagine other service managers would be similar,
> or even simpler.  I found that switching between the two only requires
> two changes - one is to pick a systemd profile relatively early in the
> process before doing a world update, and then changing one line in
> your grub config at the end.  If you emerge world after you do most of
> your system configuration systemd will automatically pick up all the
> openrc configuration and use it, which as a bonus leaves you with a
> system that is easy to boot in either mode.
> 
> Getting back to dracut - it is really just a few lines added as an
> alternative to the initramfs section.  After you build your kernel it
> is really just a one-liner, and grub2-mkconfig picks up on it
> automatically (as I imagine it probably does with genkernel as well).
> Unless you want to play with the configuration there isn't much fuss.

dracut does not assist those who do not want generic kernel
configurations. Unfortunately, the handbook does not do a good job in
saying that the initramfs generation and generic kernel configurations
are optional.

> I think we really should give strong consideration to recommending
> dracut as a default, while of course preserving the option of
> genkernel.  I'm certainly open to feedback if there is some use case
> where genkernel is better, but dracut is cross-distro, gives you
> options to easily maximize or minimize your config, and is really easy
> to tailor with modules.

There is no default and system boot without an initramfs not only works,
but is advisable for faster boot unless something fancy is being done
that needs it.

Claiming to pick a default between genkernel and dracut when both are
optional makes no sense, especially since dracut's capabilities
(initramfs generation) are a subset of genkernel's (initramfs generation
and kernel builds). dracut could replace genkernel's initramfs
generation capabilities, but it simply cannot replace genkernel for
building a generic kernel. It was never intended to do that.

By the way, pver the course of time, there have been things genkernel
did better and things dracut did better. It is unlikely one will ever be
superior to the other. However, some feedback on what genkernel does
poorly versus dracut and could therefore improve would be helpful.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Richard Yao]

On 02/17/2016 12:19 PM, Rich Freeman wrote:
> On Wed, Feb 17, 2016 at 9:24 AM, Richard Yao  wrote:
>> Systemd installs that go back into the initramfs at shutdown are rare 
>> because there is a
>> hook for the initramfs to tell systemd that it should re-exec it and very 
>> few configurations
>> do that. Even fewer that do it actually need it.
> 
> While I won't debate that it probably isn't strictly essential, dracut
> handles unmounting root for systemd just fine (well, at least on
> non-nfs - the version I'm using with an nfs root struggles in this
> regard, though unclean shutdown on nfs with no files open probably
> isn't really a problem).

Dracut handling it well is not up for dispute. When I checked last year,
dracut simply did not tell systemd to use this functionality because it
was unnecessary functionality that only served to slowed down the
shutdown process. It only enables it when a driver indicates an actual
need, which is the way that it should be.

> Is dracut still not widely used?  I know that it was all the fashion
> for a decade or two for every distro to build their own initramfs, but
> I don't get why anybody wouldn't just make the switch - it is far more
> capable and configurable.

Not many Gentoo users use dracut. It does not handle kernel compilation
or bootloader configuration. It is definite ahead of genkernel in
networking though, but there is not much demand for that among users.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Daniel Campbell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 02/17/2016 09:30 AM, James Le Cuirot wrote:
> On Wed, 17 Feb 2016 12:19:52 -0500 Rich Freeman 
> wrote:
> 
>> Is dracut still not widely used?  I know that it was all the
>> fashion for a decade or two for every distro to build their own
>> initramfs, but I don't get why anybody wouldn't just make the
>> switch - it is far more capable and configurable.
> 
> Does anyone know what most Gentoo users are doing these days? I
> don't recall the handbook mentioning initramfs at all back in 2002
> because it wasn't really needed back then. I did without for years
> until I finally put / on LVM. I used lvm2create_initrd for a while
> but that was still quite a manual process and I couldn't imagine
> going back to it now. I've switched to Dracut and it's great but I
> don't get the impression that Gentoo really endorses that option
> over the more laborious ones. Maybe it should?
> 
> https://wiki.gentoo.org/wiki/Initramfs
> 
I went without an initrd until I encrypted my / and put LVM inside it.
I used genkernel's initrd functions for that with a
manually-configured kernel.

- -- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCAAGBQJWxOssAAoJEAEkDpRQOeFwtDUQANLipE7ZGNjf8lrstbl6/6GJ
bEwrQX+xFST5c7U9OJgU1HrGmL7vrYLurg82Sqf2zqzlsXdSPAxQtvAPETvPW8Gy
lbWdV1UaaCda5W53lX2XrKVf9xQgT8wANlgWQmFlsOnbCUbaLYYdQQWN/yuW+NQL
xV2V57EUOLCmQ4LNNG/zuoo7JrrfVikNavlk70Emq0DNpt+dPhHK9hc6N6zAeUc2
6oEFDB6ILf63ZE0v1dLOc+HxnIeAZ06GwLnx+1ta6o2hfoqPDBqym2V6lplRk1KF
xQeWllMNcl5ez3d+T8BP6sm0gkv0WPpVAMBm58IdxhX0ITqIVDo/A8zR+tp7x5U+
Ih5+2vjhoFmpr+8NVo2HHDHCddokzcqgd6ZL4ZA5RYYLF4JC755QntTGTKuGdU1e
nVYySqTnmzxXIzEaplixesgKiG70KBDfStmLGjIbxl+E8fyZkx6fNg1dP7/QMZUY
GF/HiO5dpEynDvsO1VgJY6cbFbHW0Ee1AWhZmvpT5eOZTUtCiipaHwGrS6Tds3N2
R4odNpPBvnV9G6EvGrQ6rtAcdH+j2XsVcFsaJZT+YOcvIZBT6UEg3YcucTNJELRE
OSJzROeVau4p82o5YJTqwPAqvu9TI39kJ13ps8pTqJuNrj+A33ag3jiPh8Azpfbo
APYj1uIIrl6OcOinlHt8
=0SBA
-END PGP SIGNATURE-

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Rich Freeman]

On Wed, Feb 17, 2016 at 1:06 PM, Ian Stakenvicius  wrote:
>
> Genkernel's initramfs generation was what we endorsed for the most
> part, until dracut came around.  it's hard to say what "most" are
> doing but i expect dracut and genkernel based initramfs's make up
> the vast majority in use by gentoo users, with a small minority
> rolling their own through other means.
>

While I personally endorse dracut over genkernel, the reality is that
only genkernel is actually documented in the handbook.  This is due at
least in part to laziness on my part as I've been meaning to add it
since forever.

Likewise I intend to update the handbook to make selection of
openrc/systemd less convoluted as well.  The current handbook does
offer systemd as an option but then basically refers you out to
another page that doesn't follow the same flow as the handbook.

In my notes I've found that it is a pretty trivial change to pick one
or the other actually if you do it at the right time, so this could be
added to the handbook with very little disruption to the flow for
non-systemd users.  I imagine other service managers would be similar,
or even simpler.  I found that switching between the two only requires
two changes - one is to pick a systemd profile relatively early in the
process before doing a world update, and then changing one line in
your grub config at the end.  If you emerge world after you do most of
your system configuration systemd will automatically pick up all the
openrc configuration and use it, which as a bonus leaves you with a
system that is easy to boot in either mode.

Getting back to dracut - it is really just a few lines added as an
alternative to the initramfs section.  After you build your kernel it
is really just a one-liner, and grub2-mkconfig picks up on it
automatically (as I imagine it probably does with genkernel as well).
Unless you want to play with the configuration there isn't much fuss.

I think we really should give strong consideration to recommending
dracut as a default, while of course preserving the option of
genkernel.  I'm certainly open to feedback if there is some use case
where genkernel is better, but dracut is cross-distro, gives you
options to easily maximize or minimize your config, and is really easy
to tailor with modules.

-- 
Rich

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Ian Stakenvicius]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 17/02/16 12:30 PM, James Le Cuirot wrote:
> On Wed, 17 Feb 2016 12:19:52 -0500 Rich Freeman
>  wrote:
> 
>> Is dracut still not widely used?  I know that it was all the
>> fashion for a decade or two for every distro to build their own
>> initramfs, but I don't get why anybody wouldn't just make the
>> switch - it is far more capable and configurable.
> 
> Does anyone know what most Gentoo users are doing these days? I
> don't recall the handbook mentioning initramfs at all back in
> 2002 because it wasn't really needed back then. I did without for
> years until I finally put / on LVM. I used lvm2create_initrd for
> a while but that was still quite a manual process and I couldn't
> imagine going back to it now. I've switched to Dracut and it's
> great but I don't get the impression that Gentoo really endorses
> that option over the more laborious ones. Maybe it should?
> 
> https://wiki.gentoo.org/wiki/Initramfs
> 

Genkernel's initramfs generation was what we endorsed for the most
part, until dracut came around.  it's hard to say what "most" are
doing but i expect dracut and genkernel based initramfs's make up
the vast majority in use by gentoo users, with a small minority
rolling their own through other means.


-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iF4EAREIAAYFAlbEtr0ACgkQAJxUfCtlWe3/xwD+P4OlCY0RuTdGdPWOGRd8ePLi
EvZCZuyp2oxPdYt6xdAA/2CN/Nbgj4bVFa02KeVpuoNOHMErPU4meZfzUjNbGmTH
=iUu6
-END PGP SIGNATURE-

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[James Le Cuirot]

On Wed, 17 Feb 2016 12:19:52 -0500
Rich Freeman  wrote:

> Is dracut still not widely used?  I know that it was all the fashion
> for a decade or two for every distro to build their own initramfs, but
> I don't get why anybody wouldn't just make the switch - it is far more
> capable and configurable.

Does anyone know what most Gentoo users are doing these days? I don't
recall the handbook mentioning initramfs at all back in 2002 because it
wasn't really needed back then. I did without for years until I finally
put / on LVM. I used lvm2create_initrd for a while but that was still
quite a manual process and I couldn't imagine going back to it now.
I've switched to Dracut and it's great but I don't get the impression
that Gentoo really endorses that option over the more laborious ones.
Maybe it should?

https://wiki.gentoo.org/wiki/Initramfs

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Rich Freeman]

On Wed, Feb 17, 2016 at 9:24 AM, Richard Yao  wrote:
> Systemd installs that go back into the initramfs at shutdown are rare because 
> there is a
> hook for the initramfs to tell systemd that it should re-exec it and very few 
> configurations
> do that. Even fewer that do it actually need it.

While I won't debate that it probably isn't strictly essential, dracut
handles unmounting root for systemd just fine (well, at least on
non-nfs - the version I'm using with an nfs root struggles in this
regard, though unclean shutdown on nfs with no files open probably
isn't really a problem).

Is dracut still not widely used?  I know that it was all the fashion
for a decade or two for every distro to build their own initramfs, but
I don't get why anybody wouldn't just make the switch - it is far more
capable and configurable.


-- 
Rich

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Richard Yao]

> On Feb 16, 2016, at 9:20 PM, Duncan <1i5t5.dun...@cox.net> wrote:
> 
> William Hubbs posted on Tue, 16 Feb 2016 12:41:29 -0600 as excerpted:
> 
>> What I'm trying to figure out is, what to do about re-mounting file
>> systems read-only.
>> 
>> How does systemd do this? I didn't find an equivalent of the mount-ro
>> service there.
> 
> For quite some time now, systemd has actually had a mechanism whereby the 
> main systemd process reexecs (with a pivot-root) the initr* systemd and 
> returns control to it during the shutdown process, thereby allowing a 
> more controlled shutdown than traditional init systems because the final 
> stages are actually running from the virtual-filesystem of the initr*, 
> such that after everything running on the main root is shutdown, the main 
> root itself can actually be unmounted, not just mounted read-only, 
> because there is literally nothing running on it any longer.
> 
> There's still a fallback to read-only mounting if an initr* isn't used or 
> if reinvoking the initr* version fails for some reason, but with an 
> initr*, when everything's working properly, while there are still some 
> bits of userspace running, they're no longer actually running off of the 
> main root, so main root can actually be unmounted much like any other 
> filesystem.

Systemd installs that go back into the initramfs at shutdown are rare because 
there is a hook for the initramfs to tell systemd that it should re-exec it and 
very few configurations do that. Even fewer that do it actually need it.

The biggest user of that mechanism of which I am aware is ZFS on EL/Fedora when 
booted with Dracut. It does not need it and it was only implemented was that 
someone who did not understand how ZFS was designed to integrate with the boot 
and startup processes thought it was a good idea.

As it turns out, that behavior actually breaks the mechanism intended to make 
multipath sane by marking the pool in such a way that it tells all systems with 
access to the disks that a pool that will be used on next boot is not going to 
be used by anyone. If they import it and the system boots, the pool can be 
damaged beyond repair.

Thankfully, no one seems to boot EL/Fedora systems off ZFS pools in multipath 
environments. The code to hook into this special behavior will be removed in 
the future, but that is a low priority as none of the developers' employers 
care about it and the almost negligible possibility that the mechanism would 
save someone from data loss  has made it too low of a priority for any of us to 
spend our free time on it.

> The process is explained a bit better in the copious blogposted systemd 
> documentation.  Let's see if I can find a link...
> 
> OK, this isn't where I originally read about it, which IIRC was aimed 
> more at admins, while this is aimed at initr* devs, but that's probably a 
> good thing as it includes more specific detail...
> 
> https://www.freedesktop.org/wiki/Software/systemd/InitrdInterface/
> 
> And here's some more, this time in the storage daemon controlled root and 
> initr* context...
> 
> https://www.freedesktop.org/wiki/Software/systemd/RootStorageDaemons/
> 
> 
> But... all that doesn't answer the original question directly, does it?  
> Where there's no return to initr*, how /does/ systemd handle read-only 
> mounting?
> 
> First, the nice ascii-diagram flow charts in the bootup (7) manpage may 
> be useful, in particular here, the shutdown diagram (tho IDK if you can 
> find such things useful or not??).
> 
> https://www.freedesktop.org/software/systemd/man/bootup.html
> 
> Here's the shutdown diagram described in words:
> 
> Initial shutdown is via two targets (as opposed to specific services), 
> shutdown.target, which conflicts with all (normal) system services 
> thereby shutting them down, and umount.target, which conflicts with file 
> mounts, swaps, cryptsetup device, etc.  Here, we're obviously interested 
> in umount.target.  Then after those two targets are reached, various low 
> level services are run or stopped, in ordered to reach final.target.  
> After final.target, the appropriate systemd-(reboot|poweroff|halt|kexec) 
> service is run, to hit the ultimate (reboot|poweroff|halt|kexec).target, 
> which of course is never actually evaluated, since the service actually 
> does the intended action.
> 
> The primary takeaway is that you might not be finding a specific systemd 
> remount-ro service, because it might be a target, defined in terms of 
> conflicts with mount units, etc, rather than a specific service.
> 
> Neither shutdown.target nor umount.target have any wants or requires by 
> default, but the various normal services and mount units conflict with 
> them, either via default or specifically, so are shut down before the 
> target can be reached.
> 
> final.target has the After=shutdown.target umount.target setting, so 
> won't be reached until they are reached.
> 
> The respective 

Re: [gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Rich Freeman]

On Tue, Feb 16, 2016 at 9:20 PM, Duncan <1i5t5.dun...@cox.net> wrote:
>
> Initial shutdown is via two targets (as opposed to specific services),

Since not everybody in this thread may be familiar with systemd, I'll
just add a quick definition.

When systemd says "target" - think "virtual service."  The equivalent
in openrc would be an init.d script that has dependencies but which
doesn't actually launch any processes.

Targets also take the place of runlevels in systemd.

Just as with runlevels in openrc they are used to synchronize
milestones during bootup, but the design is much more generic.
Presumably openrc hard-codes runlevels like sysinit, boot, and
default.  In systemd I believe it just looks at the config file and
directly launches the desired target, and then the dependency chain
for that pulls in all the targets that precede it.  Targets can depend
on other targets, and services can depend on previous targets.

The other dimension is that unit files describe what target they are
typically associated with if it isn't the default.  So, you don't run
into the sorts of situations you have with openrc that if you want to
enable mdraid support you need to remember to add it to the boot
runlevel.  That might be a relatively-easy thing to add support for in
openrc actually.

Hopefully that makes Dunan's summary easier to read for anybody who
doesn't speak systemdese.

Another bit of background that might be helpful is that systemd also
manages mounts directly.  It parses fstab and creates a network of
mount units with appropriate dependencies.  Whether you unmount
/var/tmp using "umount /var/tmp" or "systemctl stop var-tmp.mount"
systemd updates the status of the var-tmp.mount unit to a stopped
status.  I believe if you add a noauto line to fstab it will create a
mount unit automatically and not start it, and if you made it mount on
boot I think it would actually be mounted as soon as you save the file
in your editor (systemd can monitor config files for changes - all of
this is governed by scripts/software called generators).

So, systemd in general tries to stay aware of the state of mounts.  I
suspect that isn't just firing off a script to find/unmount anything
that is mounted.  From Dunan's email it sounds like you could create a
mount unit and explicitly not set a conflict with the unmount target
which would cause the filesystem to remain mounted at shutdown.  I
have no idea what that would do to unmounting the root.

-- 
Rich

[gentoo-dev] Re: rfc: Does OpenRC really need mount-ro

[Duncan]

William Hubbs posted on Tue, 16 Feb 2016 12:41:29 -0600 as excerpted:

> What I'm trying to figure out is, what to do about re-mounting file
> systems read-only.
> 
> How does systemd do this? I didn't find an equivalent of the mount-ro
> service there.

For quite some time now, systemd has actually had a mechanism whereby the 
main systemd process reexecs (with a pivot-root) the initr* systemd and 
returns control to it during the shutdown process, thereby allowing a 
more controlled shutdown than traditional init systems because the final 
stages are actually running from the virtual-filesystem of the initr*, 
such that after everything running on the main root is shutdown, the main 
root itself can actually be unmounted, not just mounted read-only, 
because there is literally nothing running on it any longer.

There's still a fallback to read-only mounting if an initr* isn't used or 
if reinvoking the initr* version fails for some reason, but with an 
initr*, when everything's working properly, while there are still some 
bits of userspace running, they're no longer actually running off of the 
main root, so main root can actually be unmounted much like any other 
filesystem.

The process is explained a bit better in the copious blogposted systemd 
documentation.  Let's see if I can find a link...

OK, this isn't where I originally read about it, which IIRC was aimed 
more at admins, while this is aimed at initr* devs, but that's probably a 
good thing as it includes more specific detail...

https://www.freedesktop.org/wiki/Software/systemd/InitrdInterface/

And here's some more, this time in the storage daemon controlled root and 
initr* context...

https://www.freedesktop.org/wiki/Software/systemd/RootStorageDaemons/


But... all that doesn't answer the original question directly, does it?  
Where there's no return to initr*, how /does/ systemd handle read-only 
mounting?

First, the nice ascii-diagram flow charts in the bootup (7) manpage may 
be useful, in particular here, the shutdown diagram (tho IDK if you can 
find such things useful or not??).

https://www.freedesktop.org/software/systemd/man/bootup.html

Here's the shutdown diagram described in words:

Initial shutdown is via two targets (as opposed to specific services), 
shutdown.target, which conflicts with all (normal) system services 
thereby shutting them down, and umount.target, which conflicts with file 
mounts, swaps, cryptsetup device, etc.  Here, we're obviously interested 
in umount.target.  Then after those two targets are reached, various low 
level services are run or stopped, in ordered to reach final.target.  
After final.target, the appropriate systemd-(reboot|poweroff|halt|kexec) 
service is run, to hit the ultimate (reboot|poweroff|halt|kexec).target, 
which of course is never actually evaluated, since the service actually 
does the intended action.

The primary takeaway is that you might not be finding a specific systemd 
remount-ro service, because it might be a target, defined in terms of 
conflicts with mount units, etc, rather than a specific service.

Neither shutdown.target nor umount.target have any wants or requires by 
default, but the various normal services and mount units conflict with 
them, either via default or specifically, so are shut down before the 
target can be reached.

final.target has the After=shutdown.target umount.target setting, so 
won't be reached until they are reached.

The respective (reboot|poweroff|halt|kexec).target units Requires= and 
After= their respective systemd-*.service units, and reboot and poweroff 
(but not halt and kexec) have 30-minute timeouts after which they run 
reboot-force or poweroff-force, respectively.

The respective systemd-(reboot|poweroff|halt|kexec).service units 
Requires= and After= shutdown.target, umount.target and final.target, all 
three, so won't be run until those complete.  They simply 
ExecStart=/usr/bin/systemctl --force their respective actions.

And here's what the systemd.special (7) manpage says about umount.target:

  umount.target
A special target unit that umounts all mount and automount points
on system shutdown.

Mounts that shall be unmounted on system shutdown shall add
Conflicts dependencies to this unit for their mount unit,
which is implicitly done when DefaultDependencies=yes is set
(the default).

But that /still/ doesn't reveal what actually does the remount-ro, as 
opposed to umount.  I don't see that either, at the unit level, nor do I 
see anything related to it in for instance my auto-generated from fstab 
/run/systemd/generators/-.mount file or in the systemd-fstab-generator 
(8) manpage.

Thus I must conclude that it's actually resolved in the mount-unit 
conflicts handling in systemd's source code, itself.

And indeed... in systemd's tarball, we see in src/core/umount.c, in 
mount_points_list_umount...

That the function actually remounts /everything/ (well, everything not in 
a container) read-only,