Re: [systemd-devel] systemd shutdown vs ostree

2013-09-17 Thread Lennart Poettering
On Sun, 15.09.13 17:26, Koen Kooi (k...@dominion.thruhere.net) wrote:

  On Sat, Sep 14, 2013 at 10:52 AM, Koen Kooi k...@dominion.thruhere.net 
  wrote:
  Please keep in mind that pstore is x86 only. EFI isn't x86 only, but 
  exceedingly rare outside of that arch.
  
  What? Pstore itself isn't. It's a generic interface to persistent
  platform storage. There are backends using EFI variables, NVRAM
  (PowerPC) or the APEI ERST (x86). As a last resort there's also a
  platform-independent RAM backend which may at least survive a simple
  reboot.
 
 OK, so x86 and ppc only. That will leaves out arm, mips and others.

I am pretty sure that it would be relatively easy to add support for an
addition ARM backend to pstore if such functionality is available on ARM
systems. The thing is simply that we should support pstore where it is
available, and get people to use that as the one and only API for these
kinds of things.

That all said, currently we cannot use pstore anyway, since there is no
way to tag the pstore data with the machine identity the data is
from. Which means that if you have a suse/Fedora dual-boot system there
is no way how we could make sure that the Suse instance doesn't end up
with Fedora's pstore logs in the journal and vice versa. 

To make pstore workable for us we'd need some scheme how we could echo
the machine ID into some virtual kernel file and then be sure that the
kernel adds that into the pstore data if something happens.

Lennart

-- 
Lennart Poettering - Red Hat, Inc.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-15 Thread Koen Kooi

Op 14 sep. 2013, om 11:04 heeft Jan Alexander Steffens jan.steff...@gmail.com 
het volgende geschreven:

 On Sat, Sep 14, 2013 at 10:52 AM, Koen Kooi k...@dominion.thruhere.net 
 wrote:
 Please keep in mind that pstore is x86 only. EFI isn't x86 only, but 
 exceedingly rare outside of that arch.
 
 What? Pstore itself isn't. It's a generic interface to persistent
 platform storage. There are backends using EFI variables, NVRAM
 (PowerPC) or the APEI ERST (x86). As a last resort there's also a
 platform-independent RAM backend which may at least survive a simple
 reboot.

OK, so x86 and ppc only. That will leaves out arm, mips and others.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-14 Thread Koen Kooi

Op 13 sep. 2013, om 00:31 heeft Jan Alexander Steffens jan.steff...@gmail.com 
het volgende geschreven:

 On Thu, Sep 12, 2013 at 10:51 PM, Lennart Poettering
 lenn...@poettering.net wrote:
 I think the really late shutdown logs should more liekly end up in some
 EFI var and then flushed out on next boot or so...
 
 That sounds good. Maybe use the pstore system? 

Please keep in mind that pstore is x86 only. EFI isn't x86 only, but 
exceedingly rare outside of that arch.

regards,

Koen
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-14 Thread Jan Alexander Steffens
On Sat, Sep 14, 2013 at 10:52 AM, Koen Kooi k...@dominion.thruhere.net wrote:
 Please keep in mind that pstore is x86 only. EFI isn't x86 only, but 
 exceedingly rare outside of that arch.

What? Pstore itself isn't. It's a generic interface to persistent
platform storage. There are backends using EFI variables, NVRAM
(PowerPC) or the APEI ERST (x86). As a last resort there's also a
platform-independent RAM backend which may at least survive a simple
reboot.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-13 Thread Colin Guthrie
'Twas brillig, and Lennart Poettering at 13/09/13 00:02 did gyre and gimble:
 On Thu, 12.09.13 23:31, Colin Guthrie (gm...@colin.guthr.ie) wrote:
 

 'Twas brillig, and Lennart Poettering at 12/09/13 21:51 did gyre and gimble:
 On Thu, 12.09.13 21:27, Colin Guthrie (gm...@colin.guthr.ie) wrote:

 So, as mentioned in the other thread, /usr should probably be on some
 OS resource blacklist or so, and not attempted to be shutdown.

 But unmounting /var during shutdown should actually work. The only thing
 that currently stops this from working cleanly is that journald is
 configured to stay around until the last moment, but currently has not
 interface to tell it to move its logging back to /run from /var. When we
 have that then the /var issue should go away for you, no?

 It would be quite nice to have an easy way for a compliant initrd to say
 it can and will handle /var unmounting (perhaps by dropping a
 var.mount.d/ snippet in the /run/systemd/system/ folder at boot?). 

 Yeah, such a drop-in should just work. Just add DefaultDependencies=no
 in there, and maybe instead Before=local-fs.target, and you should be set.

 Nice, I'm sure dracut could implement that then.

 This will allow us to get persistent logs in which the initrd can
 report it's progress until the last possible moment. This might help
 debugging any problems in this particular link of the shutdown chain.

 I think the really late shutdown logs should more liekly end up in some
 EFI var and then flushed out on next boot or so...

 Nice plan. How would be able to trust that data tho'. Would it have to
 be signed in some way? Or could it be marked somehow as unreliable data?
 I'm not sure it really *matters* that much from a trust perspective as
 it's use would likely be 99% debugging, but all the same it would be
 nice if it were trustable.
 
 Trust?  I mean, if you managed to upload data into EFI vars you are so
 privileged you can fuck with us anyway, so not sure what you mean

Well I was originally thinking about other OS's etc. and knowing that
logs came from the right install, but then posed the question in a more
generic sense, in part due to the cryptographic guarantees given by FSS.
I just figured this was an interesting way in which someone could inject
false data at a point where you have zero way to know what went on
between the last shutdown and the current boot.

If is to be inherently trusted based on machine id then fine - just
seems odd to put all the effort into FSS and then blindly inject data
(possibly even core dumps which will be analysed later) and not mark it
as potentially untrustworthy in some way.

Col

-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-13 Thread Zbigniew Jędrzejewski-Szmek
On Fri, Sep 13, 2013 at 08:50:06AM +0100, Colin Guthrie wrote:
 'Twas brillig, and Lennart Poettering at 13/09/13 00:02 did gyre and gimble:
  On Thu, 12.09.13 23:31, Colin Guthrie (gm...@colin.guthr.ie) wrote:
  
 
  'Twas brillig, and Lennart Poettering at 12/09/13 21:51 did gyre and 
  gimble:
  On Thu, 12.09.13 21:27, Colin Guthrie (gm...@colin.guthr.ie) wrote:
 
  So, as mentioned in the other thread, /usr should probably be on some
  OS resource blacklist or so, and not attempted to be shutdown.
 
  But unmounting /var during shutdown should actually work. The only thing
  that currently stops this from working cleanly is that journald is
  configured to stay around until the last moment, but currently has not
  interface to tell it to move its logging back to /run from /var. When we
  have that then the /var issue should go away for you, no?
 
  It would be quite nice to have an easy way for a compliant initrd to say
  it can and will handle /var unmounting (perhaps by dropping a
  var.mount.d/ snippet in the /run/systemd/system/ folder at boot?). 
 
  Yeah, such a drop-in should just work. Just add DefaultDependencies=no
  in there, and maybe instead Before=local-fs.target, and you should be 
  set.
 
  Nice, I'm sure dracut could implement that then.
 
  This will allow us to get persistent logs in which the initrd can
  report it's progress until the last possible moment. This might help
  debugging any problems in this particular link of the shutdown chain.
 
  I think the really late shutdown logs should more liekly end up in some
  EFI var and then flushed out on next boot or so...
 
  Nice plan. How would be able to trust that data tho'. Would it have to
  be signed in some way? Or could it be marked somehow as unreliable data?
  I'm not sure it really *matters* that much from a trust perspective as
  it's use would likely be 99% debugging, but all the same it would be
  nice if it were trustable.
  
  Trust?  I mean, if you managed to upload data into EFI vars you are so
  privileged you can fuck with us anyway, so not sure what you mean
 
 Well I was originally thinking about other OS's etc. and knowing that
 logs came from the right install, but then posed the question in a more
 generic sense, in part due to the cryptographic guarantees given by FSS.
 I just figured this was an interesting way in which someone could inject
 false data at a point where you have zero way to know what went on
 between the last shutdown and the current boot.
 
 If is to be inherently trusted based on machine id then fine - just
 seems odd to put all the effort into FSS and then blindly inject data
 (possibly even core dumps which will be analysed later) and not mark it
 as potentially untrustworthy in some way.
FSS doesn't protect against fake data, it only allows you to detect when
data has been removed. So if I break into a machine with f-s-sealed logs,
I am still able to log anything I want, but the messages about my break-in
will be there, unless I remove some messages, in which case it will be visible
that there are no messages since some point. The same guarantee holds for
messages from pstore: you need root rights to write there. Even if there's
a second OS on the machine, if you have root right in that *other* OS, you
can write whatever you want to the logs in *this* OS, either through pstore
or directly by mounting /var. So pstore doesn't really change anything here.

Zbyszek
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-13 Thread Colin Guthrie
'Twas brillig, and Zbigniew Jędrzejewski-Szmek at 13/09/13 16:33 did
gyre and gimble:
 On Fri, Sep 13, 2013 at 08:50:06AM +0100, Colin Guthrie wrote:
 'Twas brillig, and Lennart Poettering at 13/09/13 00:02 did gyre and gimble:
 On Thu, 12.09.13 23:31, Colin Guthrie (gm...@colin.guthr.ie) wrote:


 'Twas brillig, and Lennart Poettering at 12/09/13 21:51 did gyre and 
 gimble:
 On Thu, 12.09.13 21:27, Colin Guthrie (gm...@colin.guthr.ie) wrote:

 So, as mentioned in the other thread, /usr should probably be on some
 OS resource blacklist or so, and not attempted to be shutdown.

 But unmounting /var during shutdown should actually work. The only thing
 that currently stops this from working cleanly is that journald is
 configured to stay around until the last moment, but currently has not
 interface to tell it to move its logging back to /run from /var. When we
 have that then the /var issue should go away for you, no?

 It would be quite nice to have an easy way for a compliant initrd to say
 it can and will handle /var unmounting (perhaps by dropping a
 var.mount.d/ snippet in the /run/systemd/system/ folder at boot?). 

 Yeah, such a drop-in should just work. Just add DefaultDependencies=no
 in there, and maybe instead Before=local-fs.target, and you should be 
 set.

 Nice, I'm sure dracut could implement that then.

 This will allow us to get persistent logs in which the initrd can
 report it's progress until the last possible moment. This might help
 debugging any problems in this particular link of the shutdown chain.

 I think the really late shutdown logs should more liekly end up in some
 EFI var and then flushed out on next boot or so...

 Nice plan. How would be able to trust that data tho'. Would it have to
 be signed in some way? Or could it be marked somehow as unreliable data?
 I'm not sure it really *matters* that much from a trust perspective as
 it's use would likely be 99% debugging, but all the same it would be
 nice if it were trustable.

 Trust?  I mean, if you managed to upload data into EFI vars you are so
 privileged you can fuck with us anyway, so not sure what you mean

 Well I was originally thinking about other OS's etc. and knowing that
 logs came from the right install, but then posed the question in a more
 generic sense, in part due to the cryptographic guarantees given by FSS.
 I just figured this was an interesting way in which someone could inject
 false data at a point where you have zero way to know what went on
 between the last shutdown and the current boot.

 If is to be inherently trusted based on machine id then fine - just
 seems odd to put all the effort into FSS and then blindly inject data
 (possibly even core dumps which will be analysed later) and not mark it
 as potentially untrustworthy in some way.

 FSS doesn't protect against fake data, it only allows you to detect when
 data has been removed. So if I break into a machine with f-s-sealed logs,
 I am still able to log anything I want, but the messages about my break-in
 will be there, unless I remove some messages, in which case it will be visible
 that there are no messages since some point. The same guarantee holds for
 messages from pstore: you need root rights to write there. Even if there's
 a second OS on the machine, if you have root right in that *other* OS, you
 can write whatever you want to the logs in *this* OS, either through pstore
 or directly by mounting /var. So pstore doesn't really change anything here.

Hmm, yeah, I guess the point about writing invalid new data is probably
moot - as you say any $OTHEROS could do the same - assuming of course
the disk is not encrypted...

There would be some theoretical risk of a machine, with pstore and
encrypted disks where some attacker who has limited physical access
could plant a pstore bomb and then waits for the user to boot and
enter their encryption keys at which point the fake data is logged.
But I'm sure there are many more interesting things the attacker could
do if they were to have even limited physical access, so my argument is
almost certainly invalid in practically all circumstances :)

That said, if it's possible to somehow do the FSS of the pstore data and
verify it before incorporating it on the next boot, it seems like that
would be the most sensible approach by design (theoretically, in
principle etc. etc.) Practically useful... probably not much :)

Col











-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-13 Thread Zbigniew Jędrzejewski-Szmek
On Fri, Sep 13, 2013 at 04:48:37PM +0100, Colin Guthrie wrote:
 'Twas brillig, and Zbigniew Jędrzejewski-Szmek at 13/09/13 16:33 did
 gyre and gimble:
  On Fri, Sep 13, 2013 at 08:50:06AM +0100, Colin Guthrie wrote:
  'Twas brillig, and Lennart Poettering at 13/09/13 00:02 did gyre and 
  gimble:
  On Thu, 12.09.13 23:31, Colin Guthrie (gm...@colin.guthr.ie) wrote:
 
 
  'Twas brillig, and Lennart Poettering at 12/09/13 21:51 did gyre and 
  gimble:
  On Thu, 12.09.13 21:27, Colin Guthrie (gm...@colin.guthr.ie) wrote:
 
  So, as mentioned in the other thread, /usr should probably be on some
  OS resource blacklist or so, and not attempted to be shutdown.
 
  But unmounting /var during shutdown should actually work. The only 
  thing
  that currently stops this from working cleanly is that journald is
  configured to stay around until the last moment, but currently has not
  interface to tell it to move its logging back to /run from /var. When 
  we
  have that then the /var issue should go away for you, no?
 
  It would be quite nice to have an easy way for a compliant initrd to 
  say
  it can and will handle /var unmounting (perhaps by dropping a
  var.mount.d/ snippet in the /run/systemd/system/ folder at boot?). 
 
  Yeah, such a drop-in should just work. Just add DefaultDependencies=no
  in there, and maybe instead Before=local-fs.target, and you should be 
  set.
 
  Nice, I'm sure dracut could implement that then.
 
  This will allow us to get persistent logs in which the initrd can
  report it's progress until the last possible moment. This might help
  debugging any problems in this particular link of the shutdown chain.
 
  I think the really late shutdown logs should more liekly end up in some
  EFI var and then flushed out on next boot or so...
 
  Nice plan. How would be able to trust that data tho'. Would it have to
  be signed in some way? Or could it be marked somehow as unreliable data?
  I'm not sure it really *matters* that much from a trust perspective as
  it's use would likely be 99% debugging, but all the same it would be
  nice if it were trustable.
 
  Trust?  I mean, if you managed to upload data into EFI vars you are so
  privileged you can fuck with us anyway, so not sure what you mean
 
  Well I was originally thinking about other OS's etc. and knowing that
  logs came from the right install, but then posed the question in a more
  generic sense, in part due to the cryptographic guarantees given by FSS.
  I just figured this was an interesting way in which someone could inject
  false data at a point where you have zero way to know what went on
  between the last shutdown and the current boot.
 
  If is to be inherently trusted based on machine id then fine - just
  seems odd to put all the effort into FSS and then blindly inject data
  (possibly even core dumps which will be analysed later) and not mark it
  as potentially untrustworthy in some way.
 
  FSS doesn't protect against fake data, it only allows you to detect when
  data has been removed. So if I break into a machine with f-s-sealed logs,
  I am still able to log anything I want, but the messages about my break-in
  will be there, unless I remove some messages, in which case it will be 
  visible
  that there are no messages since some point. The same guarantee holds for
  messages from pstore: you need root rights to write there. Even if there's
  a second OS on the machine, if you have root right in that *other* OS, you
  can write whatever you want to the logs in *this* OS, either through pstore
  or directly by mounting /var. So pstore doesn't really change anything here.
 
 Hmm, yeah, I guess the point about writing invalid new data is probably
 moot - as you say any $OTHEROS could do the same - assuming of course
 the disk is not encrypted...
 
 There would be some theoretical risk of a machine, with pstore and
 encrypted disks where some attacker who has limited physical access
 could plant a pstore bomb and then waits for the user to boot and
 enter their encryption keys at which point the fake data is logged.
 But I'm sure there are many more interesting things the attacker could
 do if they were to have even limited physical access, so my argument is
 almost certainly invalid in practically all circumstances :)
 
 That said, if it's possible to somehow do the FSS of the pstore data and
 verify it before incorporating it on the next boot, it seems like that
 would be the most sensible approach by design (theoretically, in
 principle etc. etc.) Practically useful... probably not much :)
I think we need a new _TRANSPORT=pstore tag.

We currently don't do any verificiation for normal logs either: it is up
to the reader to notice that fields don't match. But it would be nice to
have an external tool like this.

Zbyszek
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-12 Thread Colin Guthrie
'Twas brillig, and Lennart Poettering at 12/09/13 17:43 did gyre and gimble:
 On Sat, 20.07.13 18:50, Colin Walters (walt...@verbum.org) wrote:
 
 Heya,
 
 So OSTree sets up systemd inside a chroot - /usr is a read-only bind
 mount, and /var is a bind mount outside the root to a shared location.
 Furthermore, /sysroot points to the real root.

 Since last time we discussed this:
 http://lists.freedesktop.org/archives/systemd-devel/2012-September/006668.html
 I now use this service inside dracut:
 https://git.gnome.org/browse/ostree/tree/src/dracut/ostree-prepare-root.service
 Which executes:
 https://git.gnome.org/browse/ostree/tree/src/switchroot/ostree-prepare-root.c

 Then finally we do dracut's normal systemctl switch-root, and everything
 continues as normal.  I haven't had to patch the systemd codebase at all
 for this.

 The problem is that on shutdown, systemd will synthesize usr.mount and
 var.mount from /proc/self/mountinfo, but it can't really unmount them
 until the same point as the rootfs.  Because these units fail to
 unmount, the normal shutdown process wedges.
 
 So, as mentioned in the other thread, /usr should probably be on some
 OS resource blacklist or so, and not attempted to be shutdown.
 
 But unmounting /var during shutdown should actually work. The only thing
 that currently stops this from working cleanly is that journald is
 configured to stay around until the last moment, but currently has not
 interface to tell it to move its logging back to /run from /var. When we
 have that then the /var issue should go away for you, no?

It would be quite nice to have an easy way for a compliant initrd to say
it can and will handle /var unmounting (perhaps by dropping a
var.mount.d/ snippet in the /run/systemd/system/ folder at boot?). This
will allow us to get persistent logs in which the initrd can report it's
progress until the last possible moment. This might help debugging any
problems in this particular link of the shutdown chain.

Col


-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-12 Thread Lennart Poettering
On Thu, 12.09.13 21:27, Colin Guthrie (gm...@colin.guthr.ie) wrote:

  So, as mentioned in the other thread, /usr should probably be on some
  OS resource blacklist or so, and not attempted to be shutdown.
  
  But unmounting /var during shutdown should actually work. The only thing
  that currently stops this from working cleanly is that journald is
  configured to stay around until the last moment, but currently has not
  interface to tell it to move its logging back to /run from /var. When we
  have that then the /var issue should go away for you, no?
 
 It would be quite nice to have an easy way for a compliant initrd to say
 it can and will handle /var unmounting (perhaps by dropping a
 var.mount.d/ snippet in the /run/systemd/system/ folder at boot?). 

Yeah, such a drop-in should just work. Just add DefaultDependencies=no
in there, and maybe instead Before=local-fs.target, and you should be set.

 This will allow us to get persistent logs in which the initrd can
 report it's progress until the last possible moment. This might help
 debugging any problems in this particular link of the shutdown chain.

I think the really late shutdown logs should more liekly end up in some
EFI var and then flushed out on next boot or so...

Lennart

-- 
Lennart Poettering - Red Hat, Inc.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-12 Thread Lennart Poettering
On Sat, 20.07.13 18:50, Colin Walters (walt...@verbum.org) wrote:

Heya,

 So OSTree sets up systemd inside a chroot - /usr is a read-only bind
 mount, and /var is a bind mount outside the root to a shared location.
 Furthermore, /sysroot points to the real root.
 
 Since last time we discussed this:
 http://lists.freedesktop.org/archives/systemd-devel/2012-September/006668.html
 I now use this service inside dracut:
 https://git.gnome.org/browse/ostree/tree/src/dracut/ostree-prepare-root.service
 Which executes:
 https://git.gnome.org/browse/ostree/tree/src/switchroot/ostree-prepare-root.c
 
 Then finally we do dracut's normal systemctl switch-root, and everything
 continues as normal.  I haven't had to patch the systemd codebase at all
 for this.
 
 The problem is that on shutdown, systemd will synthesize usr.mount and
 var.mount from /proc/self/mountinfo, but it can't really unmount them
 until the same point as the rootfs.  Because these units fail to
 unmount, the normal shutdown process wedges.

So, as mentioned in the other thread, /usr should probably be on some
OS resource blacklist or so, and not attempted to be shutdown.

But unmounting /var during shutdown should actually work. The only thing
that currently stops this from working cleanly is that journald is
configured to stay around until the last moment, but currently has not
interface to tell it to move its logging back to /run from /var. When we
have that then the /var issue should go away for you, no?

Now, moving journald at shutdown back from /var to /run would ideally be
done via some sane synchronous bus calls. However, we can only do that
when we have kdbus because dbus-daemon would otherwise be both client
and server to journald, which is deadlock territorry then...

Lennart

-- 
Lennart Poettering - Red Hat, Inc.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-12 Thread Jan Alexander Steffens
On Thu, Sep 12, 2013 at 10:51 PM, Lennart Poettering
lenn...@poettering.net wrote:
 I think the really late shutdown logs should more liekly end up in some
 EFI var and then flushed out on next boot or so...

That sounds good. Maybe use the pstore system? A service could then
read that data into the journal at boot, as well as any kernel panic
dumps it finds.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-12 Thread Colin Guthrie
'Twas brillig, and Lennart Poettering at 12/09/13 21:51 did gyre and gimble:
 On Thu, 12.09.13 21:27, Colin Guthrie (gm...@colin.guthr.ie) wrote:
 
 So, as mentioned in the other thread, /usr should probably be on some
 OS resource blacklist or so, and not attempted to be shutdown.

 But unmounting /var during shutdown should actually work. The only thing
 that currently stops this from working cleanly is that journald is
 configured to stay around until the last moment, but currently has not
 interface to tell it to move its logging back to /run from /var. When we
 have that then the /var issue should go away for you, no?

 It would be quite nice to have an easy way for a compliant initrd to say
 it can and will handle /var unmounting (perhaps by dropping a
 var.mount.d/ snippet in the /run/systemd/system/ folder at boot?). 
 
 Yeah, such a drop-in should just work. Just add DefaultDependencies=no
 in there, and maybe instead Before=local-fs.target, and you should be set.

Nice, I'm sure dracut could implement that then.

 This will allow us to get persistent logs in which the initrd can
 report it's progress until the last possible moment. This might help
 debugging any problems in this particular link of the shutdown chain.
 
 I think the really late shutdown logs should more liekly end up in some
 EFI var and then flushed out on next boot or so...

Nice plan. How would be able to trust that data tho'. Would it have to
be signed in some way? Or could it be marked somehow as unreliable data?
I'm not sure it really *matters* that much from a trust perspective as
it's use would likely be 99% debugging, but all the same it would be
nice if it were trustable.

Cheers

Col


-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-12 Thread Lennart Poettering
On Fri, 13.09.13 00:31, Jan Alexander Steffens (jan.steff...@gmail.com) wrote:

 
 On Thu, Sep 12, 2013 at 10:51 PM, Lennart Poettering
 lenn...@poettering.net wrote:
  I think the really late shutdown logs should more liekly end up in some
  EFI var and then flushed out on next boot or so...
 
 That sounds good. Maybe use the pstore system? A service could then
 read that data into the journal at boot, as well as any kernel panic
 dumps it finds.

I'd be happy to support pstore in the journald, so that we flush pstore
into it on boot. However, to make that workable we'd need some way how
we can handle multiple different parallel OS installations, so that if
you have Suse and Fedora installed in dual-boot the logs from Suse don't
end up in the Fedora journal and vice versa.

All we'd need for that if we could configure some string at boot for
pstore which would be included in the pstore dumps. systemd would then
initialize that string to the machine ID, and we could easily match the
pstore to the machine ID at each boot.

Lennart

-- 
Lennart Poettering - Red Hat, Inc.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-09-12 Thread Lennart Poettering
On Thu, 12.09.13 23:31, Colin Guthrie (gm...@colin.guthr.ie) wrote:

 
 'Twas brillig, and Lennart Poettering at 12/09/13 21:51 did gyre and gimble:
  On Thu, 12.09.13 21:27, Colin Guthrie (gm...@colin.guthr.ie) wrote:
  
  So, as mentioned in the other thread, /usr should probably be on some
  OS resource blacklist or so, and not attempted to be shutdown.
 
  But unmounting /var during shutdown should actually work. The only thing
  that currently stops this from working cleanly is that journald is
  configured to stay around until the last moment, but currently has not
  interface to tell it to move its logging back to /run from /var. When we
  have that then the /var issue should go away for you, no?
 
  It would be quite nice to have an easy way for a compliant initrd to say
  it can and will handle /var unmounting (perhaps by dropping a
  var.mount.d/ snippet in the /run/systemd/system/ folder at boot?). 
  
  Yeah, such a drop-in should just work. Just add DefaultDependencies=no
  in there, and maybe instead Before=local-fs.target, and you should be set.
 
 Nice, I'm sure dracut could implement that then.
 
  This will allow us to get persistent logs in which the initrd can
  report it's progress until the last possible moment. This might help
  debugging any problems in this particular link of the shutdown chain.
  
  I think the really late shutdown logs should more liekly end up in some
  EFI var and then flushed out on next boot or so...
 
 Nice plan. How would be able to trust that data tho'. Would it have to
 be signed in some way? Or could it be marked somehow as unreliable data?
 I'm not sure it really *matters* that much from a trust perspective as
 it's use would likely be 99% debugging, but all the same it would be
 nice if it were trustable.

Trust?  I mean, if you managed to upload data into EFI vars you are so
privileged you can fuck with us anyway, so not sure what you mean

Lennart

-- 
Lennart Poettering - Red Hat, Inc.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-07-24 Thread Colin Guthrie
'Twas brillig, and Colin Walters at 20/07/13 23:50 did gyre and gimble:
 So OSTree sets up systemd inside a chroot - /usr is a read-only bind
 mount, and /var is a bind mount outside the root to a shared location.
 Furthermore, /sysroot points to the real root.
 
 Since last time we discussed this:
 http://lists.freedesktop.org/archives/systemd-devel/2012-September/006668.html
 I now use this service inside dracut:
 https://git.gnome.org/browse/ostree/tree/src/dracut/ostree-prepare-root.service
 Which executes:
 https://git.gnome.org/browse/ostree/tree/src/switchroot/ostree-prepare-root.c
 
 Then finally we do dracut's normal systemctl switch-root, and everything
 continues as normal.  I haven't had to patch the systemd codebase at all
 for this.
 
 The problem is that on shutdown, systemd will synthesize usr.mount and
 var.mount from /proc/self/mountinfo, but it can't really unmount them
 until the same point as the rootfs.  Because these units fail to
 unmount, the normal shutdown process wedges.
 
 I can shutdown fine with systemctl --force poweroff, but then I don't
 get plymouth integration etc.
 
 One way to fix this might be to somehow tell systemd to just ignore
 these mount points during shutdown.  Or possibly, switch back to the
 initramfs and unmount them from there.
 
 The ugly thing about switching back to the initramfs is that it requires
 unpacking it from the cpio blob again, which requires /boot to be
 mounted, only to run a few unmount syscalls, and then finally power off.
 
 But if there was a way to tell systemd to just ignore the mounts, then
 we'd drop into the final poweroff SIGTERM/SIGKILL/umount spree like
 sysvinit did, and things would work.
 
 Anyone else doing bind mount tricks like this?

Interestingly, I asked something similar a while back. See the thread:
x-initrd.mount + shutdown umount logic question

http://thread.gmane.org/gmane.comp.sysutils.systemd.devel/11210

In my case it was pretty much a cosmetic issue but you want a harder
exclusion I think...

Col

-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
  Tribalogic Limited http://www.tribalogic.net/
Open Source:
  Mageia Contributor http://www.mageia.org/
  PulseAudio Hacker http://www.pulseaudio.org/
  Trac Hacker http://trac.edgewall.org/

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd shutdown vs ostree

2013-07-24 Thread Daniel P. Berrange
On Sat, Jul 20, 2013 at 06:50:13PM -0400, Colin Walters wrote:
 So OSTree sets up systemd inside a chroot - /usr is a read-only bind
 mount, and /var is a bind mount outside the root to a shared location.
 Furthermore, /sysroot points to the real root.
 
 Since last time we discussed this:
 http://lists.freedesktop.org/archives/systemd-devel/2012-September/006668.html
 I now use this service inside dracut:
 https://git.gnome.org/browse/ostree/tree/src/dracut/ostree-prepare-root.service
 Which executes:
 https://git.gnome.org/browse/ostree/tree/src/switchroot/ostree-prepare-root.c
 
 Then finally we do dracut's normal systemctl switch-root, and everything
 continues as normal.  I haven't had to patch the systemd codebase at all
 for this.
 
 The problem is that on shutdown, systemd will synthesize usr.mount and
 var.mount from /proc/self/mountinfo, but it can't really unmount them
 until the same point as the rootfs.  Because these units fail to
 unmount, the normal shutdown process wedges.
 
 I can shutdown fine with systemctl --force poweroff, but then I don't
 get plymouth integration etc.
 
 One way to fix this might be to somehow tell systemd to just ignore
 these mount points during shutdown.  Or possibly, switch back to the
 initramfs and unmount them from there.
 
 The ugly thing about switching back to the initramfs is that it requires
 unpacking it from the cpio blob again, which requires /boot to be
 mounted, only to run a few unmount syscalls, and then finally power off.
 
 But if there was a way to tell systemd to just ignore the mounts, then
 we'd drop into the final poweroff SIGTERM/SIGKILL/umount spree like
 sysvinit did, and things would work.
 
 Anyone else doing bind mount tricks like this?

A while back had a similar-ish kind of problem with LXC, when the original
FS had something mounted at say /foo/bar/wizz, and then libvirt bind mounted
something at /foo, making /foo/bar/wizz inaccessible. systemd would
still see these over-mounted mounts and fail to unmount them at shutdown.
I fixed libvirt LXC to remove all sub-mounts before bind mounting the
new thing at /foo, so not sure if the problems I saw with systemd would
still exist or not.

There is also a change proposed for the kernel namespaces yesterday to
make it possible to stop a process inside a container from unmounting
things that wasn't originally mounted inside the namespace. So if that
is merged, systemd inside a container wouldn't be able to assume it
has the privileges to unmount all filesystems it can see.

Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


[systemd-devel] systemd shutdown vs ostree

2013-07-20 Thread Colin Walters
So OSTree sets up systemd inside a chroot - /usr is a read-only bind
mount, and /var is a bind mount outside the root to a shared location.
Furthermore, /sysroot points to the real root.

Since last time we discussed this:
http://lists.freedesktop.org/archives/systemd-devel/2012-September/006668.html
I now use this service inside dracut:
https://git.gnome.org/browse/ostree/tree/src/dracut/ostree-prepare-root.service
Which executes:
https://git.gnome.org/browse/ostree/tree/src/switchroot/ostree-prepare-root.c

Then finally we do dracut's normal systemctl switch-root, and everything
continues as normal.  I haven't had to patch the systemd codebase at all
for this.

The problem is that on shutdown, systemd will synthesize usr.mount and
var.mount from /proc/self/mountinfo, but it can't really unmount them
until the same point as the rootfs.  Because these units fail to
unmount, the normal shutdown process wedges.

I can shutdown fine with systemctl --force poweroff, but then I don't
get plymouth integration etc.

One way to fix this might be to somehow tell systemd to just ignore
these mount points during shutdown.  Or possibly, switch back to the
initramfs and unmount them from there.

The ugly thing about switching back to the initramfs is that it requires
unpacking it from the cpio blob again, which requires /boot to be
mounted, only to run a few unmount syscalls, and then finally power off.

But if there was a way to tell systemd to just ignore the mounts, then
we'd drop into the final poweroff SIGTERM/SIGKILL/umount spree like
sysvinit did, and things would work.

Anyone else doing bind mount tricks like this?





___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel