Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Monday 06 November 2006 16:53, Roy Marples wrote: However, one issue is a concern. All baselayouts defined svcdir in /etc/conf.d/rc which defines where we hold the state information of the running services. This defaulted to /var/lib/init.d - which is bad as /var could be on a different partition. In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced to /lib/rcscripts/init.d which is safe as /lib is always on the same partition as /. The 1.13 ebuild will copy across existing state data, this is not the problem. However, downgrading back to 1.12 is a problem as services may have been stop, started etc in the middle. One solution is to ensure that we only hold one copy of the state data and move it to the new location. However, this does require altering the stable ebuild as well. This is still an issue. What do people think - hack the 1.12 ebuilds and try and get an upgrade/downgrade path or just slap a large warning on the ebuild? You get upgraded for free :) -- Roy Marples [EMAIL PROTECTED] Gentoo/Linux Developer (baselayout, networking) -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Wednesday 08 November 2006 19:00, Roy Marples wrote: just slap a large warning on the ebuild? that's fine by me -mike pgpgK8CQrFiYy.pgp Description: PGP signature
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
Roy Marples wrote: [Mon Nov 06 2006, 12:44:42PM EST]
+if [[ $'\n'$(get_mounts) =~ $'\n'${svcdir}\ -w ${svclib} ]] ; then
Shouldn't this be:
if [[ $'\n'$(get_mounts) == $'\n'${svcdir} ...
because I don't think you want to treat the RHS as either a regex (=~)
or a glob (unquoted).
Aron
--
gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Tuesday 07 November 2006 15:59, Aron Griffis wrote:
+if [[ $'\n'$(get_mounts) =~ $'\n'${svcdir}\ -w ${svclib} ]] ; then
Shouldn't this be:
if [[ $'\n'$(get_mounts) == $'\n'${svcdir} ...
because I don't think you want to treat the RHS as either a regex (=~)
or a glob (unquoted).
Needs to be regex so I can match $'\n' as iirc you loose that in globbing
--
Roy Marples [EMAIL PROTECTED]
Gentoo/Linux/FreeBSD Developer (baselayout, networking)
--
gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
Roy Marples [EMAIL PROTECTED] wrote:
On Tuesday 07 November 2006 15:59, Aron Griffis wrote:
+if [[ $'\n'$(get_mounts) =~ $'\n'${svcdir}\ -w ${svclib} ]] ; then
Shouldn't this be:
if [[ $'\n'$(get_mounts) == $'\n'${svcdir} ...
because I don't think you want to treat the RHS as either a regex
(=~) or a glob (unquoted).
Needs to be regex so I can match $'\n' as iirc you loose that in
globbing
I could be missing something, but:
[[ $'\nwombat' =~ $'wombat' ]] \
echo These compare as equal, with or without the leading \n
They do not compare as equal with the == operator or the = operator. You
probably want the = operator, because the == operator _does_ interpret
the RHS as a glob. The = operator just uses simple string comparison,
without interpreting anything.
I am curious about why the space was included at the end of the in the
expression:
$'\n'${svcdir}\
Does get_mounts add a space to the end of its output?
--
Every absurdity has a champion to defend it.
-- Oliver Goldsmith
pgpzmDvJMG5B3.pgp
Description: PGP signature
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Tuesday 07 November 2006 20:32, Drake Wyrm wrote: I could be missing something, but: [[ $'\nwombat' =~ $'wombat' ]] \ echo These compare as equal, with or without the leading \n A working example in bash-3.2 :) [[ $(/proc/mounts) =~ $'\n'/dev/root\ ]] \ echo Yay, I matched ^/dev/root I challenge you to get an exact match of /dev/root being on the first line using the == operator and/or quoting. Remember, /dev/root/foo and /dev/foo /dev/root must not match either. If you can you get credit in the ChangeLog :) Yes, you could also write grep -E ^/dev/root /proc/mounts, but using bash like this is faster [1] [1] http://roy.marples.name/node/267 Thanks -- Roy Marples [EMAIL PROTECTED] Gentoo/Linux Developer (baselayout, networking) -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Tuesday 07 November 2006 20:32, Drake Wyrm wrote: I could be missing something, but: [[ $'\nwombat' =~ $'wombat' ]] \ echo These compare as equal, with or without the leading \n They do not compare as equal with the == operator or the = operator. You probably want the = operator, because the == operator _does_ interpret the RHS as a glob. The = operator just uses simple string comparison, without interpreting anything. Interesting. You just asked the regex(3) command if $'\nwombat' contains $'wombat' Maybe you meant to write [[ $'\n'wombat =~ wombat ]] That will still return true [[ wombat =~ $'\n'wombat ]] Will return false, which is what we want as wombat isn't at the start of the line. In my example, I'm trying to match something at the start of a line. -- Roy Marples [EMAIL PROTECTED] Gentoo/Linux Developer (baselayout, networking) -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
Roy Marples ha scritto:
[[ $(/proc/mounts) =~ $'\n'/dev/root\ ]] \
echo Yay, I matched ^/dev/root
I challenge you to get an exact match of /dev/root being on the first line
using the == operator and/or quoting.
Remember, /dev/root/foo and /dev/foo /dev/root must not match either.
mounts=( $(/proc/mounts) )
for m in 0 6 12 ; do
[[ ${mounts[${m}]} == /dev/root ]] \
echo Yay, I matched ^/dev/root
done
don't know which is faster/safer (bet on ~=)
--
gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Tue, Nov 07, 2006 at 08:47:18PM +, Roy Marples wrote: On Tuesday 07 November 2006 20:32, Drake Wyrm wrote: I could be missing something, but: [[ $'\nwombat' =~ $'wombat' ]] \ echo These compare as equal, with or without the leading \n A working example in bash-3.2 :) [[ $(/proc/mounts) =~ $'\n'/dev/root\ ]] \ echo Yay, I matched ^/dev/root I challenge you to get an exact match of /dev/root being on the first line You mean being first on a line, right? using the == operator and/or quoting. Remember, /dev/root/foo and /dev/foo /dev/root must not match either. If you can you get credit in the ChangeLog :) [[ $(/proc/mounts) == * /dev/root * ]] echo Yay, I matched ^/dev/root You can use $'\n' instead of actual newline characters, of course. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Wednesday 08 November 2006 06:57, Harald van Dijk wrote: [[ $(/proc/mounts) == * /dev/root * ]] echo Yay, I matched ^/dev/root You can use $'\n' instead of actual newline characters, of course. You get credit :) I couldn't get that to work with bash-3.1 for some reason though, as I prefer == over =~ myself. Thanks -- Roy Marples [EMAIL PROTECTED] Gentoo/Linux Developer (baselayout, networking) -- gentoo-dev@gentoo.org mailing list
[gentoo-dev] baselayout-1.13 going into ~ARCH soon
Hi List! This is a heads up to say that I'm going to be putting baselayout-1.13 into ~ARCH soon as all the exciting new features I wanted are in - FreeBSD and vserver support, buffered and wrapped einfo/ewarn/eerror output, rc-depend for lightning fast dependency sorting, no more critical services, no more net service specific code. So if you're concerned about any of the above features breaking your precious Gentoo, now is a very good time to test :) However, one issue is a concern. All baselayouts defined svcdir in /etc/conf.d/rc which defines where we hold the state information of the running services. This defaulted to /var/lib/init.d - which is bad as /var could be on a different partition. In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced to /lib/rcscripts/init.d which is safe as /lib is always on the same partition as /. The 1.13 ebuild will copy across existing state data, this is not the problem. However, downgrading back to 1.12 is a problem as services may have been stop, started etc in the middle. One solution is to ensure that we only hold one copy of the state data and move it to the new location. However, this does require altering the stable ebuild as well. Or we could just slap a very large warning on it. Ideas are welcome :) -- Roy Marples [EMAIL PROTECTED] Gentoo/Linux/FreeBSD Developer (baselayout, networking) -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
So if you're concerned about any of the above features breaking your precious Gentoo, now is a very good time to test :) Mon Oct 2 22:24:05 2006 sys-apps/baselayout-1.13.0_alpha1-r1 ^^ Using 1.13* for over a month and no problems whatsover. -- Piotr Jaroszyński Gentoo Developer -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Monday 06 November 2006 17:53, Roy Marples wrote: ... However, one issue is a concern. All baselayouts defined svcdir in /etc/conf.d/rc which defines where we hold the state information of the running services. This defaulted to /var/lib/init.d - which is bad as /var could be on a different partition. In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced to /lib/rcscripts/init.d which is safe as /lib is always on the same partition as /. The 1.13 ebuild will copy across existing state data, this is not the problem. However, downgrading back to 1.12 is a problem as services may have been stop, started etc in the middle. ... How is the case where the / partition always remains ro handled? Is rc-state information put into a tmpfs partition on that location, is the location configured differently for those? Bruno -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Monday 06 November 2006 17:12, Bruno wrote: How is the case where the / partition always remains ro handled? Is rc-state information put into a tmpfs partition on that location, is the location configured differently for those? Good question! / is always ro at boot and the checkroot init script remounts it rw after checking it. As such, we mount /lib/rcscripts/init.d/ as tmpfs (or ramfs or similar). Yes, this means that the kernel must be configured with a ramdisk or similar OR /lib/rscripts/init.d is writeable. At the end of the rc process (ie end of boot runlevel) we tar up our state dir, unmount it and untar it back (taking good care with locking due to parallel starts and event driven services). Admittedly, an always ro / isn't handled right now, but I'll ensure it will be for the next release :) Thanks -- Roy Marples [EMAIL PROTECTED] Gentoo/Linux/FreeBSD Developer (baselayout, networking) -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Monday 06 November 2006 17:33, Roy Marples wrote:
Admittedly, an always ro / isn't handled right now, but I'll ensure it will
be for the next release :)
We handle it with the attached patch, just comitted to our svn repo :)
Thanks
--
Roy Marples [EMAIL PROTECTED]
Gentoo/Linux/FreeBSD Developer (baselayout, networking)
Index: ChangeLog
===
--- ChangeLog (revision 2364)
+++ ChangeLog (working copy)
@@ -1,6 +1,11 @@
# ChangeLog for Gentoo System Intialization (rc) scripts
# Copyright 1999-2006 Gentoo Foundation; Distributed under the GPLv2
+ 06 Nov 2006; Roy Marples [EMAIL PROTECTED]:
+
+Ensure that we only move $svcdir from tmpfs to disk if $svclib is rw.
+Thanks to Bruno for the idea.
+
05 Nov 2006; Roy Marples [EMAIL PROTECTED]:
Don't create /proc on Linux as build scripts like to handle it now.
Index: sbin/rc
===
--- sbin/rc (revision 2364)
+++ sbin/rc (working copy)
@@ -442,7 +442,8 @@
rm -rf ${svcdir}/failed /dev/null
# If $svcdir is mounted in ram, save it back to disk and unmount
-if [[ $'\n'$(get_mounts) =~ $'\n'${svcdir}\ ]] ; then
+# but only if $svclib is writeable.
+if [[ $'\n'$(get_mounts) =~ $'\n'${svcdir}\ -w ${svclib} ]] ; then
# Function to show the timeout message
timeout=
do_timeout() {
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
Roy Marples wrote:
[snip that change the meaning of the message ;]
Ideas are welcome :)
need to jump net.lo in symlink tests fex as tested below:
for f in ${ROOT}etc/init.d/net.*; do
[[ ${f} == ${ROOT}etc/init.d/net.lo || -L ${f} ]] continue
echo
einfo WARNING: You have older net.* files in ${ROOT}etc/init.d/
[...]
there is a bug where to report gotchas ?
--
gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Monday 06 November 2006 17:51, Francesco Riosa wrote:
Roy Marples wrote:
[snip that change the meaning of the message ;]
Ideas are welcome :)
need to jump net.lo in symlink tests fex as tested below:
for f in ${ROOT}etc/init.d/net.*; do
[[ ${f} == ${ROOT}etc/init.d/net.lo || -L ${f} ]] continue
echo
einfo WARNING: You have older net.* files in ${ROOT}etc/init.d/
[...]
No I don't - net.lo is now a symlink itself to /lib/rcscripts/sh/net.sh
This solves the issue of users not etc-updating net.lo, and then getting loads
of errors. Which happens all to frequently.
there is a bug where to report gotchas ?
Nope. Just file a normal bug.
--
Roy Marples [EMAIL PROTECTED]
Gentoo/Linux/FreeBSD Developer (baselayout, networking)
--
gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On 06 Nov 2006 04:53 PM or thereabouts, Roy Marples wrote: This is a heads up to say that I'm going to be putting baselayout-1.13 into ~ARCH soon as all the exciting new features I wanted are in - FreeBSD and vserver support, buffered and wrapped einfo/ewarn/eerror output, rc-depend for lightning fast dependency sorting, no more critical services, no more net service specific code. Very nice! However, one issue is a concern. All baselayouts defined svcdir in /etc/conf.d/rc which defines where we hold the state information of the running services. This defaulted to /var/lib/init.d - which is bad as /var could be on a different partition. In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced to /lib/rcscripts/init.d which is safe as /lib is always on the same partition as /. From a filesystem usage point of view though, storing actively changing state data on /lib is ugly. The tmpfs /lib/rcscripts/init.d overlay solution for a ro / works, but as long as tmpfs magic is needed, can't it be written to /var after localmount? --Matthew [EMAIL PROTECTED] -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
Roy Marples wrote: On Monday 06 November 2006 18:27, Matthew Snelham wrote: In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced to /lib/rcscripts/init.d which is safe as /lib is always on the same partition as /. From a filesystem usage point of view though, storing actively changing state data on /lib is ugly. The tmpfs /lib/rcscripts/init.d overlay solution for a ro / works, but as long as tmpfs magic is needed, can't it be written to /var after localmount? It could be written to /var at the end of an rc. However, you then have to guarantee that /var is always available too and that may not always be the case as some people want to have /var net mounted or something equally silly :) /lib may be ugly, but it's also guaranteed which is what I'm more interested in. Thanks This screams vustomizable Just give us a variable we can set to move it; obviously there is no One True Location for everyone. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Monday 06 November 2006 22:06, Alec Warner wrote: Roy Marples wrote: On Monday 06 November 2006 18:27, Matthew Snelham wrote: In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced to /lib/rcscripts/init.d which is safe as /lib is always on the same partition as /. From a filesystem usage point of view though, storing actively changing state data on /lib is ugly. The tmpfs /lib/rcscripts/init.d overlay solution for a ro / works, but as long as tmpfs magic is needed, can't it be written to /var after localmount? It could be written to /var at the end of an rc. However, you then have to guarantee that /var is always available too and that may not always be the case as some people want to have /var net mounted or something equally silly :) /lib may be ugly, but it's also guaranteed which is what I'm more interested in. Thanks This screams vustomizable Just give us a variable we can set to move it; obviously there is no One True Location for everyone. If you want that level of flexability then simply symlink /lib/rcscripts to /var/rcscripts or where-ever you like. From my perspective, state data always has to be available and writeable, regardless of how simple or fancy the user has configured their layout. So this means I have access to /bin, /dev, /etc, /lib, /sbin as those directories have to exist on /. Why do we have to have everything configured by variables? -- Roy Marples [EMAIL PROTECTED] Gentoo Developer (baselayout, networking) -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On 06 Nov 2006 09:57 PM or thereabouts, Roy Marples wrote: On Monday 06 November 2006 22:06, Alec Warner wrote: Roy Marples wrote: On Monday 06 November 2006 18:27, Matthew Snelham wrote: From a filesystem usage point of view though, storing actively changing state data on /lib is ugly. The tmpfs /lib/rcscripts/init.d overlay solution for a ro / works, but as long as tmpfs magic is needed, can't it be written to /var after localmount? It could be written to /var at the end of an rc. However, you then have to guarantee that /var is always available too and that may not always be the case as some people want to have /var net mounted or something equally silly :) /lib may be ugly, but it's also guaranteed which is what I'm more interested in. This screams vustomizable Just give us a variable we can set to move it; obviously there is no One True Location for everyone. If you want that level of flexability then simply symlink /lib/rcscripts to /var/rcscripts or where-ever you like. But then baselayout is still 'behaving badly' by sttempting to store dynamic state information in /lib. Something it has not done before, to the best of my knowledge (with the exception of /dev state tarballs, which are generally acceptable, since they don't change while the system is up). UNIX filesystem usage patterns are older than a good chunk of gentoo devs, so in the name of defaulting to expected behaviour, I think /lib should be avoided. From my perspective, state data always has to be available and writeable, regardless of how simple or fancy the user has configured their layout. So this means I have access to /bin, /dev, /etc, /lib, /sbin as those directories have to exist on /. /usr and /var have to be accessable for a working system... and if init fails before those are availible, the system is non-functional regardless. So a tmp storage location is needed for state data early in the boot process (before writeable filesystems can be assured), and can flip before the boot runlevel is completed. (I've built a number of clusters with NFS root fs, but I've never even heard of a disk backed root with an NFS /var. Can we say that's pathologically odd, and unsupported/unsupportable?) Why do we have to have everything configured by variables? Eh, I'm not sure I see the need for this to be a variable. I'd just like to see it well behaved. --Matthew [EMAIL PROTECTED] -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
Matthew Snelham wrote: If you want that level of flexability then simply symlink /lib/rcscripts to /var/rcscripts or where-ever you like. But then baselayout is still 'behaving badly' by sttempting to store dynamic state information in /lib. Something it has not done before, to the best of my knowledge (with the exception of /dev state tarballs, which are generally acceptable, since they don't change while the system is up). UNIX filesystem usage patterns are older than a good chunk of gentoo devs, so in the name of defaulting to expected behaviour, I think /lib should be avoided. +1 This is a very good point, why are we breaking from accepted UNIX standards uselessly? Generally, a live system should never need to write to /lib, but a writable /var is pretty much standard. This new behavior breaks standards, if /var is on a separate filesystem, maybe we can use a subdir in /tmp for the init stuff until we get /var up, then move it over. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Monday 06 November 2006 21:42, Matthew Snelham wrote: But then baselayout is still 'behaving badly' by sttempting to store dynamic state information in /lib. it is and it isnt ... the dir is memory based so /lib could be read-only and that's fine -mike pgpTSguX5K8Nu.pgp Description: PGP signature
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
Patrick McLean wrote: Matthew Snelham wrote: If you want that level of flexability then simply symlink /lib/rcscripts to /var/rcscripts or where-ever you like. But then baselayout is still 'behaving badly' by sttempting to store dynamic state information in /lib. Something it has not done before, to the best of my knowledge (with the exception of /dev state tarballs, which are generally acceptable, since they don't change while the system is up). UNIX filesystem usage patterns are older than a good chunk of gentoo devs, so in the name of defaulting to expected behaviour, I think /lib should be avoided. +1 This is a very good point, why are we breaking from accepted UNIX standards uselessly? Generally, a live system should never need to write to /lib, but a writable /var is pretty much standard. This new behavior breaks standards, if /var is on a separate filesystem, maybe we can use a subdir in /tmp for the init stuff until we get /var up, then move it over. Agreed, this is a good point. Writing to /lib will also cause security complications for things like AIDE and other intrusion detection systems that look for writes to certain directories. If they see /lib changing quite often, it might confuse 'em and the sysadmin, who will get a rash of spurious alerts. signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Tuesday 07 November 2006 04:06, Josh Saddler wrote: Agreed, this is a good point. Writing to /lib will also cause security complications for things like AIDE and other intrusion detection systems that look for writes to certain directories. If they see /lib changing quite often, it might confuse 'em and the sysadmin, who will get a rash of spurious alerts. H? It would only be updated when a service starts or stops or a user updates their config, which would imply a restart I suppose. The state data is generally only written to when starting up and shutting down - in the meantime it should be minimal. -- Roy Marples [EMAIL PROTECTED] Gentoo/Linux Developer (baselayout, networking) -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon
On Tuesday 07 November 2006 02:42, Matthew Snelham wrote: (I've built a number of clusters with NFS root fs, but I've never even heard of a disk backed root with an NFS /var. Can we say that's pathologically odd, and unsupported/unsupportable?) OK, I have /var mounted on an LVM. I need to run an fsck on it, so I unmount it and do the stuff I need to in single user mode. In baselayout-1.13 I can bring the entire system back up by going back to the default runlevel (although there is one error in checkfs I have to address). Now how can we do that if our state data no longer exists as /var is not available? Yes the user could remount it and it's back, but what happens if this is a laptop and the user plugs in their wifi card and they urgently need to get network running and there's a problem mounting /var? Why do we have to have everything configured by variables? Eh, I'm not sure I see the need for this to be a variable. I'd just like to see it well behaved. Well behaved as to the LFS or well behaved as in coping for as many scenarios as we can? I'm all for the later. If you want the former then you're welcome to suggest alternative fixes for this too :) -- Roy Marples [EMAIL PROTECTED] Gentoo/Linux Developer (baselayout, networking) -- gentoo-dev@gentoo.org mailing list
