Re: RFC: support for first boot rc.d scripts
On 10/14/13 10:00, Ian Lepore wrote:
The embedded systems we create at $work have readonly root and mfs /var,
but we do have writable storage on another filesystem. It would work
for us (not that we need this feature right now) if there were an rcvar
that pointed to the marker file. Of course to make it work, something
would have to get the alternate filesystem mounted early enough to be
useful (that is something we do already with a custom rc script).
New patch attached. This one re-probes for the firstboot sentinel
after ${early_late_divider}, so you can set firstboot_sentinel to
/path/to/my/writable/storage as long as that's available once the
boot process reaches FILESYSTEMS (or NETWORKING, or whatever you
set early_late_divider to). I figure that if we can assume all the
local rc.d scripts are available at that point we can assume that
wherever people decide to put the firstboot sentinel will also be
available at that point.
Does anyone see any problems with this?
--
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
Index: etc/defaults/rc.conf
===
--- etc/defaults/rc.conf(revision 256432)
+++ etc/defaults/rc.conf(working copy)
@@ -619,6 +619,9 @@
accounting_enable=NO # Turn on process accounting (or NO).
ibcs2_enable=NO # Ibcs2 (SCO) emulation loaded at startup (or NO).
ibcs2_loaders=coff # List of additional Ibcs2 loaders (or NO).
+firstboot_sentinel=/firstboot# Scripts with firstboot keyword are
run if
+ # this file exists. Should be on a R/W filesystem so
+ # the file can be deleted after the boot completes.
# Emulation/compatibility services provided by /etc/rc.d/abi
sysvipc_enable=NO# Load System V IPC primitives at startup (or NO).
Index: etc/rc
===
--- etc/rc (revision 256432)
+++ etc/rc (working copy)
@@ -82,10 +82,15 @@
fi
fi
+# If the firstboot sentinel doesn't exist, we want to skip firstboot scripts.
+if ! [ -e ${firstboot_sentinel} ]; then
+ skip_firstboot=-s firstboot
+fi
+
# Do a first pass to get everything up to $early_late_divider so that
# we can do a second pass that includes $local_startup directories
#
-files=`rcorder ${skip} /etc/rc.d/* 2/dev/null`
+files=`rcorder ${skip} ${skip_firstboot} /etc/rc.d/* 2/dev/null`
_rc_elem_done=' '
for _rc_elem in ${files}; do
@@ -107,7 +112,13 @@
*) find_local_scripts_new ;;
esac
-files=`rcorder ${skip} /etc/rc.d/* ${local_rc} 2/dev/null`
+# The firstboot sentinel might be on a newly mounted filesystem; look for it
+# again and unset skip_firstboot if we find it.
+if [ -e ${firstboot_sentinel} ]; then
+ skip_firstboot=
+fi
+
+files=`rcorder ${skip} ${skip_firstboot} /etc/rc.d/* ${local_rc} 2/dev/null`
for _rc_elem in ${files}; do
case $_rc_elem_done in
* $_rc_elem *)continue ;;
@@ -116,6 +127,15 @@
run_rc_script ${_rc_elem} ${_boot}
done
+# Remove the firstboot sentinel, and reboot if it was requested.
+if [ -e ${firstboot_sentinel} ]; then
+ rm ${firstboot_sentinel}
+ if [ -e ${firstboot_sentinel}-reboot ]; then
+ rm ${firstboot_sentinel}-reboot
+ kill -INT 1
+ fi
+fi
+
echo ''
date
exit 0
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
Yes, it's hard to store state on diskless systems... but I figured that anyone building a diskless system would know to not create a run firstboot scripts marker. And not all embedded systems are diskless... The embedded systems we create at $work have readonly root and mfs /var, but we do have writable storage on another filesystem. It would work for us (not that we need this feature right now) if there were an rcvar that pointed to the marker file. Of course to make it work, something would have to get the alternate filesystem mounted early enough to be useful (that is something we do already with a custom rc script). Indeed... the way my patch currently does things, it looks for the firstboot sentinel at the start of /etc/rc, which means it *has* to be on /. Making the path an rcvar is a good idea (updated patch attached) but we still need some way to re-probe for that file after mounting extra filesystems. In many cases a simple test -f /firstboot bla_enable='YES' || bla_enable='NO' rm -f /firstboot in your specific rc.d script would suffice. Or for installing packages: for pkg in $PKGS; do if ! pkg_info $pkg-'[0-9]*' /dev/null 21; then pkg_add /some/dir/$pkg.txz fi done I am not quite sure why we need /firstboot handling in /etc/rc. Perhaps it is a better idea to make this more generic, to move the rc.d script containing a 'runonce' keyword to a subdirectory as the last step in rc (or make that an rc.d script in itself!). That way you could consider moving it back if you need to re-run it. Or have an rc.d script setup something like a database after installing a package by creating a rc.d runonce script. Default dir could be ./run-once relative to the rc.d dir it is in, configurable through runonce_directory . Note: The move would need to be done at the very end of rc.d to prevent rcorder returning a different ordering and skipping scripts because of that. Nick ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
Wonderful! This capability is long overdue. On Oct 13, 2013, at 3:58 PM, Colin Percival cperc...@freebsd.org wrote: As examples of what such scripts could do: More examples: I've been experimenting with putting gpart resize and growfs into rc.d scripts to construct images that can be dd'ed onto some medium and then automatically grow to fill the medium. When cross-installing ports, there are certain operations (e.g., updating 'info' database) that can really only be done after the system next boots. I'd like to get this into HEAD in the near future in the hope that I can convince re@ that this is a simple enough (and safe enough) change to merge before 10.0-RELEASE. Please. Tim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
On 2013-10-15 15:33, Tim Kientzle wrote: Wonderful! This capability is long overdue. On Oct 13, 2013, at 3:58 PM, Colin Percival cperc...@freebsd.org wrote: As examples of what such scripts could do: More examples: I've been experimenting with putting gpart resize and growfs into rc.d scripts to construct images that can be dd'ed onto some medium and then automatically grow to fill the medium. I didn't think of that, that is a 'killer app' for rpi and other such devices, or any kind of 'embedded' image really When cross-installing ports, there are certain operations (e.g., updating 'info' database) that can really only be done after the system next boots. I'd like to get this into HEAD in the near future in the hope that I can convince re@ that this is a simple enough (and safe enough) change to merge before 10.0-RELEASE. Please. Tim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org -- Allan Jude ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
On Sun, Oct 13, 2013 at 3:58 PM, Colin Percival cperc...@freebsd.orgwrote: Hi all, I've attached a very simple patch which makes /etc/rc: 1. Skip any rc.d scripts with the firstboot keyword if /var/db/firstboot does not exist, 2. If /var/db/firstboot and /var/db/firstboot-reboot exist after running rc.d scripts, reboot. 3. Delete /var/db/firstboot (and firstboot-reboot) after the first boot. We use something like this at work. However, our version creates a file after the firstboot scripts have run, and doesn't run if the file exists. Is there a reason to prefer one choice over the other? Naively I'd expect it to be better to run when the file doesn't exist, creating when done; it solves the problem of making sure the magic file exists before first boot, for the other polarity. Thanks, matthew ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
On 10/15/13 01:58, Nick Hibma wrote: Indeed... the way my patch currently does things, it looks for the firstboot sentinel at the start of /etc/rc, which means it *has* to be on /. Making the path an rcvar is a good idea (updated patch attached) but we still need some way to re-probe for that file after mounting extra filesystems. In many cases a simple test -f /firstboot bla_enable='YES' || bla_enable='NO' rm -f /firstboot in your specific rc.d script would suffice. [...] I am not quite sure why we need /firstboot handling in /etc/rc. Your suggestion wouldn't work if you have several scripts doing it; the first one would remove the sentinel and the others wouldn't run. In my EC2 code I have a single script which runs after all the others and removes the sentinel file, but that still means that every script has to be executed on every boot (even if just to check if it should do anything); putting the logic into /etc/rc would allow rcorder to skip those scripts entirely. Perhaps it is a better idea to make this more generic, to move the rc.d script containing a 'runonce' keyword to a subdirectory as the last step in rc (or make that an rc.d script in itself!). That way you could consider moving it back if you need to re-run it. Or have an rc.d script setup something like a database after installing a package by creating a rc.d runonce script. Default dir could be ./run-once relative to the rc.d dir it is in, configurable through runonce_directory . Note: The move would need to be done at the very end of rc.d to prevent rcorder returning a different ordering and skipping scripts because of that. I considered this, but decided that the most common requirement use of run once would be for run when the system is first booted, and it would be much simpler to provide just the firstboot functionality. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
On 10/15/13 13:09, Matthew Fleming wrote: We use something like this at work. However, our version creates a file after the firstboot scripts have run, and doesn't run if the file exists. Is there a reason to prefer one choice over the other? Naively I'd expect it to be better to run when the file doesn't exist, creating when done; it solves the problem of making sure the magic file exists before first boot, for the other polarity. I don't see that making sure that the magic file exists is a problem, since you'd also need to make sure you have knobs turned on in /etc/rc.conf and/or extra rc.d scripts installed. In a very marginal sense, deleting a file is safer than creating one, since if the filesystem is full you can delete but not create. It also seems to me that the sensible polarity is that having something extra lying around makes extra things happen rather than inhibiting them. But probably the best argument has to do with upgrading systems -- if you update a 9.2-RELEASE system to 10.1-RELEASE and there's a first boot script in that new release, you don't want to have it accidentally get run simply because you failed to create a /firstboot file during the upgrade process. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
Colin, Sounds useful: We have nanobsd images that configure a hard disk if present, but obviously only need to be run once. However: NanoBSD stores uses a memory disk for /etc and stores it's permanent scripts in /conf/* (/etc/rc.initdiskless) and/or /cfg (NanoBSD) so I doubt whether the 'embedded systems' argument is of much use, as deleting the script or flagging 'firstboot' is non-permanent. Nick Hibma n...@van-laarhoven.org Want to feel like going on a holiday tomorrow? Try GTD. On 14 Oct 2013, at 00:58, Colin Percival cperc...@freebsd.org wrote: Hi all, I've attached a very simple patch which makes /etc/rc: 1. Skip any rc.d scripts with the firstboot keyword if /var/db/firstboot does not exist, 2. If /var/db/firstboot and /var/db/firstboot-reboot exist after running rc.d scripts, reboot. 3. Delete /var/db/firstboot (and firstboot-reboot) after the first boot. The purpose of this is to support run on first boot rc.d scripts. These can be useful for both virtual machines and embedded systems; unlike conventional desktops and servers, these may have a lengthy gap between installing and turning on the system. As examples of what such scripts could do: * In Amazon EC2, I use a first boot script to download an SSH public key from EC2 so that users can log in to newly provisioned EC2 instances. * Now that (starting from 10.0-BETA1) it is possible to use FreeBSD Update to update everything on EC2 instances, I'm planning on writing a script which runs 'freebsd-update fetch install' when the system first boots, and then reboots if there were updates installed. (I imagine this would be useful to other embedded / VM providers too.) * Once packages are provided (properly) for 10.0 I'd like to allow people to specify a list of packages they want installed onto an EC2 instance and have them downloaded and installed when the EC2 instance launches. I'd like to get this into HEAD in the near future in the hope that I can convince re@ that this is a simple enough (and safe enough) change to merge before 10.0-RELEASE. Comments? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid firstboot.patch___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
Colin Percival cperc...@freebsd.org wrote in 525b258f.3030...@freebsd.org: cp I've attached a very simple patch which makes /etc/rc: cp +if ! [ -e /var/db/firstboot ]; then cp + skip=$skip -s firstboot cp +fi At this stage, it is possible that /var/db does not exist because it is before rc.d/mountcritlocal. -- Hiroki pgpuB725TWmQM.pgp Description: PGP signature
Re: RFC: support for first boot rc.d scripts
Hi Nick, On 10/14/13 00:59, Nick Hibma wrote: Sounds useful: We have nanobsd images that configure a hard disk if present, but obviously only need to be run once. However: NanoBSD stores uses a memory disk for /etc and stores it's permanent scripts in /conf/* (/etc/rc.initdiskless) and/or /cfg (NanoBSD) so I doubt whether the 'embedded systems' argument is of much use, as deleting the script or flagging 'firstboot' is non-permanent. Yes, it's hard to store state on diskless systems... but I figured that anyone building a diskless system would know to not create a run firstboot scripts marker. And not all embedded systems are diskless... -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
On 10/14/13 05:07, Hiroki Sato wrote: Colin Percival cperc...@freebsd.org wrote in 525b258f.3030...@freebsd.org: cp I've attached a very simple patch which makes /etc/rc: cp +if ! [ -e /var/db/firstboot ]; then cp + skip=$skip -s firstboot cp +fi At this stage, it is possible that /var/db does not exist because it is before rc.d/mountcritlocal. Ah, good point. I guess we need something on / then? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
On Mon, 2013-10-14 at 09:51 -0700, Colin Percival wrote: Hi Nick, On 10/14/13 00:59, Nick Hibma wrote: Sounds useful: We have nanobsd images that configure a hard disk if present, but obviously only need to be run once. However: NanoBSD stores uses a memory disk for /etc and stores it's permanent scripts in /conf/* (/etc/rc.initdiskless) and/or /cfg (NanoBSD) so I doubt whether the 'embedded systems' argument is of much use, as deleting the script or flagging 'firstboot' is non-permanent. Yes, it's hard to store state on diskless systems... but I figured that anyone building a diskless system would know to not create a run firstboot scripts marker. And not all embedded systems are diskless... The embedded systems we create at $work have readonly root and mfs /var, but we do have writable storage on another filesystem. It would work for us (not that we need this feature right now) if there were an rcvar that pointed to the marker file. Of course to make it work, something would have to get the alternate filesystem mounted early enough to be useful (that is something we do already with a custom rc script). Note that I'm not asking for any changes here, just babbling. -- Ian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
On 10/14/13 10:00, Ian Lepore wrote:
On Mon, 2013-10-14 at 09:51 -0700, Colin Percival wrote:
Yes, it's hard to store state on diskless systems... but I figured
that anyone building a diskless system would know to not create a
run firstboot scripts marker. And not all embedded systems are
diskless...
The embedded systems we create at $work have readonly root and mfs /var,
but we do have writable storage on another filesystem. It would work
for us (not that we need this feature right now) if there were an rcvar
that pointed to the marker file. Of course to make it work, something
would have to get the alternate filesystem mounted early enough to be
useful (that is something we do already with a custom rc script).
Indeed... the way my patch currently does things, it looks for the
firstboot sentinel at the start of /etc/rc, which means it *has* to
be on /. Making the path an rcvar is a good idea (updated patch
attached) but we still need some way to re-probe for that file after
mounting extra filesystems.
Note that I'm not asking for any changes here, just babbling.
Babbling is good. Between us we might babble a useful solution. ;-)
--
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
Index: etc/defaults/rc.conf
===
--- etc/defaults/rc.conf(revision 256432)
+++ etc/defaults/rc.conf(working copy)
@@ -619,6 +619,9 @@
accounting_enable=NO # Turn on process accounting (or NO).
ibcs2_enable=NO # Ibcs2 (SCO) emulation loaded at startup (or NO).
ibcs2_loaders=coff # List of additional Ibcs2 loaders (or NO).
+firstboot_sentinel=/firstboot# Scripts with firstboot keyword are
run if
+ # this file exists. Should be on a R/W filesystem so
+ # the file can be deleted after the boot completes.
# Emulation/compatibility services provided by /etc/rc.d/abi
sysvipc_enable=NO# Load System V IPC primitives at startup (or NO).
Index: etc/rc
===
--- etc/rc (revision 256432)
+++ etc/rc (working copy)
@@ -81,6 +81,9 @@
skip=$skip -s nojailvnet
fi
fi
+if ! [ -e ${firstboot_sentinel} ]; then
+ skip=$skip -s firstboot
+fi
# Do a first pass to get everything up to $early_late_divider so that
# we can do a second pass that includes $local_startup directories
@@ -116,6 +119,13 @@
run_rc_script ${_rc_elem} ${_boot}
done
+if [ -e ${firstboot_sentinel} ]; then
+ rm ${firstboot_sentinel}
+ if [ -e ${firstboot_sentinel}-reboot ]; then
+ rm ${firstboot_sentinel}-reboot
+ kill -INT 1
+ fi
+fi
echo ''
date
exit 0
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RFC: support for first boot rc.d scripts
Hi all,
I've attached a very simple patch which makes /etc/rc:
1. Skip any rc.d scripts with the firstboot keyword if /var/db/firstboot
does not exist,
2. If /var/db/firstboot and /var/db/firstboot-reboot exist after running rc.d
scripts, reboot.
3. Delete /var/db/firstboot (and firstboot-reboot) after the first boot.
The purpose of this is to support run on first boot rc.d scripts. These can
be useful for both virtual machines and embedded systems; unlike conventional
desktops and servers, these may have a lengthy gap between installing and
turning on the system.
As examples of what such scripts could do:
* In Amazon EC2, I use a first boot script to download an SSH public key
from EC2 so that users can log in to newly provisioned EC2 instances.
* Now that (starting from 10.0-BETA1) it is possible to use FreeBSD Update
to update everything on EC2 instances, I'm planning on writing a script which
runs 'freebsd-update fetch install' when the system first boots, and then
reboots if there were updates installed. (I imagine this would be useful
to other embedded / VM providers too.)
* Once packages are provided (properly) for 10.0 I'd like to allow people to
specify a list of packages they want installed onto an EC2 instance and have
them downloaded and installed when the EC2 instance launches.
I'd like to get this into HEAD in the near future in the hope that I can
convince re@ that this is a simple enough (and safe enough) change to merge
before 10.0-RELEASE.
Comments?
--
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
Index: etc/rc
===
--- etc/rc (revision 256432)
+++ etc/rc (working copy)
@@ -81,6 +81,9 @@
skip=$skip -s nojailvnet
fi
fi
+if ! [ -e /var/db/firstboot ]; then
+ skip=$skip -s firstboot
+fi
# Do a first pass to get everything up to $early_late_divider so that
# we can do a second pass that includes $local_startup directories
@@ -116,6 +119,13 @@
run_rc_script ${_rc_elem} ${_boot}
done
+if [ -e /var/db/firstboot ]; then
+ rm /var/db/firstboot
+ if [ -e /var/db/firstboot-reboot ]; then
+ rm /var/db/firstboot-reboot
+ kill -INT 1
+ fi
+fi
echo ''
date
exit 0
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: support for first boot rc.d scripts
On 2013-10-13 18:58, Colin Percival wrote: Hi all, I've attached a very simple patch which makes /etc/rc: 1. Skip any rc.d scripts with the firstboot keyword if /var/db/firstboot does not exist, 2. If /var/db/firstboot and /var/db/firstboot-reboot exist after running rc.d scripts, reboot. 3. Delete /var/db/firstboot (and firstboot-reboot) after the first boot. The purpose of this is to support run on first boot rc.d scripts. These can be useful for both virtual machines and embedded systems; unlike conventional desktops and servers, these may have a lengthy gap between installing and turning on the system. As examples of what such scripts could do: * In Amazon EC2, I use a first boot script to download an SSH public key from EC2 so that users can log in to newly provisioned EC2 instances. * Now that (starting from 10.0-BETA1) it is possible to use FreeBSD Update to update everything on EC2 instances, I'm planning on writing a script which runs 'freebsd-update fetch install' when the system first boots, and then reboots if there were updates installed. (I imagine this would be useful to other embedded / VM providers too.) * Once packages are provided (properly) for 10.0 I'd like to allow people to specify a list of packages they want installed onto an EC2 instance and have them downloaded and installed when the EC2 instance launches. I'd like to get this into HEAD in the near future in the hope that I can convince re@ that this is a simple enough (and safe enough) change to merge before 10.0-RELEASE. Comments? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org This looks extremely useful. Thank you. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
