Re: starting jails in the background & dependencies
On Tue, 5 Jan 2010 11:24:47 +0100 Alexander Leidinger wrote: > On Mon, 07 Dec 2009 08:03:53 +0100 Alexander Leidinger > wrote: > > > Hi, > > > > now that jails are started in the background (which is good, to > > I just realized yesterday that it also stops in parallel (in the > background). This is bad. It may be the case that a jail is not fully > stopped via the rc scripts when the OS decides to kill the remaining > processes during a shutdown. > > My first reaction is to only allow to start in the background, but > everything else needs to be serialized. I committed now what was discussed in this thread: - no start in the background by default - only start is allowed to happen in background when jail_parallel_start is set to yes in rc.conf - stdin is redirected from /dev/null If someone is not happy about the name of the rc.conf variable: feel free to change it, I do not care about the name. Bye, Alexander. ___ freebsd-jail@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: starting jails in the background & dependencies
Quoting Remko Lodder (from Mon, 25 Jan 2010 07:44:10 +0100): Note that I haven't tsted it, but I don't see any errors in the patch. ---snip--- -- Simon L. Nielsen Snipping a whole lot of data... Thanks Simon, I will try to get to that as soon as possible, Alexander: please feel free to do it earlier if possible, my internet access is "limited" (or at least commit capabilities). I have this running as I posted it. I can confirm that the jail_parallel_start=no works as expected. I didn't try the YES case. I am not happy about my man page change. Anyone with a better description? We do not start the jails in parallel, we start the jails serially in the background. I think the variable name is ok, as we start the jails in parallel to the rest of the system start scripts. I do not want to limit the wording so that it prevents to really start the jails in parallel instead of serially in the background, while still telling that it is done in parallel to the rest of the scripts. If I get some time today, I will think about a better wording (if I do not get something from the people reading this before). Bye, Alexander. -- There's nothing remarkable about it. All one has to do is hit the right keys at the right time and the instrument plays itself. -- J. S. Bach http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-jail@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: starting jails in the background & dependencies
> Note that I haven't tsted it, but I don't see any errors in the patch. > >> ---snip--- > -- > Simon L. Nielsen > Snipping a whole lot of data... Thanks Simon, I will try to get to that as soon as possible, Alexander: please feel free to do it earlier if possible, my internet access is "limited" (or at least commit capabilities). Thanks, Remko -- /"\ Best regards, | re...@freebsd.org \ / Remko Lodder | re...@efnet Xhttp://www.evilcoder.org/ | / \ ASCII Ribbon Campaign | Against HTML Mail and News ___ freebsd-jail@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: starting jails in the background & dependencies
B0;251;0cOn 2010.01.14 13:35:16 +0100, Alexander Leidinger wrote:
> Quoting Remko Lodder (from Tue, 5 Jan 2010
> 11:35:48 +0100):
>
> > On Tue, January 5, 2010 11:24 am, Alexander Leidinger wrote:
> >> On Mon, 07 Dec 2009 08:03:53 +0100 Alexander Leidinger
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> now that jails are started in the background (which is good, to
> >>
> >> I just realized yesterday that it also stops in parallel (in the
> >> background). This is bad. It may be the case that a jail is not fully
> >> stopped via the rc scripts when the OS decides to kill the remaining
> >> processes during a shutdown.
> >>
> >> My first reaction is to only allow to start in the background, but
> >> everything else needs to be serialized.
> >>
> >> Any objections or better ideas out there?
>
> > I think the best way at this moment is to revert the change ( I can do
> > that , or someone else, I dont mind ) and think of a better concept. Simon
> > also mentioned that he didn't like the current way of doing things, so I
> > kept it in, for possible suggestions. Reverting the change would mean that
> > the old behaviour at least works and is with what people are used to. We
> > can then further improve it where needed.
>
> What about the following? Just have a look at the principle, I haven't
> tested it yet. What it does is:
> - revert back to serial startup by default
> - allow to only start in the background (jail_parallel_start=YES)
In some thread there was talk about parallel stop as well, but I must
admit I never looked at it.
> - take input from /dev/null: in case a start script inside the
> jail wants to read from stdin (it shouldn't), it will not
> switch the process into STOP state (but should generate some
> message in the application log)
This seems like a fine change - especially since the output from the
actual jail is hidden.
> Copy&paste, so maybe messed up tabs:
The bottom part of rc.d/jail after the patch seems well, "messy"
in lack of a better word, but since I can't come up with a better
solution right now I think this patch should be committed, and then we
can always improve the implementation later.
Note that I haven't tsted it, but I don't see any errors in the patch.
> ---snip---
> Index: share/man/man5/rc.conf.5
> ===
> --- share/man/man5/rc.conf.5(Revision 202277)
> +++ share/man/man5/rc.conf.5(Arbeitskopie)
> @@ -24,7 +24,7 @@
> .\"
> .\" $FreeBSD$
> .\"
> -.Dd November 11, 2009
> +.Dd January 14, 2010
> .Dt RC.CONF 5
> .Os
> .Sh NAME
> @@ -3472,6 +3472,11 @@
> If set to
> .Dq Li NO ,
> any configured jails will not be started.
> +.It jail_parallel_start
> +.Pq Vt bool
> +If set to
> +.Dq Li YES
> +all configured jails will be started in the background (= in parallel).
> .It Va jail_list
> .Pq Vt str
> A space separated list of names for jails.
> Index: etc/rc.d/jail
> ===
> --- etc/rc.d/jail (Revision 202277)
> +++ etc/rc.d/jail (Arbeitskopie)
> @@ -636,7 +636,8 @@
> done
>
> eval ${_setfib} jail ${_flags} -i ${_rootdir} ${_hostname} \
> - \"${_addrl}\" ${_exec_start} > ${_tmp_jail} 2>&1
> + \"${_addrl}\" ${_exec_start} > ${_tmp_jail} 2>&1 \
> +
> if [ "$?" -eq 0 ] ; then
> _jail_id=$(head -1 ${_tmp_jail})
> @@ -728,4 +729,19 @@
> if [ -n "$*" ]; then
> jail_list="$*"
> fi
> -run_rc_command "${cmd}" &
> +
> +# Only allow the parallel start of jails, other commands are not
> +# safe to execute in parallel.
> +case "${cmd}" in
> +*start)
> + ;;
> +*)
> + jail_parallel_start=NO
> +esac
> +
> +if checkyesno jail_parallel_start; then
> + run_rc_command "${cmd}" &
> +else
> + run_rc_command "${cmd}"
> +fi
> +
> Index: etc/defaults/rc.conf
> ===
> --- etc/defaults/rc.conf(Revision 202277)
> +++ etc/defaults/rc.conf(Arbeitskopie)
> @@ -630,6 +630,7 @@
> ### Jail Configuration ###
> ##
> jail_enable="NO" # Set to NO to disable starting of any jails
> +jail_parallel_start="NO" # Start jails in the background
> jail_list="" # Space separated list of names of jails
> jail_set_hostname_allow="YES" # Allow root user in a jail to change
> its hostname
> jail_socket_unixiproute_only="YES" # Route only TCP/IP within a jail
> ---snip---
--
Simon L. Nielsen
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: starting jails in the background & dependencies
Alexander Leidinger wrote: Quoting Miroslav Lachman <000.f...@quip.cz> (from Tue, 05 Jan 2010 11:45:34 +0100): Alexander Leidinger wrote: On Mon, 07 Dec 2009 08:03:53 +0100 Alexander Leidinger wrote: Hi, now that jails are started in the background (which is good, to I just realized yesterday that it also stops in parallel (in the background). This is bad. It may be the case that a jail is not fully stopped via the rc scripts when the OS decides to kill the remaining processes during a shutdown. My first reaction is to only allow to start in the background, but everything else needs to be serialized. Any objections or better ideas out there? Maybe stopping can be done in parallel, but rc script should wait (in loop) until all jails are stopped or some configurable timeout (for example 60 seconds). Feel free to come up with a proof of concept... but the timeout on stop should be "forever" IMO. If you have a busy software which needs to be shutdown correctly for data safety or consistency reasons, I do not want that a reboot or shutdown prevents the correct shutdown. I misunderstand the whole thing from the begining. It's all about wording "background" and "parallel". My first understanding was if I have 4 jails, they are started in parallel (each other) something like: for J in jail1 jail2 jail3 jail4 do jail_start $J & done and similar for stoping them. But now I see that it is just a start jails in serial as usual but rc.d/jail runs in the background, so next rc script will start right after rc.d/jail, not waiting to jails come up. Both approaches have its pros and cons. In the first case (starting and stopping each jail in the background) stopping can be easy as: for J in $jail_list do jail_stop $J & done while [ -n "`jls`" ] do sleep 1 done echo "all jails were stopped" For the second case, where jails are started / stopped as usual but whole rc.d/jail is backgrounded the only solution I got in my mind is the second rc script (for example bgjail_stop) with similar loop as above executed as one of the last rc scripts on system shutdown. (but I know it is ugly solution) I hope somebody will come with better idea :) Miroslav Lachman PS: as my english is not so well, it is sometimes hard to me to understand and sometimes hard to explain things ___ freebsd-jail@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: starting jails in the background & dependencies
Quoting Miroslav Lachman <000.f...@quip.cz> (from Tue, 05 Jan 2010 11:45:34 +0100): Alexander Leidinger wrote: On Mon, 07 Dec 2009 08:03:53 +0100 Alexander Leidinger wrote: Hi, now that jails are started in the background (which is good, to I just realized yesterday that it also stops in parallel (in the background). This is bad. It may be the case that a jail is not fully stopped via the rc scripts when the OS decides to kill the remaining processes during a shutdown. My first reaction is to only allow to start in the background, but everything else needs to be serialized. Any objections or better ideas out there? Maybe stopping can be done in parallel, but rc script should wait (in loop) until all jails are stopped or some configurable timeout (for example 60 seconds). Feel free to come up with a proof of concept... but the timeout on stop should be "forever" IMO. If you have a busy software which needs to be shutdown correctly for data safety or consistency reasons, I do not want that a reboot or shutdown prevents the correct shutdown. Bye, Alexander. -- Beware of Programmers who carry screwdrivers. -- Leonard Brandwein http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-jail@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: starting jails in the background & dependencies
Quoting Remko Lodder (from Tue, 5 Jan 2010
11:35:48 +0100):
On Tue, January 5, 2010 11:24 am, Alexander Leidinger wrote:
On Mon, 07 Dec 2009 08:03:53 +0100 Alexander Leidinger
wrote:
Hi,
now that jails are started in the background (which is good, to
I just realized yesterday that it also stops in parallel (in the
background). This is bad. It may be the case that a jail is not fully
stopped via the rc scripts when the OS decides to kill the remaining
processes during a shutdown.
My first reaction is to only allow to start in the background, but
everything else needs to be serialized.
Any objections or better ideas out there?
I think the best way at this moment is to revert the change ( I can do
that , or someone else, I dont mind ) and think of a better concept. Simon
also mentioned that he didn't like the current way of doing things, so I
kept it in, for possible suggestions. Reverting the change would mean that
the old behaviour at least works and is with what people are used to. We
can then further improve it where needed.
What about the following? Just have a look at the principle, I haven't
tested it yet. What it does is:
- revert back to serial startup by default
- allow to only start in the background (jail_parallel_start=YES)
- take input from /dev/null: in case a start script inside the
jail wants to read from stdin (it shouldn't), it will not
switch the process into STOP state (but should generate some
message in the application log)
Copy&paste, so maybe messed up tabs:
---snip---
Index: share/man/man5/rc.conf.5
===
--- share/man/man5/rc.conf.5(Revision 202277)
+++ share/man/man5/rc.conf.5(Arbeitskopie)
@@ -24,7 +24,7 @@
.\"
.\" $FreeBSD$
.\"
-.Dd November 11, 2009
+.Dd January 14, 2010
.Dt RC.CONF 5
.Os
.Sh NAME
@@ -3472,6 +3472,11 @@
If set to
.Dq Li NO ,
any configured jails will not be started.
+.It jail_parallel_start
+.Pq Vt bool
+If set to
+.Dq Li YES
+all configured jails will be started in the background (= in parallel).
.It Va jail_list
.Pq Vt str
A space separated list of names for jails.
Index: etc/rc.d/jail
===
--- etc/rc.d/jail (Revision 202277)
+++ etc/rc.d/jail (Arbeitskopie)
@@ -636,7 +636,8 @@
done
eval ${_setfib} jail ${_flags} -i ${_rootdir} ${_hostname} \
- \"${_addrl}\" ${_exec_start} > ${_tmp_jail} 2>&1
+ \"${_addrl}\" ${_exec_start} > ${_tmp_jail} 2>&1 \
+jail_set_hostname_allow="YES" # Allow root user in a jail to change
its hostname
jail_socket_unixiproute_only="YES" # Route only TCP/IP within a jail
---snip---
Bye,
Alexander.
--
For certain people, after fifty, litigation takes the place of sex.
-- Gore Vidal
http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: starting jails in the background & dependencies
On 1/5/10 5:35 AM, Remko Lodder wrote: My first reaction is to only allow to start in the background, but everything else needs to be serialized. i second that 'start in parallel', stop in serial, however, even with stop in serial, if I have 64 jails, even in a fast, quad/quad core system, I find that I stop jails prior to reboot/shutdown. even at that, for some reason, mysql doesn't always stop. in reboot, it does take a LONG time for them to all come up. -- Michael Scheidell, CTO Phone: 561-999-5000, x 1259 > *| *SECNAP Network Security Corporation * Certified SNORT Integrator * 2008-9 Hot Company Award Winner, World Executive Alliance * Five-Star Partner Program 2009, VARBusiness * Best Anti-Spam Product 2008, Network Products Guide * King of Spam Filters, SC Magazine 2008 _ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.spammertrap.com _ ___ freebsd-jail@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: starting jails in the background & dependencies
On Tue, January 5, 2010 11:24 am, Alexander Leidinger wrote: > On Mon, 07 Dec 2009 08:03:53 +0100 Alexander Leidinger > wrote: > >> Hi, >> >> now that jails are started in the background (which is good, to > > I just realized yesterday that it also stops in parallel (in the > background). This is bad. It may be the case that a jail is not fully > stopped via the rc scripts when the OS decides to kill the remaining > processes during a shutdown. > > My first reaction is to only allow to start in the background, but > everything else needs to be serialized. > > Any objections or better ideas out there? > > Bye, > Alexander. > I think the best way at this moment is to revert the change ( I can do that , or someone else, I dont mind ) and think of a better concept. Simon also mentioned that he didn't like the current way of doing things, so I kept it in, for possible suggestions. Reverting the change would mean that the old behaviour at least works and is with what people are used to. We can then further improve it where needed. Cheerio, Remko -- /"\ Best regards, | re...@freebsd.org \ / Remko Lodder | re...@efnet Xhttp://www.evilcoder.org/ | / \ ASCII Ribbon Campaign | Against HTML Mail and News ___ freebsd-jail@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: starting jails in the background & dependencies
Alexander Leidinger wrote: On Mon, 07 Dec 2009 08:03:53 +0100 Alexander Leidinger wrote: Hi, now that jails are started in the background (which is good, to I just realized yesterday that it also stops in parallel (in the background). This is bad. It may be the case that a jail is not fully stopped via the rc scripts when the OS decides to kill the remaining processes during a shutdown. My first reaction is to only allow to start in the background, but everything else needs to be serialized. Any objections or better ideas out there? Maybe stopping can be done in parallel, but rc script should wait (in loop) until all jails are stopped or some configurable timeout (for example 60 seconds). Miroslav Lachman ___ freebsd-jail@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: starting jails in the background & dependencies
On Mon, 07 Dec 2009 08:03:53 +0100 Alexander Leidinger wrote: > Hi, > > now that jails are started in the background (which is good, to I just realized yesterday that it also stops in parallel (in the background). This is bad. It may be the case that a jail is not fully stopped via the rc scripts when the OS decides to kill the remaining processes during a shutdown. My first reaction is to only allow to start in the background, but everything else needs to be serialized. Any objections or better ideas out there? Bye, Alexander. ___ freebsd-jail@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
starting jails in the background & dependencies
Hi, now that jails are started in the background (which is good, to prevent that a broken jail causes a good jail not to start), I have to problem how to express dependencies. Scenario: - several jails on the same machine (via ezjail) - one jail depends on the services of another jail, e.g. - jail0 with a DNS server - jailA with mysql (requires that jail0 is up) - jailB needs access to the mysql of the jailA (and DNS of jail0) Currently all jails are started in parallel. This may lead to a situation where something in jailB wants to access jailA before mysql is available. In my case I have the special condition that I need to run a script (rc.d) on the jail-host, after two specific jails are started: - I have a good PROVIDE line in /usr/local/etc/ezjail/jailA (and jail0) - I have a corresponding REQUIRE line in /usr/local/etc/ezjail/jailB (and jailA for jail0) - rc.d/ezjail is called before my script (it hardlinks the mysql socket into jailB) - the jails are started in parallel -> rc.d/ezjail finishes before mysql is started - my link-script starts before mysql is up -> no DB connection possible from jailB (configured to use the unix domain socket) In my case it would be OK to block the start of everything if one jail starts, so the easy solution would be to introduce a jail_background_start variable (default: yes). Does someone have a better idea how to solve this? If not, any objections against the jail_background_start solution? Bye, Alexander. -- Sanity and insanity overlap a fine grey line. http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-jail@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
