Re: [Bug 350936] Re: Should shut down domains on system shutdown
On Fri, 2011-03-18 at 11:27 +, gollum53 wrote: Hi, I have installed the latest mentioned files here (sendsigs, omit-kvm-vm-pid, libvirt-bin.conf with 'break' command suggested by Martin Rusko) on our Lucid 64bit. servers, and it did not help. When doing a reboot, the system does not wait for the domains to perform a clean shutdown...is there anyone here with working configuration? What could be wrong? Can you please post your files? Thanx Roman Smid I'd be interested to hear if adding this to libvirt-bin.conf would work: pre-stop script if [ x$UPSTART_EVENTS = xrunlevel ] ; then shutdown_guests fi end script Obviously the 'shutdown_guests' command needs to be a single command which stops the guests and waits for them to die. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. https://bugs.launchpad.net/bugs/350936 Title: Should shut down domains on system shutdown -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
The problem with the upstart solution is that it doesn't work since Upstart will kill the virsh instances that you spawn in shutdown_guests. So far the only somewhat satisfactory solution is molly-guard. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. https://bugs.launchpad.net/bugs/350936 Title: Should shut down domains on system shutdown -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
On Fri, Mar 18, 2011 at 05:13:55PM -, Lars Hansson wrote: The problem with the upstart solution is that it doesn't work since Upstart will kill the virsh instances that you spawn in shutdown_guests. That's only the case if you don't follow the instructions to patch sendsigs to avoid that bug: #44: John Morrissey wrote on 2010-09-17: Finally, this modified upstart job requires the fix for https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/639940. Otherwise, the libvirt-bin pre-stop script isn't guaranteed to finish successfully, since sendsigs races the child processes executed during the course of the job script. john -- John Morrissey _o/\ __o j...@horde.net_- \_ / \ \, www.horde.net/__(_)/_(_)/\___(_) /_(_)__ -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. https://bugs.launchpad.net/bugs/350936 Title: Should shut down domains on system shutdown -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
On Fri, 2011-03-18 at 17:13 +, Lars Hansson wrote: The problem with the upstart solution is that it doesn't work since Upstart will kill the virsh instances that you spawn in shutdown_guests. So far the only somewhat satisfactory solution is molly-guard. Right, so you would need shutdown_guests to block until everything is in fact shut down: while box in `virsh list --all | grep '^ -'|grep -qv 'shut off'` ; do virsh shutdown all sleep 1 done Note that currently libvirt-bin.conf has stop on runlevel [!2345] There is a bug in sendsigs that does not wait for these upstart jobs to die before moving on to the end of the shutdown... so really, we also need to fix that bug before the technique above will work. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. https://bugs.launchpad.net/bugs/350936 Title: Should shut down domains on system shutdown -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
RE: [Bug 350936] Re: Should shut down domains on system shutdown
¿? Lenguage spanisch ?? Date: Wed, 2 Feb 2011 00:22:03 + From: cl...@fewbar.com To: jlguti...@hotmail.com Subject: [Bug 350936] Re: Should shut down domains on system shutdown Oops, just noticed, that should be $UPSTART_STOP_EVENTS -- You received this bug notification because you are subscribed to Ubuntu ubuntu-10.04.2. https://bugs.launchpad.net/bugs/350936 Title: Should shut down domains on system shutdown Status in “kvm” package in Ubuntu: Invalid Status in “libvirt” package in Ubuntu: Triaged Bug description: I'm using autostart feature of libvirtd (my guest defined in /etc/libvirt/qemu/autostart/). It's starts normally, but if I reboot server then guest fs will be corrupted becouse its cant stop in time: WARNING: / was not properly dismounted We need to give more time to libvirtd to stop all guests. I'm running jaunty amd64 with all updates. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to kvm in ubuntu. https://bugs.launchpad.net/bugs/350936 Title: Should shut down domains on system shutdown -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
RE: [Bug 350936] Re: Should shut down domains on system shutdown
¿? LENGUAGE SPANISCH Date: Tue, 1 Feb 2011 22:35:00 + From: cl...@fewbar.com To: jlguti...@hotmail.com Subject: [Bug 350936] Re: Should shut down domains on system shutdown Wow this one has been around a while. The simplest way to run a script before stopping, only on shutdown, but not on restarts, is to use the UPSTART_EVENTS. environment variable. pre-stop script if [ x$UPSTART_EVENTS = xrunlevel ] ; then shutdown_guests fi end script -- You received this bug notification because you are subscribed to Ubuntu ubuntu-10.04.2. https://bugs.launchpad.net/bugs/350936 Title: Should shut down domains on system shutdown Status in “kvm” package in Ubuntu: Invalid Status in “libvirt” package in Ubuntu: Triaged Bug description: I'm using autostart feature of libvirtd (my guest defined in /etc/libvirt/qemu/autostart/). It's starts normally, but if I reboot server then guest fs will be corrupted becouse its cant stop in time: WARNING: / was not properly dismounted We need to give more time to libvirtd to stop all guests. I'm running jaunty amd64 with all updates. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to kvm in ubuntu. https://bugs.launchpad.net/bugs/350936 Title: Should shut down domains on system shutdown -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
Don't think so. It's part of the independence of libvirt from the the actual virtualization technology (KVM, ...) to permit updates without shutdown of guests. 2010/11/4 Valentijn Sessink valent...@sessink.nl: Hmm. If libvirt shuts down the virtual machines it monitors, then these virtual machines will also be shut down when updating libvirt. -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
On Thu, Nov 04, 2010 at 11:43:10AM -, Valentijn Sessink wrote: The proposals in this report (mine, John Morrissey's) all have a pre- stop shutdown of virtual machines in their Upstart conf-file. When you update libvirt-bin, dpkg will stop libvirt-bin before upgrading. As a result — if you have implemented the pre-stop shutdown — your virtual machines will all stop. Sorry, that's incorrect. The libvirt-bin upstart job I wrote will only shut down VMs if the physical machine is entering runlevels 0 or 6 (i.e., shutdown or reboot): -- pre-stop script if [ -z $RUNLEVEL ]; then exit 0 fi if [ $RUNLEVEL -ne 0 ] [ $RUNLEVEL -ne 6 ]; then exit 0 fi -- john -- John Morrissey _o/\ __o j...@horde.net_- \_ / \ \, www.horde.net/__(_)/_(_)/\___(_) /_(_)__ -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
On Sat, Oct 02, 2010 at 02:54:47AM -, Nathan Crawford wrote: FWIW, you might want to mark /etc/default/libvirt-bin as a conffile in the libvirt packaging. I think it already is? (dpkg --status libvirt-bin says it is... but I've been changing things all over the place so I might have confused it) Yes, my fault. debhelper probably adds it as a conffile automatically. Also, are these changes based on Lucid packages? The Lucid machine I'm using for testing has a kernel panic on shutdown with the packages from my PPA installed. Yes, I'm running these changes on a number of lucid machines. Maybe the changed behavior triggers something in the kernel that's causing a panic? I can't really say without seeing the full panic output. john -- John Morrissey _o/\ __o j...@horde.net_- \_ / \ \, www.horde.net/__(_)/_(_)/\___(_) /_(_)__ -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
On Fri, Oct 01, 2010 at 03:47:07AM -, Nathan Crawford wrote:
@John Morrissey - Nice work on getting this working! I hope you don't
mind, I've taken the liberty of packaging all your changes and putting
them in a PPA for easier testing.
Thanks, and packaging the changes is no problem at all.
FWIW, you might want to mark /etc/default/libvirt-bin as a conffile in the
libvirt packaging. Also, it might make more sense to package
omit-kvm-vm-pids as part of libvirt instead of a separate package; makes it
harder to forget to install it.
One question on omit-kvm-vm-pids from #51, what does the '##*/' mean
toward the end of line 26?
It's a shell construct that removes text at the beginning of a parameter. In
this case, the net effect is to remove the leading pathname, so all that's
left is the filename ('##' is longest match, and the pattern is '*/'). Since
it's a shell builtin, it's faster than calling basename(1), although speed
is arguably important here. :-)
Looking at it again, that for loop should probably have a guard so it
gracefully handles the case where no QEMU VMs are running:
for pidfile in /var/run/libvirt/qemu/*.pid; do
if [ $pidfile = '/var/run/libvirt/qemu/*.pid' ]; then
break
fi
if ! cp -f $pidfile
/lib/init/rw/sendsigs.omit.d/libvirt-bin-${pidfile##*/}; then
status=1
fi
done
john
--
John Morrissey _o/\ __o
j...@horde.net_- \_ / \ \,
www.horde.net/__(_)/_(_)/\___(_) /_(_)__
--
Should shut down domains on system shutdown
https://bugs.launchpad.net/bugs/350936
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in ubuntu.
--
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
Clap! :) 2010/9/29 Andy a...@xillean.com: It is a dismal situation when at this point there is still no official simple way from Canonical to gracefully shutdown KVM VMs if the host has to go down for any reason such as a UPS issued shutdown command. -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
On Wed, Sep 22, 2010 at 09:47:47AM -, Valentijn Sessink wrote: - if the shutdown script searches for the word running (like mine does, - but John Morrissey's script is affected, too), it must set LANG=C at the - top, otherwise virsh will run in the local language. Good catch. I've updated libvirt-bin.conf; let's see if LP accepts attachments via e-mail. john -- John Morrissey _o/\ __o j...@horde.net_- \_ / \ \, www.horde.net/__(_)/_(_)/\___(_) /_(_)__ ** Attachment added: libvirt-bin.conf https://bugs.launchpad.net/bugs/350936/+attachment/1622533/+files/libvirt-bin.conf ** Attachment added: omit-kvm-vm-pids https://bugs.launchpad.net/bugs/350936/+attachment/1622534/+files/omit-kvm-vm-pids -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
2010/9/17 John Morrissey j...@horde.net: The attached file updates libvirt-bin's upstart job to gracefully shut down VMs when the system shuts down. You can alter the shutdown timeout (SHUTDOWN_TIMEOUT, default: 300s) and list of virsh(1) URIs to use (URIS, default: qemu:///system) in /etc/default/libvirt-bin. While I am beginning to hate upstart because of its deficiencies (#494141, #406397), I don't think adding a sysvinit script to fix an upstart job is the best workaround. Just handle pid omits in sendsigs as a special case for libvirt, as done for other problems. -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
Andy, I didn't try your suggestion (yet), but is it really necessary to create the kvmguests init script? Can't you just place the script inside the pre-stop script section of /etc/init/libvirt-bin.conf? Moreover, I've noticed that your kvmguests script uses upstart's 'start' command to ensure that libvirtd is actually running. is that really necessary? AFAIUI this can be dropped if integrated in libvirt-bin.conf As for your change to /etc/init.d/sendsigs, wouldn't it be less intrusive to make /etc/init/libvirt-bin.conf or probably libvirtd itself drop or copy the pid files into /lib/init/rw/sendsigs.omit.d? AFAIUI, the upstart job should be executed before the sendsigs init script, or is there another race here? -- Gruesse/greetings, Reinhard Tartler, KeyID 945348A4 -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
I have seen the commit that implements this functionality and it's, as usual, a sysvinit script. Unfortunately, I think this is a use case where upstart can't do anything yet and the only solution (without moving back to a sysvinit script) is doing an ugly workaround in /etc/init.d/sendsigs. I've tried to subscribe Scott James Remnant (maintainer of upstart: thanks!): this is not a bug in upstart (maybe a feature lack) but I hope he have something to share with us about this problem. 2010/8/6 Sergey Svishchev 350...@bugs.launchpad.net: This is implemented in libvirt 0.8.2: https://bugzilla.redhat.com/show_bug.cgi?id=444273 ** Bug watch added: Red Hat Bugzilla #444273 https://bugzilla.redhat.com/show_bug.cgi?id=444273 -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
instead of destroying vms, wouldn't be a more elegant solution to save the state of the VM in a statefile? On bootup, upstart would check for such statefiles and restore the VMs. This would work even for VMs that don't react to ACPI events properly and saves the VM's uptimes. -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
Why are you not doing this in the pre-stop part of the libvirt-bin job, by the way? -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
On Fri, Jun 25, 2010 at 10:32:07AM -, ossjunkie wrote: @soren: as i simple don't know how to trigger it only on shutdown, as the premise was it should run on regular stop. but i know it may ease the situation and would like to do so. do you know some solution here. maybe we could do a simple condition that checks for an enviromental stuff thats only present on shutdown/reboot and is a reliable source. and yes i left some ugly debugging stuff in, but because it isn't ready so far ;) I actually had a modified version of libvirt's init script that did what you're doing. It checked whether $0 was called K??libvirt-bin, in which case it was being called as part of a runlevel change. The equivalent in the world of Upstart is probably something like the UPSTART_EVENTS environment variable. It will not be set if you're running stop libvirt-bin, but will be set if the libvirt-bin job is being stopped as a result of an Upstart event. regarding the killing by /etc/init.d/sendsings we could easily test when the VMs got killed and how to prevent when libvirtd would be still accessable. so i would say we should get the upstart things right first and then go on with that. maybe we could try to track and bound the pids of the guest somehow to the upstart job (in case we keep it seperate) and prevent it that way. libvirt makes sure qemu creates pidfiles for kvm processes anyway. It shouldn't be hard at all to make sendsigs omit them from its killing spree. -- Soren Hansen Ubuntu Developer http://www.ubuntu.com/ -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
On Thu, Jun 24, 2010 at 01:38:04PM -, ossjunkie wrote: thanks for the hint, but after trying i found that start on (runlevel [06] and stopping libvirt-bin) should do the trick. but i still got problems as i can use libvirt at first in the upstart job, but when testing the timeout loop it seems that libvirt-bin still stopped before the script finished. That shouldn't happen. You're doing the right thing with upstart. I double checked with one of the experts. :) Something else seems to be killing libvirt-bin. Try putting a a post-stop thing in the libvirt-bin job to log when it's being killed, just to double check. why does libvirt-bin still stop before the libvirt-shutdown-guests job has finished the script? Good question. Upstart shouldn't be killing it, but something else might. I can't imagine what, though. do i have to modify the libvirt-bin job to wait for libvirt-shutdown-guests? Nope. does upstart jobs also receive some sort of kill signal? does upstart in general allow to delay the shutdown just by a loop in the script of a upstart job? Yes. BTW i have no problems when testing it with initctl start libvirt- shutdown-guests, so any testing needs to done on real shutdown ;( ** Attachment added: libvirt-shutdown-guests.conf http://launchpadlibrarian.net/50850625/libvirt-shutdown-guests.conf The script still has some debugging stuff in it. That would need to go away before we can include this in the package. Thanks for your efforts so far! This has been a problem for a looong time. -- Soren Hansen Ubuntu Developer http://www.ubuntu.com/ -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
2010/6/24 Soren Hansen so...@ubuntu.com: why does libvirt-bin still stop before the libvirt-shutdown-guests job has finished the script? Good question. Upstart shouldn't be killing it, but something else might. I can't imagine what, though. I'm not sure Upstart isn't supposed to kill them. Let's suppose upstart is able to track all pids forked for livbvirt-bin job start (and it doesn't do so, even if the expect daemon stanza is used: this may be an upstart bug): there would be a pid for libvirt-bin and many pids for kvm guests. Suppose you want to kill the libvirt-bin job: how can you tell upstart to kill just libvirt-bin and not kvm guests? I don't think upstart jobs are so flexible. -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 350936] Re: Should shut down domains on system shutdown
On Wed, Jun 23, 2010 at 03:53:18PM -, ossjunkie wrote: -how can we ensure in upstart that the script got run on shutdown reboots but always before libvirt-bin and qemu-kvm got stopped? I /think/ this should do the trick: start on shutdown and stopping libvirt-bin IIUIC, libvirt-bin will block its stop procedure until your script is done. -- Soren Hansen Ubuntu Developer http://www.ubuntu.com/ -- Should shut down domains on system shutdown https://bugs.launchpad.net/bugs/350936 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
