Re: Detect of BHyve VM was powered off or rebooted?
On Tue, Feb 11, 2014 at 3:18 PM, Neel Natu neeln...@gmail.com wrote: One question, if bhyve exits, do I have to call bhyveload again before calling bhyve? Yes, that is correct. You will also need to destroy the VM after bhyve exits. Why can't bhyve clean up after itself as the bhyve process exits and destroy the VM? It seems pointless to me for the user to have to explictly destroy the VM. Once the bhyve process exits, I doesn't seem that I can use the VM again anyways. To run the VM again, I seem to need to call: - bhyvectl to destroy the VM - bhyveload to load - bhyve to run the VM Aryeh has pointed out to me that it is the responsibility of the user to explictly destroy and tap and bridge devices used by the bhyve process after it exits. Is this right? What if I have a single bridge0 device, and 10 tap devices, one per VM? If one VM exits and needs to be restarted, do I need to tear down all the taps and bridge? -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
On Wed, Feb 12, 2014 at 4:28 AM, Craig Rodrigues rodr...@freebsd.orgwrote: On Tue, Feb 11, 2014 at 3:18 PM, Neel Natu neeln...@gmail.com wrote: One question, if bhyve exits, do I have to call bhyveload again before calling bhyve? Yes, that is correct. You will also need to destroy the VM after bhyve exits. Why can't bhyve clean up after itself as the bhyve process exits and destroy the VM? It seems pointless to me for the user to have to explictly destroy the VM. Once the bhyve process exits, I doesn't seem that I can use the VM again anyways. To run the VM again, I seem to need to call: - bhyvectl to destroy the VM - bhyveload to load - bhyve to run the VM Aryeh has pointed out to me that it is the responsibility of the user to explictly destroy and tap and bridge devices used by the bhyve process after it exits. Is this right? What if I have a single bridge0 device, and 10 tap devices, one per VM? If one VM exits and needs to be restarted, do I need to tear down all the taps and bridge? Yes you do. It gets really harry when you have multple vm's and taps per instance that need to start and stop at arbitrary times. We are currently disccusing this issue on the petitecloud mailing list it might be off topic here to get into too much detail. -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
Hi Craig, It seems pointless to me for the user to have to explictly destroy the VM. We're working on fixing it. What it gave during the development process (and still gives) is the ability to get post-mortem information when the bhyve process exits. Once the bhyve process exits, I doesn't seem that I can use the VM again anyways. To run the VM again, I seem to need to call: - bhyvectl to destroy the VM - bhyveload to load - bhyve to run the VM Yes, that's correct. The future version will be: /usr/sbin/bhyve to load/run/reboot the VM in a single process. Aryeh has pointed out to me that it is the responsibility of the user to explictly destroy and tap and bridge devices used by the bhyve process after it exits. Is this right? Only if you won't be using them anymore. What if I have a single bridge0 device, and 10 tap devices, one per VM? If one VM exits and needs to be restarted, do I need to tear down all the taps and bridge? No - leave them there. Think of the tap interfaces as ports on a switch, and the bhyve processes as being plugged/unplugged from the ports. later, Peter. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
On Wed, Feb 12, 2014 at 10:28 AM, Peter Grehan gre...@freebsd.org wrote: No - leave them there. Think of the tap interfaces as ports on a switch, and the bhyve processes as being plugged/unplugged from the ports. Every time I have tried this it has lead to bricking the bridge if done in the wrong order and/or there was a packet in transit on the bridge while you have open end points -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
On Sunday, February 09, 2014 7:03:41 pm Neel Natu wrote:
Hi Craig,
On Sun, Feb 9, 2014 at 1:11 PM, Craig Rodrigues rodr...@freebsd.org wrote:
Hi,
I posted some rc.d scripts that I am using to boot a BHyve VM
and send the output to a serial console using the /dev/nmdm
driver:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-
January/002040.html
It works quite well. There is some things I would like to improve,
and would like some advice on the best way to do it.
(1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
then I do not want to automatically restart the VM in the script.
User should manually: service bhyvevm start
(2) If the VM was powered down, via shutdown -p, or halt -p,
then in my script I do not want to restart the VM in the script.
User should manually: service bhyvevm start
(3) If the VM was rebooted via reboot or shutdown -r,
then I *do* want the script to restart the VM.
I think if I change my start_vm.sh script to do something like:
(
while [ -e /dev/vmm/${VM} ]; do
/usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc
-s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A} ${VM}
done
)
then this might cover cases (1) and (3), but what will cover
case (2)?
The exit code of the bhyve process will be 0 if it exited because the
guest rebooted and will be non-zero if the guest did an acpi poweroff.
You can use that to distinguish between cases (2) and (3).
Having said that there are error conditions for which bhyve exits with
a non-zero exit code. So, we'll need to explicitly define an exit code
to distinguish between an acpi poweroff and these error conditions.
OTOH, in all the cases when bhyve exits with a non-zero exit code, you
will want to exit the loop which would treat it the same as shutdown -p. I
think you can just do this:
while [ -e /dev/vmm/${VM} ]; do
if ! bhyve ...; then
break
fi
done
--
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
On Tue, Feb 11, 2014 at 11:11 AM, John Baldwin j...@freebsd.org wrote:
On Sunday, February 09, 2014 7:03:41 pm Neel Natu wrote:
Hi Craig,
On Sun, Feb 9, 2014 at 1:11 PM, Craig Rodrigues rodr...@freebsd.org
wrote:
Hi,
I posted some rc.d scripts that I am using to boot a BHyve VM
and send the output to a serial console using the /dev/nmdm
driver:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-
January/002040.html
It works quite well. There is some things I would like to improve,
and would like some advice on the best way to do it.
(1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
then I do not want to automatically restart the VM in the script.
User should manually: service bhyvevm start
(2) If the VM was powered down, via shutdown -p, or halt -p,
then in my script I do not want to restart the VM in the script.
User should manually: service bhyvevm start
(3) If the VM was rebooted via reboot or shutdown -r,
then I *do* want the script to restart the VM.
I think if I change my start_vm.sh script to do something like:
(
while [ -e /dev/vmm/${VM} ]; do
/usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s
1:0,lpc
-s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A}
${VM}
done
)
then this might cover cases (1) and (3), but what will cover
case (2)?
The exit code of the bhyve process will be 0 if it exited because the
guest rebooted and will be non-zero if the guest did an acpi poweroff.
You can use that to distinguish between cases (2) and (3).
Having said that there are error conditions for which bhyve exits with
a non-zero exit code. So, we'll need to explicitly define an exit code
to distinguish between an acpi poweroff and these error conditions.
OTOH, in all the cases when bhyve exits with a non-zero exit code, you
will want to exit the loop which would treat it the same as shutdown -p. I
think you can just do this:
while [ -e /dev/vmm/${VM} ]; do
if ! bhyve ...; then
break
fi
done
One question, if bhyve exits, do I have to call bhyveload again before
calling bhyve?
The /usr/share/examples/bhyve/vmrun.sh has a loop which does:
while [ 1 ] ; do
...
bhyvectl --destroy
bhyveload
bhyve
...
done
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
Hi Craig,
On Feb 11, 2014, at 2:29 PM, Craig Rodrigues rodr...@freebsd.org wrote:
On Tue, Feb 11, 2014 at 11:11 AM, John Baldwin j...@freebsd.org wrote:
On Sunday, February 09, 2014 7:03:41 pm Neel Natu wrote:
Hi Craig,
On Sun, Feb 9, 2014 at 1:11 PM, Craig Rodrigues rodr...@freebsd.org
wrote:
Hi,
I posted some rc.d scripts that I am using to boot a BHyve VM
and send the output to a serial console using the /dev/nmdm
driver:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-
January/002040.html
It works quite well. There is some things I would like to improve,
and would like some advice on the best way to do it.
(1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
then I do not want to automatically restart the VM in the script.
User should manually: service bhyvevm start
(2) If the VM was powered down, via shutdown -p, or halt -p,
then in my script I do not want to restart the VM in the script.
User should manually: service bhyvevm start
(3) If the VM was rebooted via reboot or shutdown -r,
then I *do* want the script to restart the VM.
I think if I change my start_vm.sh script to do something like:
(
while [ -e /dev/vmm/${VM} ]; do
/usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s
1:0,lpc
-s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A}
${VM}
done
)
then this might cover cases (1) and (3), but what will cover
case (2)?
The exit code of the bhyve process will be 0 if it exited because the
guest rebooted and will be non-zero if the guest did an acpi poweroff.
You can use that to distinguish between cases (2) and (3).
Having said that there are error conditions for which bhyve exits with
a non-zero exit code. So, we'll need to explicitly define an exit code
to distinguish between an acpi poweroff and these error conditions.
OTOH, in all the cases when bhyve exits with a non-zero exit code, you
will want to exit the loop which would treat it the same as shutdown -p. I
think you can just do this:
while [ -e /dev/vmm/${VM} ]; do
if ! bhyve ...; then
break
fi
done
One question, if bhyve exits, do I have to call bhyveload again before
calling bhyve?
Yes, that is correct. You will also need to destroy the VM after bhyve exits.
Best
Neel
The /usr/share/examples/bhyve/vmrun.sh has a loop which does:
while [ 1 ] ; do
...
bhyvectl --destroy
bhyveload
bhyve
...
done
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
On Sun, Feb 9, 2014 at 4:11 PM, Craig Rodrigues rodr...@freebsd.org wrote:
Hi,
I posted some rc.d scripts that I am using to boot a BHyve VM
and send the output to a serial console using the /dev/nmdm
driver:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-January/002040.html
It works quite well. There is some things I would like to improve,
and would like some advice on the best way to do it.
(1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
then I do not want to automatically restart the VM in the script.
User should manually: service bhyvevm start
(2) If the VM was powered down, via shutdown -p, or halt -p,
then in my script I do not want to restart the VM in the script.
User should manually: service bhyvevm start
(3) If the VM was rebooted via reboot or shutdown -r,
then I *do* want the script to restart the VM.
I think if I change my start_vm.sh script to do something like:
(
while [ -e /dev/vmm/${VM} ]; do
/usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc
-s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A} ${VM}
done
)
then this might cover cases (1) and (3), but what will cover
case (2)?
Thanks for any advice.
The options you gave (which are really the only ones available) do not
distinguish between the reason for the termination of the instance of bhyve
pointing to /dev/vmm/XXX (it just does a normal termination). In my
playing with this for almost a month the only solution seems to be put a
trigger on instance shutdown via whatever method to leave a cookie file
behind on the host (note I have not nor plan to do this in any of my own
work). An other question is it wise to auto-reboot regardless of reason of
the hypervisor termination?
--
Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
On Sun, Feb 9, 2014 at 5:18 PM, Craig Rodrigues rodr...@freebsd.org wrote:
On Sun, Feb 9, 2014 at 1:32 PM, Aryeh Friedman
aryeh.fried...@gmail.comwrote:
On Sun, Feb 9, 2014 at 4:11 PM, Craig Rodrigues rodr...@freebsd.orgwrote:
Hi,
I posted some rc.d scripts that I am using to boot a BHyve VM
and send the output to a serial console using the /dev/nmdm
driver:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-January/002040.html
It works quite well. There is some things I would like to improve,
and would like some advice on the best way to do it.
(1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
then I do not want to automatically restart the VM in the script.
User should manually: service bhyvevm start
(2) If the VM was powered down, via shutdown -p, or halt -p,
then in my script I do not want to restart the VM in the script.
User should manually: service bhyvevm start
(3) If the VM was rebooted via reboot or shutdown -r,
then I *do* want the script to restart the VM.
I think if I change my start_vm.sh script to do something like:
(
while [ -e /dev/vmm/${VM} ]; do
/usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s
1:0,lpc
-s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A}
${VM}
done
)
then this might cover cases (1) and (3), but what will cover
case (2)?
Thanks for any advice.
The options you gave (which are really the only ones available) do not
distinguish between the reason for the termination of the instance of bhyve
pointing to /dev/vmm/XXX (it just does a normal termination).
Did you play with any of these flags to bhyvectl?
[--get-vmcs-exit-ctls]
[--get-vmcs-exit-reason]
[--get-vmcs-exit-qualification]
[--get-vmcs-exit-interruption-info]
If I do:
bhyvectl --get-vmcs-exit-reason --vm vm1
I get:
vmcs_exit_reason[0] 0x001e
If I look at:
http://svnweb.freebsd.org/base/head/sys/amd64/vmm/intel/vmcs.h?view=markup#l310
we have:
#define EXIT_REASON_INOUT 30
You will likely need to detect the following ones also (all for different
reasons that may or may not have anything to do with an issue with the VM
instead of some host [OS or hardware] issue):
#define EXIT_REASON_EXT_INTR 1
#define EXIT_REASON_INIT 3
#define EXIT_REASON_HLT 12
#define EXIT_REASON_CPUID 10
#define EXIT_REASON_VMLAUNCH 20
Linux has this stuff also:
http://lxr.free-electrons.com/source/arch/x86/include/asm/vmx.h?v=3.1#L250
so I guess these values are defined in some Intel manual.
In my playing with this for almost a month the only solution seems to be
put a trigger on instance shutdown via whatever method to leave a cookie
file behind on the host (note I have not nor plan to do this in any of my
own work).
Yes, I can see why this might need to be done. I might need to do that
myself. However, if bhyvectl can be used to provide adequate status
information as to why the VM exited, I might be able to avoid this.
It would be nicer though if /usr/sbin/bhyve returned
the EXIT_REASON in its status code.
An other question is it wise to auto-reboot regardless of reason of the
hypervisor termination?
I think the answer is it depends. If someone
does reboot or shutdown -r, I think it is reasonable to restart the
VM. If someone does halt -p or shutdown -p, I think it is reasonable
to not restart the VM. For any other error condition, I think it is
reasonable to not restart the VM.
I think that there is no single answer for all users and all applications.
In my case, I am running BHyve VM's as headless which
I access via the /dev/nmdm driver. If the VM reboots normally, I want it
to restart.
It seems almost any boot failure would be host side for example if there is
no SSH access (which if the instance is never logged into then it is likely
a host issue) then it is highly unlikely the VM is in bootable form (this
is after a manual reboot of it)... an other reason (just learned the
hardway) is the user uses a mismatch between backing file formats (my case
using raw images with ahci-hd)
--
Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
Hi Craig,
On Sun, Feb 9, 2014 at 1:11 PM, Craig Rodrigues rodr...@freebsd.org wrote:
Hi,
I posted some rc.d scripts that I am using to boot a BHyve VM
and send the output to a serial console using the /dev/nmdm
driver:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-January/002040.html
It works quite well. There is some things I would like to improve,
and would like some advice on the best way to do it.
(1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
then I do not want to automatically restart the VM in the script.
User should manually: service bhyvevm start
(2) If the VM was powered down, via shutdown -p, or halt -p,
then in my script I do not want to restart the VM in the script.
User should manually: service bhyvevm start
(3) If the VM was rebooted via reboot or shutdown -r,
then I *do* want the script to restart the VM.
I think if I change my start_vm.sh script to do something like:
(
while [ -e /dev/vmm/${VM} ]; do
/usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc
-s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A} ${VM}
done
)
then this might cover cases (1) and (3), but what will cover
case (2)?
The exit code of the bhyve process will be 0 if it exited because the
guest rebooted and will be non-zero if the guest did an acpi poweroff.
You can use that to distinguish between cases (2) and (3).
Having said that there are error conditions for which bhyve exits with
a non-zero exit code. So, we'll need to explicitly define an exit code
to distinguish between an acpi poweroff and these error conditions.
best
Neel
Thanks for any advice.
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
Hi Craig,
On Sun, Feb 9, 2014 at 2:18 PM, Craig Rodrigues rodr...@freebsd.org wrote:
On Sun, Feb 9, 2014 at 1:32 PM, Aryeh Friedman
aryeh.fried...@gmail.comwrote:
On Sun, Feb 9, 2014 at 4:11 PM, Craig Rodrigues rodr...@freebsd.orgwrote:
Hi,
I posted some rc.d scripts that I am using to boot a BHyve VM
and send the output to a serial console using the /dev/nmdm
driver:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-January/002040.html
It works quite well. There is some things I would like to improve,
and would like some advice on the best way to do it.
(1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
then I do not want to automatically restart the VM in the script.
User should manually: service bhyvevm start
(2) If the VM was powered down, via shutdown -p, or halt -p,
then in my script I do not want to restart the VM in the script.
User should manually: service bhyvevm start
(3) If the VM was rebooted via reboot or shutdown -r,
then I *do* want the script to restart the VM.
I think if I change my start_vm.sh script to do something like:
(
while [ -e /dev/vmm/${VM} ]; do
/usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s 1:0,lpc
-s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A} ${VM}
done
)
then this might cover cases (1) and (3), but what will cover
case (2)?
Thanks for any advice.
The options you gave (which are really the only ones available) do not
distinguish between the reason for the termination of the instance of bhyve
pointing to /dev/vmm/XXX (it just does a normal termination).
Did you play with any of these flags to bhyvectl?
[--get-vmcs-exit-ctls]
[--get-vmcs-exit-reason]
[--get-vmcs-exit-qualification]
[--get-vmcs-exit-interruption-info]
If I do:
bhyvectl --get-vmcs-exit-reason --vm vm1
I get:
vmcs_exit_reason[0] 0x001e
If I look at:
http://svnweb.freebsd.org/base/head/sys/amd64/vmm/intel/vmcs.h?view=markup#l310
we have:
#define EXIT_REASON_INOUT 30
Linux has this stuff also:
http://lxr.free-electrons.com/source/arch/x86/include/asm/vmx.h?v=3.1#L250
so I guess these values are defined in some Intel manual.
Yes, but this an entirely different thing. The VMCS exit reason is
used to inform the hypervisor what condition caused the guest to stop
executing - e.g. a nested page fault.
Please don't use it to infer whether the guest wanted to reboot or poweroff :-)
In my playing with this for almost a month the only solution seems to be
put a trigger on instance shutdown via whatever method to leave a cookie
file behind on the host (note I have not nor plan to do this in any of my
own work).
Yes, I can see why this might need to be done. I might need to do that
myself. However, if bhyvectl can be used to provide adequate status
information as to why the VM exited, I might be able to avoid this.
It would be nicer though if /usr/sbin/bhyve returned
the EXIT_REASON in its status code.
Agreed - see my reply to your original post.
best
Neel
An other question is it wise to auto-reboot regardless of reason of the
hypervisor termination?
I think the answer is it depends. If someone
does reboot or shutdown -r, I think it is reasonable to restart the
VM. If someone does halt -p or shutdown -p, I think it is reasonable
to not restart the VM. For any other error condition, I think it is
reasonable to not restart the VM.
I think that there is no single answer for all users and all applications.
In my case, I am running BHyve VM's as headless which
I access via the /dev/nmdm driver. If the VM reboots normally, I want it
to restart.
--
Craig
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
Re: Detect of BHyve VM was powered off or rebooted?
Hi Aryeh,
On Sun, Feb 9, 2014 at 2:34 PM, Aryeh Friedman aryeh.fried...@gmail.com wrote:
On Sun, Feb 9, 2014 at 5:18 PM, Craig Rodrigues rodr...@freebsd.org wrote:
On Sun, Feb 9, 2014 at 1:32 PM, Aryeh Friedman
aryeh.fried...@gmail.comwrote:
On Sun, Feb 9, 2014 at 4:11 PM, Craig Rodrigues rodr...@freebsd.orgwrote:
Hi,
I posted some rc.d scripts that I am using to boot a BHyve VM
and send the output to a serial console using the /dev/nmdm
driver:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-January/002040.html
It works quite well. There is some things I would like to improve,
and would like some advice on the best way to do it.
(1) If the VM was destroyed with bhyvectl --destroy --vm ${VM_NAME},
then I do not want to automatically restart the VM in the script.
User should manually: service bhyvevm start
(2) If the VM was powered down, via shutdown -p, or halt -p,
then in my script I do not want to restart the VM in the script.
User should manually: service bhyvevm start
(3) If the VM was rebooted via reboot or shutdown -r,
then I *do* want the script to restart the VM.
I think if I change my start_vm.sh script to do something like:
(
while [ -e /dev/vmm/${VM} ]; do
/usr/sbin/bhyve -c 16 -m 8G -A -H -P -g 0 -s 0:0,hostbridge -s
1:0,lpc
-s 2:0,virtio-net,${TAP} -s 3:0,virtio-blk,${IMG} -l com1,${CONS_A}
${VM}
done
)
then this might cover cases (1) and (3), but what will cover
case (2)?
Thanks for any advice.
The options you gave (which are really the only ones available) do not
distinguish between the reason for the termination of the instance of bhyve
pointing to /dev/vmm/XXX (it just does a normal termination).
Did you play with any of these flags to bhyvectl?
[--get-vmcs-exit-ctls]
[--get-vmcs-exit-reason]
[--get-vmcs-exit-qualification]
[--get-vmcs-exit-interruption-info]
If I do:
bhyvectl --get-vmcs-exit-reason --vm vm1
I get:
vmcs_exit_reason[0] 0x001e
If I look at:
http://svnweb.freebsd.org/base/head/sys/amd64/vmm/intel/vmcs.h?view=markup#l310
we have:
#define EXIT_REASON_INOUT 30
You will likely need to detect the following ones also (all for different
reasons that may or may not have anything to do with an issue with the VM
instead of some host [OS or hardware] issue):
#define EXIT_REASON_EXT_INTR 1
#define EXIT_REASON_INIT 3
#define EXIT_REASON_HLT 12
#define EXIT_REASON_CPUID 10
#define EXIT_REASON_VMLAUNCH 20
These are VMCS exit reasons that have nothing to do whatsoever with
the guest wanting to reboot or power itself off.
best
Neel
Linux has this stuff also:
http://lxr.free-electrons.com/source/arch/x86/include/asm/vmx.h?v=3.1#L250
so I guess these values are defined in some Intel manual.
In my playing with this for almost a month the only solution seems to be
put a trigger on instance shutdown via whatever method to leave a cookie
file behind on the host (note I have not nor plan to do this in any of my
own work).
Yes, I can see why this might need to be done. I might need to do that
myself. However, if bhyvectl can be used to provide adequate status
information as to why the VM exited, I might be able to avoid this.
It would be nicer though if /usr/sbin/bhyve returned
the EXIT_REASON in its status code.
An other question is it wise to auto-reboot regardless of reason of the
hypervisor termination?
I think the answer is it depends. If someone
does reboot or shutdown -r, I think it is reasonable to restart the
VM. If someone does halt -p or shutdown -p, I think it is reasonable
to not restart the VM. For any other error condition, I think it is
reasonable to not restart the VM.
I think that there is no single answer for all users and all applications.
In my case, I am running BHyve VM's as headless which
I access via the /dev/nmdm driver. If the VM reboots normally, I want it
to restart.
It seems almost any boot failure would be host side for example if there is
no SSH access (which if the instance is never logged into then it is likely
a host issue) then it is highly unlikely the VM is in bootable form (this
is after a manual reboot of it)... an other reason (just learned the
hardway) is the user uses a mismatch between backing file formats (my case
using raw images with ahci-hd)
--
Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
freebsd-virtualization-unsubscr...@freebsd.org
