[CentOS-virt] XEN multiple bridge problem - VM won' start!
Finally, I managed to solve the problem. The problem was rather strange -
after configuring bridges I added new ifaces to the VM. For some reason,
during boot process of the VM's, wrong interface was started (VM tried to
bring up eth2 instead eth1?).
After adding the new ifaces through virt-manager, the VM started regulary
and created a new network script for eth2
(/etc/sysconfig/network-scripts/ifcfg-eth2). I think that the problem was
this - when the VM is restarted it tried to bring up eth2 while vif-script
tried to propagate second bridge via eth1 iface. I resolved this simply by
replacing ifcfg-eth2 script with ifcfg-eth1. Now, VM starts/restarts
regulary ;)
So, the process for creating and configuring new bridges that I follow is
(tried on multiple servers):
- create new script that sets up multiple bridges in /etc/xen/scripts/
example: network-xen-multi-bridge
#!/bin/sh
# network-xen-multi-bridge
# Exit if anything goes wrong.
set -e
# First arg is the operation.
OP=$1
shift
script=/etc/xen/scripts/network-bridge.xen
case ${OP} in
start)
$script start vifnum=3 bridge=xenbr3 netdev=eth3
$script start vifnum=0 bridge=xenbr0 netdev=eth0
;;
stop)
$script stop vifnum=3 bridge=xenbr3 netdev=eth3
$script stop vifnum=0 bridge=xenbr0 netdev=eth0
;;
status)
$script status vifnum=3 bridge=xenbr3 netdev=eth3
$script status vifnum=0 bridge=xenbr0 netdev=eth0
;;
*)
echo 'Unknown command: ' ${OP}
echo 'Valid commands are: start, stop, status'
exit 1
esac
- make it executable (chmod 755 network-xen-multi-bridge)
- configure /etc/xen/xend-config.sxp (comment out the current script and
add the new one);
# It is possible to use the network-bridge script in more complicated
# scenarios, such as having two outgoing interfaces, with two bridges, and
# two fake interfaces per guest domain. To do things like this, write
# yourself a wrapper script, and call network-bridge from it, as appropriate.
#
#(network-script network-bridge)
(network-script network-xen-multi-bridge)
- restart physical server so the bridges are properly created
- add virt. ifaces to the VM's (via virt-manager or via config. files in
/etc/xen/ ..)
- boot VM
- configure newly added iface in /etc/sysconfig/network-scripts/
- restart VM
Additionaly I check everything by restarting phys. server to make sure that
everything is ok ;)
___
CentOS-virt mailing list
CentOS-virt@centos.org
http://lists.centos.org/mailman/listinfo/centos-virt
[CentOS-virt] XEN multiple bridge problem - VM won' start!
Ok, after trying multiple different configurations I decided to play by the
book - I removed ifcfg script for creating the additional bridge and set up
an Xen script. So my new configuration is:
-> eth3 (my physical interface)
# NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter
DEVICE=eth3
BOOTPROTO=static
HWADDR= ...
ONBOOT=yes
IPADDR= ...
NETMASK= ...
NETWORK= ...
and the network-xen-multi-bridge script:
#!/bin/sh
# network-xen-multi-bridge
# Exit if anything goes wrong.
set -e
# First arg is the operation.
OP=$1
shift
script=/etc/xen/scripts/network-bridge.xen
case ${OP} in
start)
$script start vifnum=3 bridge=xenbr3 netdev=eth3
$script start vifnum=0 bridge=xenbr0 netdev=eth0
;;
stop)
$script stop vifnum=3 bridge=xenbr3 netdev=eth3
$script stop vifnum=0 bridge=xenbr0 netdev=eth0
;;
status)
$script status vifnum=3 bridge=xenbr3 netdev=eth3
$script status vifnum=0 bridge=xenbr0 netdev=eth0
;;
*)
echo 'Unknown command: ' ${OP}
echo 'Valid commands are: start, stop, status'
exit 1
esac
After I restart the physical server bridges are set up correctly:
# brctl show
bridge name bridge id STP enabled interfaces
virbr0 8000. yes
xenbr0 8000.feff no vif0.0
peth0
xenbr3 8000.feff no vif0.3
peth3
Unfortunately, this did not solve the problem - I manage to start one VM
(connected to both of the bridges), but when I try to start the other one,
it just hangs when trying to bring up the second interface (connected to
xenbr3) :/
I'm currently completely out of ideas, but I'm not giving up ;)
___
CentOS-virt mailing list
CentOS-virt@centos.org
http://lists.centos.org/mailman/listinfo/centos-virt
[CentOS-virt] XEN multiple bridge problem - VM won' start!
I tried to remove vifname and vif-script but the problem remains. I think that the bridges are ok, but for some reason VM's hang when they try to connect their vifaces to them. I'll try some different configurations of the physical ifaces (an vifaces also), maybe I'll be able to find out something. Also, I'll focus on the vTPM, since the following line shows up in the Xen log when the VM tries to bring up the iface connected to the second bridge: [2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for devices vtpm Furthermore, it appears that sometimes the VM's manage to start regulary, I just can't figure out when & why :/ Tnx for your comments .. Br, M. ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] XEN multiple bridge problem - VM won' start!
Matija Draganoviæ wrote on Fri, 18 Nov 2011 18:17:17 +0100: > The only diference between your config and mine is that I have vifname-s > listed Then remove it. I've never seen it like that. If I remember it correctly the vifnames are created dynamically. You can't rely on them I suppose. And I don't remember using "script" either. If you can't get it to work then use the second method Peter describes. That one should also work for Xen 3.0 and is guaranteed to create stable bridging. Kai -- Get your web at Conactive Internet Services: http://www.conactive.com ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
[CentOS-virt] XEN multiple bridge problem - VM won' start!
Dear Peter, Thanks for your reply. I was so occupied with troubleshooting that I forgot to mention the OS & Xen version :) My CentOS version is CentOS release 5.7 (Final) Linux 2.6.18-274.7.1.el5xen #1 SMP Thu Oct 20 17:06:34 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux and Xen v3.0. I tried creating bridges the same way you did and succedded. I also tried doing the same via ifcfg scripts, and it also worked. I have no problem with creating bridges and connecting them with physical ifaces, but I can't start my VM's (most of the time one VM start but the other one hangs while trying to initalize eth2 that is connected to my second (non default - xenbr1) bridge)). The only diference between your config and mine is that I have vifname-s listed (example below): vif = [ "mac=00:16:36:72:35:e5,bridge=xenbr0,script=vif-bridge,vifname=vif4.0", "mac=00:16:3e:57:de:0d,bridge=xenbr1,script=vif-bridge, vifname=vif4.1" ] I'm not saying that this could be the problem, but since I'm slowly running out of ideas - it's a longshot :) So, to sum it all up - I can create bridges properly, but my VM's hang when they try to bring up vifaces connected to the xenbr1. Also, I have no problem with having only one bridge active - problem appears when I have multiple bridges. Br, Matija ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] XEN multiple bridge problem - VM won' start!
Hi, 2011/11/18 Matija Draganović : > I've been using CentOS & Xen on a server that has 2 VM's configured. The > default configuration includes one physical iface that is propagated (by a > default bridge) to the VM's. You do not mention which version of CentOS and Xen you are using? > > Since I wanted to configure additional physical iface, define a new bridge > and propagate it to the viface-s of the VM's, i configured the bridge/phys. > iface and brought it up (here are configurations that I set up): > > -> eth3 > DEVICE=eth3 > BOOTPROTO=static > HWADDR=D4:85:64:4B:76:AB > ONBOOT=yes > #HOTPLUG=no > #DHCP_HOSTNAME=kdr-3k-4r-3o-07 > BRIDGE=br0 > TYPE=Ethernet > > -> br0 > DEVICE=br0 > TYPE=Bridge > BOOTPROTO=none > ONBOOT=yes If this is a CentOS5 machine with the CentOS provided Xen 3.0 packages, then here is info how i got my bridges to work with that setup: * For dom0 I configured eth0 and eth1 as usual in /etc/sysconfig/network-scripts * I did not configure br0 or br1 in network-scripts, but instead I created a file called /etc/xen/scripts/my-network-script having this contents: #!/bin/sh dir=$(dirname "$0") "$dir/network-bridge" "$@" vifnum=0 netdev=eth0 bridge=xenbr0 "$dir/network-bridge" "$@" vifnum=1 netdev=eth1 bridge=xenbr1 * Then I modified in dom0 the file /etc/xen/xend-config.sxp in the following way: #(network-script network-bridge) (network-script my-network-script) * Now rebooting dom0 made the bridges available * After that I could configure them in my domU config: vif = [ "mac=00:16:3E:69:29:25,bridge=xenbr0,script=vif-bridge","mac=00:16:3E:E6:B0:6D,bridge=xenbr1,script=vif-bridge" ] * After starting the domU I could configure the interfaces in network-scripts using the hardware addresses specified in the domU config It seems that configuring bridging is done a bit differently in different Xen versions, so this might not work if you are using some other kind of config. With CentOS6 and 3rd party Xen 4.1 packages this procedure did not work at all and instead I needed to do the following: * in dom0 create the br0 and br1 devices in network-scripts * in dom0 /etc/xen/xend-config.sxp put instead: #(network-script network-bridge) (network-script /bin/true) * and in dom0 use different configuration for the domU config files like this: vif = [ "mac=00:16:3E:69:29:25,bridge=br0","mac=00:16:3E:E6:B0:6D,bridge=br1" ] Hope this helps, unfortuantely I am not familiar with virsh at all. Best, Peter ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
[CentOS-virt] XEN multiple bridge problem - VM won' start!
Hi,
I've been using CentOS & Xen on a server that has 2 VM's configured. The
default configuration includes one physical iface that is propagated (by a
default bridge) to the VM's.
Since I wanted to configure additional physical iface, define a new bridge
and propagate it to the viface-s of the VM's, i configured the bridge/phys.
iface and brought it up (here are configurations that I set up):
-> eth3
DEVICE=eth3
BOOTPROTO=static
HWADDR=D4:85:64:4B:76:AB
ONBOOT=yes
#HOTPLUG=no
#DHCP_HOSTNAME=kdr-3k-4r-3o-07
BRIDGE=br0
TYPE=Ethernet
-> br0
DEVICE=br0
TYPE=Bridge
BOOTPROTO=none
ONBOOT=yes
Then I configured new virt. ifaces on VM's (using virsh edit, config is
listed below):
You can see the default bridge (xenbr0 -> vif5.0) and my new bridge br0
connected to vif5.1.
So, here I got stuck up - when I try tu start a VM (xm create VIRT_SRV),
the VM starts booting but hangs up (I can ping it form another srv., but
when I try SSH it says "connection refused"). When I remove configuration
for vif5.1 the VM starts up normally. I actually managed to start VM with
vif5.1 up once, but next time I restarted it, the problem was there :/
Furthermore, I noticed this in the Xen log:
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for
devices vif.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:166) Waiting for 0.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:544)
hotplugStatusCallback /local/domain/0/backend/vif/1/0/hotplug-status.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:544)
hotplugStatusCallback /local/domain/0/backend/vif/1/0/hotplug-status.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:558)
hotplugStatusCallback 1.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:166) Waiting for 1.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:544)
hotplugStatusCallback /local/domain/0/backend/vif/1/1/hotplug-status.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:558)
hotplugStatusCallback 1.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for
devices usb.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for
devices vbd.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:166) Waiting for 768.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:544)
hotplugStatusCallback /local/domain/0/backend/vbd/1/768/hotplug-status.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:558)
hotplugStatusCallback 1.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:166) Waiting for 5632.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:544)
hotplugStatusCallback /local/domain/0/backend/vbd/1/5632/hotplug-status.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:558)
hotplugStatusCallback 1.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for
devices irq.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for
devices vkbd.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for
devices vfb.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for
devices pci.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for
devices ioports.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for
devices tap.
[2011-11-18 09:27:16 xend 8213] DEBUG (DevController:160) Waiting for
devices vtpm.
So the boot process goes on regulary until the last line ("Waiting for
devices vtpm"). It seems like TPM makes it impossible for VM's to start
regulary. When I open the VM's console (with virt-manager) I see the whole
boot process but it hangs when it tries to start iface eth2 (attached to
the second bridge) ...
Does anybody have a clue why is this happening?
Thanks in advance ..
___
CentOS-virt mailing list
CentOS-virt@centos.org
http://lists.centos.org/mailman/listinfo/centos-virt
