Hello, We are experimenting network issues each time we start a new VM.
The other VMs have their network blocked until I see “a topology change
is detected” in Open vSwitch logs.
Do you have any advice for such configuration?
Here is our setup:
We are using Open vSwitch for networking in our OpenNebula setup[1]:
- 4 physical servers (corosync+pacemaker+GFS2 => STONITH)
- Ubuntu Trusty Tahr 14.04.2 (Open vSwitch 2.0.2-0ubuntu0.14.04.1)
- internal bridges “in-br” connected between nodes with vxlan tunnels,
with a loop and spanning tree
- User VM on their own VLAN (1 VLAN per user)
- OpenNebula manages flows with ovs-ofctl, here are the logs when
starting a VM:
#+begin_src text
Sat Mar 7 15:21:37 2015 [Z0][DiM][I]: New VM state is ACTIVE.
Sat Mar 7 15:21:37 2015 [Z0][LCM][I]: New VM state is PROLOG.
Sat Mar 7 15:21:37 2015 [Z0][LCM][I]: New VM state is BOOT
Sat Mar 7 15:21:37 2015 [Z0][VMM][I]: Generating deployment file:
/var/lib/one/vms/14/deployment.0
Sat Mar 7 15:21:37 2015 [Z0][VMM][I]: ExitCode: 0
Sat Mar 7 15:21:37 2015 [Z0][VMM][I]: Successfully execute network driver
operation: pre.
Sat Mar 7 15:21:38 2015 [Z0][VMM][I]: ExitCode: 0
Sat Mar 7 15:21:38 2015 [Z0][VMM][I]: Successfully execute virtualization
driver operation: deploy.
Sat Mar 7 15:21:38 2015 [Z0][VMM][I]: post: Executed "sudo ovs-vsctl set
Port vnet18 tag=20".
Sat Mar 7 15:21:38 2015 [Z0][VMM][I]: post: Executed "sudo ovs-ofctl
add-flow in-br
in_port=17,dl_src=02:03:00:00:00:00,priority=40000,actions=normal".
Sat Mar 7 15:21:38 2015 [Z0][VMM][I]: post: Executed "sudo ovs-ofctl
add-flow in-br in_port=17,priority=39000,actions=drop".
Sat Mar 7 15:21:38 2015 [Z0][VMM][I]: ExitCode: 0
Sat Mar 7 15:21:38 2015 [Z0][VMM][I]: Successfully execute network driver
operation: post.
Sat Mar 7 15:21:38 2015 [Z0][LCM][I]: New VM state is RUNNING
#+end_src
A picture is better than thousand words:
#+begin_src picture
------+---------------------+---------------------+---------------------+-------
physical network (192.168.1.0/24)
|.1 |.2 |.3 |.4
+------+------+ +------+------+ +------+------+
+------+------+
| | | | | | | | | | |
|
| +----+----+ | | +----+----+ | | +----+----+ | |
+----+----+ |
| | | | | | | | | | | | | |
| |
| | eth0 | | | | eth0 | | | | eth0 | | | | eth0
| |
| | | | | | | | | | | | | |
| |
| +---------+ | | +---------+ | | +---------+ | |
+---------+ |
| | | | | | |
|
| | | | | | |
|
| | | | | | |
|
| | | | | | |
|
| +---------+ | | +---------+ | | +---------+ | |
+---------+ |
| | | | vxlan | | | | vxlan | | | | vxlan | |
| |
| | in-br +-+-------+-+ in-br +-+-------+-+ in-br +-+-------+-+ in-br
+-+---+
| | | | | | | | | | | | | |
| | |
| +----+----+ | | +---------+ | | +---------+ | |
+---------+ | |
| | | | | | | |
| |
+------+------+ +-------------+ +-------------+
+-------------+ |
|
|
| vxlan
|
+----------------------------------------------------------------------------+
#+end_src
I setup the isolated bridges and the VXLAN tunnels between hosts simply with:
#+begin_src sh
#!/bin/bash
[[ "${DEBUG}" =~ ^[tT]([rR][uU][eE])?$ ]] && DRY=echo
[[ "${DRY_RUN}" =~ ^[tT]([rR][uU][eE])?$ ]] && DRY=:
NET=192.168.1
for hostip in {1..4}
do
echo "Creating isolated bridge"
${DRY} ssh root@${NET}.${hostip} \
ovs-vsctl add-br in-br
# Enable STP first
echo "Enable spanning tree on node ${hostip}"
${DRY} ssh root@${NET}.${hostip} ovs-vsctl set Bridge in-br
stp_enable=true
PREV_NODE_ID=$(( hostip - 1 ))
# Loop on node4
[ ${PREV_NODE_ID} -eq 0 ] && PREV_NODE_ID=4
NEXT_NODE_ID=$(( (hostip+1) % 5 ))
# Loop on node1
[ ${NEXT_NODE_ID} -eq 0 ] && NEXT_NODE_ID=1
INTERFACE_TO_PREV="vx-to-node${PREV_NODE_ID}"
INTERFACE_TO_NEXT="vx-to-node${NEXT_NODE_ID}"
# Link to previous node switch
echo "Connect node ${hostip} to node ${PREV_NODE_ID}"
${DRY} ssh root@${NET}.${hostip} \
ovs-vsctl add-port in-br ${INTERFACE_TO_PREV} -- \
set interface ${INTERFACE_TO_PREV} type=vxlan \
options:local_ip=${hostip} \
remote_ip=${NET}.${PREV_NODE_ID}
# Link to next node switch
echo "Connect node ${hostip} to node ${NEXT_NODE_ID}"
${DRY} ssh root@${NET}.${hostip} \
ovs-vsctl add-port in-br ${INTERFACE_TO_NEXT} -- \
set interface ${INTERFACE_TO_NEXT} type=vxlan \
options:local_ip=${hostip} \
options:remote_ip=${NET}.${NEXT_NODE_ID}
echo
done
#+end_src
Regards.
Footnotes:
[1] http://docs.opennebula.org/4.12/administration/networking/nm.html
--
Daniel Dehennin
Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF
signature.asc
Description: PGP signature
_______________________________________________ discuss mailing list [email protected] http://openvswitch.org/mailman/listinfo/discuss
