Public bug reported:
Issue:
======
Installation steps for Openstack Queens Compute Nodes:
1- Install and configure openvswitch, neutron-openvswitch-agent,
openstack-nova-compute,libvirt
2- create virtual bridges
/bin/ovs-vsctl add-br br-int
/bin/ovs-vsctl add-br br-bond0
/bin/ovs-vsctl --may-exist add-bond br-bond0 bond0 eno49 eno50
bond_mode=active-backup
3- add mgmt0 port to br-bond0 bridge which will be used by hypervisor to
connect to openstack controllers by reconfigure
/etc/sysconfig/network-scripts/ifcfg-*
5- delete libvirt default network.
Note: I share the same physical network interfaces between Data Network(VM
network) and Management Network(nova and neutron network to controllers). And
all the physical interfaces are bonding(active-backup).
6- reboot for the first time
After the first reboot the network connection is functioning as expected.
Then try the second reboot, without changing any config, the network never come
back again.
This can be fix by rerun the step 2 and do another reboot but if i reboot the
machine another time, the network will break again. Also, if i stop and disable
neutron-openvswitch-agent, and rerun step 2, the problem will disappear however
many time i reboot the network connection will still function.
I'm not sure if i miss something here but i have this same setup working from
Juno through Ocata, i never test on Pike though.
Setup
=====
OS: CentOS Linux release 7.5.1804 (Core)
Neutron Openvswitch agent tested version:
openstack-neutron-openvswitch-12.0.2-1.el7.noarch,
openstack-neutron-openvswitch-12.0.4-1.el7.noarch
Openvswitch tested version: openvswitch-2.9.0-3, openvswitch-2.9.0-4
Network Setup:
[eno5,eno6] <= Br-bond0[bond0,mgmt0,int-br-bond0] <= Br-int[phy-br-bond0] <=
veths (VMs IF)
# ovs-vsctl show
0f950035-7f7a-4e5f-a337-04ab76945679
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "int-br-bond0"
Interface "int-br-bond0"
type: patch
options: {peer="phy-br-bond0"}
Port br-int
Interface br-int
type: internal
Bridge "br-bond0"
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "mgmt0"
Interface "mgmt0"
type: internal
Port "br-bond0"
Interface "br-bond0"
type: internal
Port "bond0"
Interface "eno6"
Interface "eno5"
Port "phy-br-bond0"
Interface "phy-br-bond0"
type: patch
options: {peer="int-br-bond0"}
ovs_version: "2.9.0"
# cat /etc/sysconfig/network-scripts/ifcfg-mgmt0
DEVICE=mgmt0
ONBOOT=yes
BOOTPROTO=static
TYPE=OVSIntPort
DEVICETYPE=ovs
OVS_BRIDGE=br-bond0
HOTPLUG=no
IPADDR=10.1.1.11
PREFIX=24
DEFROUTE=yes
# cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSBond
OVS_BRIDGE="br-bond0"
BOOTPROTO=none
BOND_IFACES="eno5 eno6"
OVS_OPTIONS="bond_mode=active-backup"
HOTPLUG=no
NM_CONTROLLED=no
# egrep -v "^$|^#" /etc/neutron/neutron.conf
[DEFAULT]
auth_strategy = keystone
core_plugin = ml2
service_plugins = router
debug = true
rpc_backend = rabbit
[agent]
[cors]
[database]
[keystone_authtoken]
auth_uri = http://xxxxx:5000
auth_type = password
auth_url = http://xxxxx:35357
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = xxxxx
[matchmaker_redis]
[nova]
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
amqp_durable_queues = True
rabbit_hosts = xxxx:5672,xxxx:5672,xxxx:5672
rabbit_userid = xxxx
rabbit_password = xxxxx
rabbit_retry_interval = 1
rabbit_retry_backoff = 2
rabbit_max_retries = 0
rabbit_ha_queues = True
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[quotas]
[ssl]
# egrep -v "^$|^#" /etc/neutron/plugin.ini
[DEFAULT]
debug = true
[l2pop]
[ml2]
type_drivers = flat,vlan,vxlan
mechanism_drivers = openvswitch,linuxbridge,l2population
[ml2_type_flat]
flat_networks = physnet30
[ml2_type_geneve]
[ml2_type_gre]
tunnel_id_ranges = 1:1000
[ml2_type_vlan]
network_vlan_ranges = physnet30
[ml2_type_vxlan]
[securitygroup]
firewall_driver = neutron.agent.firewall.NoopFirewallDriver
enable_security_group = True
[ovs]
enable_tunneling = False
local_ip = 10.1.1.11
network_vlan_ranges = physnet30
bridge_mappings = physnet30:br-bond0
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1798588
Title:
neutron-openvswitch-agent break network connection on second reboot
Status in neutron:
New
Bug description:
Issue:
======
Installation steps for Openstack Queens Compute Nodes:
1- Install and configure openvswitch, neutron-openvswitch-agent,
openstack-nova-compute,libvirt
2- create virtual bridges
/bin/ovs-vsctl add-br br-int
/bin/ovs-vsctl add-br br-bond0
/bin/ovs-vsctl --may-exist add-bond br-bond0 bond0 eno49 eno50
bond_mode=active-backup
3- add mgmt0 port to br-bond0 bridge which will be used by hypervisor to
connect to openstack controllers by reconfigure
/etc/sysconfig/network-scripts/ifcfg-*
5- delete libvirt default network.
Note: I share the same physical network interfaces between Data Network(VM
network) and Management Network(nova and neutron network to controllers). And
all the physical interfaces are bonding(active-backup).
6- reboot for the first time
After the first reboot the network connection is functioning as expected.
Then try the second reboot, without changing any config, the network never come
back again.
This can be fix by rerun the step 2 and do another reboot but if i reboot
the machine another time, the network will break again. Also, if i stop and
disable neutron-openvswitch-agent, and rerun step 2, the problem will disappear
however many time i reboot the network connection will still function.
I'm not sure if i miss something here but i have this same setup working
from Juno through Ocata, i never test on Pike though.
Setup
=====
OS: CentOS Linux release 7.5.1804 (Core)
Neutron Openvswitch agent tested version:
openstack-neutron-openvswitch-12.0.2-1.el7.noarch,
openstack-neutron-openvswitch-12.0.4-1.el7.noarch
Openvswitch tested version: openvswitch-2.9.0-3, openvswitch-2.9.0-4
Network Setup:
[eno5,eno6] <= Br-bond0[bond0,mgmt0,int-br-bond0] <= Br-int[phy-br-bond0] <=
veths (VMs IF)
# ovs-vsctl show
0f950035-7f7a-4e5f-a337-04ab76945679
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "int-br-bond0"
Interface "int-br-bond0"
type: patch
options: {peer="phy-br-bond0"}
Port br-int
Interface br-int
type: internal
Bridge "br-bond0"
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "mgmt0"
Interface "mgmt0"
type: internal
Port "br-bond0"
Interface "br-bond0"
type: internal
Port "bond0"
Interface "eno6"
Interface "eno5"
Port "phy-br-bond0"
Interface "phy-br-bond0"
type: patch
options: {peer="int-br-bond0"}
ovs_version: "2.9.0"
# cat /etc/sysconfig/network-scripts/ifcfg-mgmt0
DEVICE=mgmt0
ONBOOT=yes
BOOTPROTO=static
TYPE=OVSIntPort
DEVICETYPE=ovs
OVS_BRIDGE=br-bond0
HOTPLUG=no
IPADDR=10.1.1.11
PREFIX=24
DEFROUTE=yes
# cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSBond
OVS_BRIDGE="br-bond0"
BOOTPROTO=none
BOND_IFACES="eno5 eno6"
OVS_OPTIONS="bond_mode=active-backup"
HOTPLUG=no
NM_CONTROLLED=no
# egrep -v "^$|^#" /etc/neutron/neutron.conf
[DEFAULT]
auth_strategy = keystone
core_plugin = ml2
service_plugins = router
debug = true
rpc_backend = rabbit
[agent]
[cors]
[database]
[keystone_authtoken]
auth_uri = http://xxxxx:5000
auth_type = password
auth_url = http://xxxxx:35357
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = xxxxx
[matchmaker_redis]
[nova]
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
amqp_durable_queues = True
rabbit_hosts = xxxx:5672,xxxx:5672,xxxx:5672
rabbit_userid = xxxx
rabbit_password = xxxxx
rabbit_retry_interval = 1
rabbit_retry_backoff = 2
rabbit_max_retries = 0
rabbit_ha_queues = True
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[quotas]
[ssl]
# egrep -v "^$|^#" /etc/neutron/plugin.ini
[DEFAULT]
debug = true
[l2pop]
[ml2]
type_drivers = flat,vlan,vxlan
mechanism_drivers = openvswitch,linuxbridge,l2population
[ml2_type_flat]
flat_networks = physnet30
[ml2_type_geneve]
[ml2_type_gre]
tunnel_id_ranges = 1:1000
[ml2_type_vlan]
network_vlan_ranges = physnet30
[ml2_type_vxlan]
[securitygroup]
firewall_driver = neutron.agent.firewall.NoopFirewallDriver
enable_security_group = True
[ovs]
enable_tunneling = False
local_ip = 10.1.1.11
network_vlan_ranges = physnet30
bridge_mappings = physnet30:br-bond0
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1798588/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
