I operate 2 neutron gateways ( https://jujucharms.com/neutron-gateway/
<https://jujucharms.com/neutron-gateway/> ) in a freshly installed Openstack
environment. After about an hour, the following problem occurs: When I create a
new virtual router and connect a virtual network with it, then the interface
remains down. In the neutron-openvswitch-agent.log the following errors occur
every 10 seconds:
2018-12-06 07:49:19.011 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch
[req-664e8eff-df0a-4c04-b51a-ddab7806809f - - - - -] ofctl request
version=0x4,msg_type=0x12,msg_len=0x38,xid=0x5515ee88,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1)
timed out: eventlet.timeout.Timeout: 10 seconds
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
[req-664e8eff-df0a-4c04-b51a-ddab7806809f - - - - -] Failed to communicate with
the switch: RuntimeError: ofctl request
version=0x4,msg_type=0x12,msg_len=0x38,xid=0x5515ee88,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1)
timed out
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int Traceback
(most recent call last):
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py",
line 92, in _send_msg
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int result =
ofctl_api.send_msg(self._app, msg, reply_cls, reply_multi)
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/ryu/app/ofctl/api.py", line 89, in send_msg
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
reply_multi=reply_multi))()
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/ryu/base/app_manager.py", line 279, in
send_request
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int return
req.reply_q.get()
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/eventlet/queue.py", line 313, in get
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int return
waiter.wait()
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/eventlet/queue.py", line 141, in wait
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int return
get_hub().switch()
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int return
self.greenlet.switch()
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
eventlet.timeout.Timeout: 10 seconds
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int During
handling of the above exception, another exception occurred:
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int Traceback
(most recent call last):
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/br_int.py",
line 52, in check_canary_table
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int flows =
self.dump_flows(constants.CANARY_TABLE)
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py",
line 162, in dump_flows
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
reply_multi=True)
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py",
line 110, in _send_msg
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int raise
RuntimeError(m)
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
RuntimeError: ofctl request
version=0x4,msg_type=0x12,msg_len=0x38,xid=0x5515ee88,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1)
timed out
2018-12-06 07:49:19.012 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
2018-12-06 07:49:19.013 4782 WARNING
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-664e8eff-df0a-4c04-b51a-ddab7806809f - - - - -] OVS is dead.
OVSNeutronAgent will keep running and checking OVS status periodically.
2018-12-06 07:49:29.015 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch
[req-664e8eff-df0a-4c04-b51a-ddab7806809f - - - - -] ofctl request
version=0x4,msg_type=0x12,msg_len=0x38,xid=0x5515ee8a,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1)
timed out: eventlet.timeout.Timeout: 10 seconds
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
[req-664e8eff-df0a-4c04-b51a-ddab7806809f - - - - -] Failed to communicate with
the switch: RuntimeError: ofctl request
version=0x4,msg_type=0x12,msg_len=0x38,xid=0x5515ee8a,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1)
timed out
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int Traceback
(most recent call last):
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py",
line 92, in _send_msg
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int result =
ofctl_api.send_msg(self._app, msg, reply_cls, reply_multi)
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/ryu/app/ofctl/api.py", line 89, in send_msg
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
reply_multi=reply_multi))()
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/ryu/base/app_manager.py", line 279, in
send_request
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int return
req.reply_q.get()
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/eventlet/queue.py", line 313, in get
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int return
waiter.wait()
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/eventlet/queue.py", line 141, in wait
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int return
get_hub().switch()
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int return
self.greenlet.switch()
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
eventlet.timeout.Timeout: 10 seconds
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int During
handling of the above exception, another exception occurred:
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int Traceback
(most recent call last):
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/br_int.py",
line 52, in check_canary_table
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int flows =
self.dump_flows(constants.CANARY_TABLE)
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py",
line 162, in dump_flows
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
reply_multi=True)
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py",
line 110, in _send_msg
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int raise
RuntimeError(m)
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
RuntimeError: ofctl request
version=0x4,msg_type=0x12,msg_len=0x38,xid=0x5515ee8a,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1)
timed out
2018-12-06 07:49:29.017 4782 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
2018-12-06 07:49:29.018 4782 WARNING
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-664e8eff-df0a-4c04-b51a-ddab7806809f - - - - -] OVS is dead.
OVSNeutronAgent will keep running and checking OVS status periodically.
Ubuntu release: Ubuntu 18.04.1 LTS
Package versions:
neutron-common 2:13.0.1-0ubuntu1~cloud0
neutron-dhcp-agent 2:13.0.1-0ubuntu1~cloud0
neutron-fwaas-common 1:13.0.0-0ubuntu1~cloud0
neutron-l3-agent 2:13.0.1-0ubuntu1~cloud0
neutron-lbaas-common 2:13.0.0-0ubuntu1~cloud0
neutron-lbaasv2-agent 2:13.0.0-0ubuntu1~cloud0
neutron-metadata-agent 2:13.0.1-0ubuntu1~cloud0
neutron-metering-agent 2:13.0.1-0ubuntu1~cloud0
neutron-openvswitch-agent 2:13.0.1-0ubuntu1~cloud0
openvswitch-switch 2.10.0-0ubuntu2~cloud0
openvswitch-common 2.10.0-0ubuntu2~cloud0
CPU-Load average between 2-3 with 8 cores.
When I run the command: service ovs-vswitchd restart
The connections goes down and online again.
After I did that, I can again create virtual routers and connect interfaces,
which are goes active.
The errors are gone for an hour, but come back later. (Sometimes it works for
4-5 hours)
Any ideas what I can do?_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
