[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-12-18 Thread James Page
The neutron code does make some attempts to monitor the state of openvswitch - a restart of the ovs database process should be detected by the agent, and appropriate action taken. Having the agent detect and respond to the status of openvswitch and the flows its managing should be the right

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-12-18 Thread Mark Shuttleworth
Is the fix here to ensure that restarts of one are automatically sequenced with restarts of the other service? -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to neutron in Ubuntu. https://bugs.launchpad.net/bugs/1460164 Title: upgrade

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-12-18 Thread James Page
On a fresh icehouse install I see the following on a restart of ovs: 2015-12-18 11:05:26.855 6876 DEBUG neutron.agent.linux.async_process [-] Halting async process [['ovsdb-client', 'monitor', 'Interface', 'name,ofport', '--format=json']]. stop

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-12-18 Thread James Page
Critically, the observation made in comment #8 does not always happen - i.e. I don't reliable seen the openvswitch-agent process exiting abnormally and then restarting all flows. I suspect this is a racey in some way - so if you luck out with a CRITICAL failure on an ovs restart, ovs gets

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-12-18 Thread James Page
This commit is not in Icehouse, and looks to improve the general error handling in this instance: https://github.com/openstack/neutron/commit/91b7fc7f162751936f7cb15d4add932a4aebd55b -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-12-18 Thread James Page
I retested with Liberty, and saw pretty much the same behaviour; digging into this a bit deeper, I think this is related to the l2-population driver usage - with l2 pop disabled, a restart of ovs resulted in a short network outage for the instances, but service was restored quickly - with l2 pop

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-12-18 Thread James Page
Confirmed that disabling l2-population has the same effect on restarts of ovs on Icehouse as well. ** Summary changed: - upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent + restart of openvswitch-switch causes instance network down when l2population enabled --

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-12-17 Thread JuanJo Ciarlante
FYI today's openvswitch-switch upgrade triggered a cluster-wide outage on one (or more) of our production openstacks. ** Tags added: canonical-bootstack -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to neutron in Ubuntu.

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-07-24 Thread James Page
** Changed in: neutron (Ubuntu) Status: Confirmed = Triaged ** Changed in: neutron (Ubuntu) Importance: Undecided = High -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to neutron in Ubuntu.

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-06-01 Thread Launchpad Bug Tracker
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: neutron (Ubuntu) Status: New = Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to neutron in Ubuntu.

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-06-01 Thread James Page
Related bug - https://bugs.launchpad.net/tripleo/+bug/1290486 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to neutron in Ubuntu. https://bugs.launchpad.net/bugs/1460164 Title: upgrade of openvswitch-switch can sometimes break

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-06-01 Thread James Page
This is caused by the restart of the ovs daemons (part of the upgrade process - its done post install to minimize downtime): 2015-05-28 14:24:18.336 47866 ERROR neutron.agent.linux.ovsdb_monitor [-] Error received from ovsdb monitor: ovsdb-client: unix:/var/run/openvswitch/db.sock: receive failed

[Bug 1460164] Re: upgrade of openvswitch-switch can sometimes break neutron-plugin-openvswitch-agent

2015-05-29 Thread James Troup
I should have said; both clouds are Ubuntu 14.04 running OpenStack Icehouse. I've put all the relevant logs I could think of/find up at: https://chinstrap.canonical.com/~james/nx/vector-logs.tar.xz (It's only accessible by Canonical people, sorry.) -- You received this bug notification