Moving the pid files for the affected router solves the issue. mv /var/lib/neutron/ha_confs/c1cc1a5d-c0ef-47b7-8d5c-88403e134725.* /root
Found fix thanks to frickler on IRC. It has been merged for liberty https://review.openstack.org/#/c/299138/3 ** Changed in: cloud-archive Status: New => Invalid ** Changed in: neutron Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1583977 Title: liberty neutron-l3-agent ha failes to spawn keepalived Status in Ubuntu Cloud Archive: Invalid Status in neutron: Invalid Bug description: After upgrading to 7.0.4 I have several routers that fails to spawn the keepalived process. The logs say 2016-05-20 11:01:11.181 23023 ERROR neutron.agent.linux.external_process [-] default-service for router with uuid c1cc1a5d-c0ef-47b7-8d5c-88403e134725 not found. The process should not have died 2016-05-20 11:01:11.181 23023 ERROR neutron.agent.linux.external_process [-] respawning keepalived for uuid c1cc1a5d-c0ef-47b7-8d5c-88403e134725 2016-05-20 11:01:11.182 23023 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-c1cc1a5d-c0ef-47b7-8d5c-88403e134725', 'keepalived', '-P', '-f', '/var/lib/neutron/ha_confs/c1cc1a5d-c0ef-47b7-8d5c-88403e134725/keepalived.conf', '-p', '/var/lib/neutron/ha_confs/c1cc1a5d-c0ef-47b7-8d5c-88403e134725.pid', '-r', '/var/lib/neutron/ha_confs/c1cc1a5d-c0ef-47b7-8d5c-88403e134725.pid-vrrp'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:85 All these spawns fail and keepalived outputs to syslog May 20 11:01:11 neutron1 Keepalived[46558]: Starting Keepalived v1.2.19 (09/04,2015) May 20 11:01:11 neutron1 Keepalived[46558]: daemon is already running but the daemon is not running the only thing running is the neutron-keepalived-state-change root@neutron1:~# ps auxf | grep c1cc1a5d root 48137 0.0 0.0 11740 936 pts/4 S+ 11:03 0:00 | \_ grep --color=auto c1cc1a5d neutron 21671 0.0 0.0 124924 40172 ? S May19 0:00 /usr/bin/python /usr/bin/neutron-keepalived-state-change --router_id=c1cc1a5d-c0ef-47b7-8d5c-88403e134725 --namespace=qrouter-c1cc1a5d-c0ef-47b7-8d5c-88403e134725 --conf_dir=/var/lib/neutron/ha_confs/c1cc1a5-c0ef-47b7-8d5c-88403e134725 --monitor_interface=ha-ef4e2a2f-66 --monitor_cidr=169.254.0.1/24 --pid_file=/var/lib/neutron/external/pids/c1cc1a5d-c0ef-47b7-8d5c-88403e134725.monitor.pid --state_path=/var/lib/neutron --user=107 --group=112 ProblemType: Bug DistroRelease: Ubuntu 14.04 Package: neutron-l3-agent 2:7.0.4-0ubuntu1~cloud0 [origin: Canonical] ProcVersionSignature: Ubuntu 3.13.0-86.131-generic 3.13.11-ckt39 Uname: Linux 3.13.0-86-generic x86_64 NonfreeKernelModules: hcpdriver ApportVersion: 2.14.1-0ubuntu3.20 Architecture: amd64 CrashDB: { "impl": "launchpad", "project": "cloud-archive", "bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml";, } Date: Fri May 20 11:00:01 2016 PackageArchitecture: all SourcePackage: neutron UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1583977/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
