Public bug reported:
After configurinf LBaaSv2 with DVR we see ERRORs every minute in the
LBaas log.
Reproducible: 100%
Setps to reproduce:
1. AIO+compute node setup with lbaasV2 enabled.- see the there is active
lbaasv2 agent.
2. configure DVR on the setup. restart neutron. - see lbaas logs
IF NOT REPRODUCIBLE, BEFORE STEP 2 CREATE NETS SUBNETS AND ROUTER AND
THE DELETE. THEN EXECUTE STEP 2.
!!!!!! IF WE FIRST ENBALE DVR AND THEN CONFIGURE LBAAS TO V2 WE DO NOT SEE
THOSE ERRORS.
The log :
2015-08-24 09:12:38.779 16280 ERROR
neutron_lbaas.services.loadbalancer.agent.agent_manager [-] Unable to retrieve
ready devices
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager Traceback (most recent
call last):
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/neutron_lbaas/services/loadbalancer/agent/agent_manager.py",
line 152, in sync_state
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager ready_instances =
set(self.plugin_rpc.get_ready_devices())
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/neutron_lbaas/services/loadbalancer/agent/agent_api.py",
line 36, in get_ready_devices
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager return
cctxt.call(self.context, 'get_ready_devices', host=self.host)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 156, in
call
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager retry=self.retry)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in
_send
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager timeout=timeout,
retry=retry)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
350, in send
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager retry=retry)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
339, in _send
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager result =
self._waiter.wait(msg_id, timeout)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
243, in wait
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager message =
self.waiters.get(msg_id, timeout=timeout)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
149, in get
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager 'to message ID %s'
% msg_id)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager MessagingTimeout: Timed
out waiting for a reply to message ID 770300260dc94a218863238b5b49bbc8
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager
2015-08-24 09:12:38.780 16280 DEBUG neutron.openstack.common.periodic_task [-]
Running periodic task LbaasAgentManager.collect_stats run_periodic_tasks
/usr/lib/python2.7/site-packages/neutron/openstack/common/periodic_task.py:219
2015-08-24 09:12:38.780 16280 WARNING neutron.openstack.common.loopingcall [-]
task <bound method LbaasAgentManager.run_periodic_tasks of
<neutron_lbaas.services.loadbalancer.agent.agent_manager.LbaasAgentManager
object at 0x3f84a10>> run outlasted interval by 50.01 sec
2015-08-24 09:12:39.011 16292 DEBUG oslo_messaging._drivers.amqp [-] UNIQUE_ID
is c50ffdaafbe7459ebc9be07c4e1ea068. _add_unique_id
/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py:258
2015-08-24 09:12:40.252 16292 DEBUG neutron.openstack.common.periodic_task [-]
Running periodic task LbaasAgentManager.collect_stats run_periodic_tasks
/usr/lib/python2.7/site-packages/neutron/openstack/common/periodic_task.py:219
2015-08-24 09:12:40.252 16292 DEBUG neutron.openstack.common.periodic_task [-]
Running periodic task LbaasAgentManager.periodic_resync run_periodic_tasks
/usr/lib/python2.7/site-packages/neutron/openstack/common/periodic_task.py:219
2015-08-24 09:12:48.781 16280 DEBUG neutron.openstack.common.periodic_task [-]
Running periodic task LbaasAgentManager.periodic_resync run_periodic_tasks
/usr/lib/python2.7/site-packages/neutron/openstack/common/periodic_task.py:219
2015-08-24 09:12:48.781 16280 DEBUG oslo_messaging._drivers.amqpdriver [-]
MSG_ID is 37edfd235247414d910146108d8dd80e _send
/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:311
2015-08-24 09:12:48.782 16280 DEBUG oslo_messaging._drivers.amqp [-] UNIQUE_ID
is c07261b036244b95931511c5ba5512fc. _add_unique_id
/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py:258
2015-08-24 09:12:50.253 16292 DEBUG neutron.openstack.common.periodic_task [-]
Running periodic task LbaasAgentManager.collect_stats run_periodic_tasks
/usr/lib/python2.7/site-packages/neutron/openstack/common/periodic_task.py:219
** Affects: neutron
Importance: Undecided
Status: New
** Tags: lbaas
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1487972
Title:
LbbasV2+DVR- we see ERROR message in lbaas log every minute
Status in neutron:
New
Bug description:
After configurinf LBaaSv2 with DVR we see ERRORs every minute in the
LBaas log.
Reproducible: 100%
Setps to reproduce:
1. AIO+compute node setup with lbaasV2 enabled.- see the there is active
lbaasv2 agent.
2. configure DVR on the setup. restart neutron. - see lbaas logs
IF NOT REPRODUCIBLE, BEFORE STEP 2 CREATE NETS SUBNETS AND ROUTER AND
THE DELETE. THEN EXECUTE STEP 2.
!!!!!! IF WE FIRST ENBALE DVR AND THEN CONFIGURE LBAAS TO V2 WE DO NOT SEE
THOSE ERRORS.
The log :
2015-08-24 09:12:38.779 16280 ERROR
neutron_lbaas.services.loadbalancer.agent.agent_manager [-] Unable to retrieve
ready devices
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager Traceback (most recent
call last):
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/neutron_lbaas/services/loadbalancer/agent/agent_manager.py",
line 152, in sync_state
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager ready_instances =
set(self.plugin_rpc.get_ready_devices())
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/neutron_lbaas/services/loadbalancer/agent/agent_api.py",
line 36, in get_ready_devices
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager return
cctxt.call(self.context, 'get_ready_devices', host=self.host)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 156, in
call
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager retry=self.retry)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in
_send
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager timeout=timeout,
retry=retry)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
350, in send
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager retry=retry)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
339, in _send
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager result =
self._waiter.wait(msg_id, timeout)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
243, in wait
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager message =
self.waiters.get(msg_id, timeout=timeout)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager File
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line
149, in get
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager 'to message ID %s'
% msg_id)
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager MessagingTimeout: Timed
out waiting for a reply to message ID 770300260dc94a218863238b5b49bbc8
2015-08-24 09:12:38.779 16280 TRACE
neutron_lbaas.services.loadbalancer.agent.agent_manager
2015-08-24 09:12:38.780 16280 DEBUG neutron.openstack.common.periodic_task
[-] Running periodic task LbaasAgentManager.collect_stats run_periodic_tasks
/usr/lib/python2.7/site-packages/neutron/openstack/common/periodic_task.py:219
2015-08-24 09:12:38.780 16280 WARNING neutron.openstack.common.loopingcall
[-] task <bound method LbaasAgentManager.run_periodic_tasks of
<neutron_lbaas.services.loadbalancer.agent.agent_manager.LbaasAgentManager
object at 0x3f84a10>> run outlasted interval by 50.01 sec
2015-08-24 09:12:39.011 16292 DEBUG oslo_messaging._drivers.amqp [-]
UNIQUE_ID is c50ffdaafbe7459ebc9be07c4e1ea068. _add_unique_id
/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py:258
2015-08-24 09:12:40.252 16292 DEBUG neutron.openstack.common.periodic_task
[-] Running periodic task LbaasAgentManager.collect_stats run_periodic_tasks
/usr/lib/python2.7/site-packages/neutron/openstack/common/periodic_task.py:219
2015-08-24 09:12:40.252 16292 DEBUG neutron.openstack.common.periodic_task
[-] Running periodic task LbaasAgentManager.periodic_resync run_periodic_tasks
/usr/lib/python2.7/site-packages/neutron/openstack/common/periodic_task.py:219
2015-08-24 09:12:48.781 16280 DEBUG neutron.openstack.common.periodic_task
[-] Running periodic task LbaasAgentManager.periodic_resync run_periodic_tasks
/usr/lib/python2.7/site-packages/neutron/openstack/common/periodic_task.py:219
2015-08-24 09:12:48.781 16280 DEBUG oslo_messaging._drivers.amqpdriver [-]
MSG_ID is 37edfd235247414d910146108d8dd80e _send
/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:311
2015-08-24 09:12:48.782 16280 DEBUG oslo_messaging._drivers.amqp [-]
UNIQUE_ID is c07261b036244b95931511c5ba5512fc. _add_unique_id
/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py:258
2015-08-24 09:12:50.253 16292 DEBUG neutron.openstack.common.periodic_task
[-] Running periodic task LbaasAgentManager.collect_stats run_periodic_tasks
/usr/lib/python2.7/site-packages/neutron/openstack/common/periodic_task.py:219
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1487972/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
