based on the log this may be an oslo messaging issue.
** Also affects: oslo.messaging
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1393182
Title:
nova compute failed to report health when nova conductor started
Status in Compass - Open Deployment Service:
Confirmed
Status in OpenStack Compute (Nova):
New
Status in Messaging API for OpenStack:
New
Bug description:
I have an icehouse openstack deployment that includes one controller and 3
computes. the controller was the last to go up and running. when rabbitmq
started, nova compute tried to connect to it. As the log shows, it seemed that
it finally got connected, but when doing nova service-list, the nova-compute
service was still down. I later restarted nova-compute, and this time, nova
service-list showed that the nova-compute became UP.
I still have two other nova compute remain down status. probably if
restarted, they would become UP as well. If any more info is needed, let me
know.
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line
622, in ensure
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
return method(*args, **kwargs)
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line
702, in _consume
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
return self.connection.drain_events(timeout=timeout)
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/kombu/connection.py", line 139, in
drain_events
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
return self.transport.drain_events(self.connection, **kwargs)
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/kombu/transport/pyamqplib.py", line 223, in
drain_events
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
return connection.drain_events(**kwargs)
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/kombu/transport/pyamqplib.py", line 56, in
drain_events
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
return self.wait_multi(self.channels.values(), timeout=timeout)
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/kombu/transport/pyamqplib.py", line 81, in
wait_multi
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
return amqp_method(channel, args)
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/amqplib/client_0_8/connection.py", line 365,
in _close
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
self._x_close_ok()
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/amqplib/client_0_8/connection.py", line 384,
in _x_close_ok
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
self._send_method((10, 61))
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/amqplib/client_0_8/abstract_channel.py", line
70, in _send_method
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
method_sig, args, content)
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/amqplib/client_0_8/method_framing.py", line
233, in write_method
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
self.dest.write_frame(1, channel, payload)
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/amqplib/client_0_8/transport.py", line 125,
in write_frame
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
frame_type, channel, size, payload, 0xce))
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/eventlet/greenio.py", line 359, in sendall
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
tail = self.send(data, flags)
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit File
"/usr/lib/python2.6/site-packages/eventlet/greenio.py", line 342, in send
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
total_sent += fd.send(data[total_sent:], flags)
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit error:
[Errno 104] Connection reset by peer
2014-11-14 21:34:43.449 4314 TRACE oslo.messaging._drivers.impl_rabbit
2014-11-14 21:34:43.450 4314 INFO oslo.messaging._drivers.impl_rabbit [-]
Reconnecting to AMQP server on 10.1.0.144:5672
2014-11-14 21:34:43.450 4314 INFO oslo.messaging._drivers.impl_rabbit [-]
Delaying reconnect for 1.0 seconds...
2014-11-14 21:34:44.501 4314 INFO oslo.messaging._drivers.impl_rabbit [-]
Connected to AMQP server on 10.1.0.144:5672
2014-11-14 21:34:44.933 4314 INFO oslo.messaging._drivers.impl_rabbit [-]
Reconnecting to AMQP server on 10.1.0.144:5672
2014-11-14 21:34:44.933 4314 INFO oslo.messaging._drivers.impl_rabbit [-]
Delaying reconnect for 1.0 seconds...
2014-11-14 21:34:45.982 4314 INFO oslo.messaging._drivers.impl_rabbit [-]
Connected to AMQP server on 10.1.0.144:5672
To manage notifications about this bug go to:
https://bugs.launchpad.net/compass/+bug/1393182/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
