Can you verify if the servers where rabbitmq instances are running are reachable from other nodes?
# rabbit_hosts=rabbit1:5672,rabbit2:5672,rabbit3:5672 Are the above hosts reachable? If yes, check if rabbitmq is listening on correct ports: netstat –nap | grep rabbit Also, did you set up correct credentials and access for “openstack” user on all rabbitmq hosts? # rabbitmqctl add_user openstack RABBIT_PASS # rabbitmqctl set_permissions openstack ".*" ".*" ".*" -Kaustubh From: Yura Poltoratskiy [mailto:[email protected]] Sent: Thursday, March 24, 2016 9:37 AM To: [email protected] Subject: [Openstack] AMQP server is unreachable / AMQP server closed the connection. Check login credentials Hello, I am a new to OpenStack. I am deploying from scratch and my architecture as such: 2 servers - computes 3 servers - controllers 3 servers - SAN 1 server - network On every controller node I have created lxc contrainer for every service, so I have 3 containers for every service, and each container live on bare server. Operation system - Ubuntu 14.04 with cloud-archive:liberty on each server/container and there are no firewall enabled. Messages server is RabbitMQ, with I have deployed step-by-steb from this guide: http://docs.openstack.org/ha-guide/controller-ha-rabbitmq.html#rabbitmq-services My first try to launch instance was failed. From the log I assume that the problem is near the messages server. Here is some log. nova-compute.log: 2016-03-24 12:00:06.833 11111 ERROR oslo.messaging._drivers.impl_rabbit [req-8e5b0e51-ed48-48cf-9182-81b437246bcf - - - - -] AM QP server on rabbitmq01:5672 is unreachable: [Errno 32] Broken pipe. Trying again in 1 seconds. 2016-03-24 12:00:07.848 11111 INFO oslo.messaging._drivers.impl_rabbit [-] Reconnected to AMQP server on rabbitmq01:5672 or 2016-03-24 12:05:23.910 11111 ERROR oslo.messaging._drivers.impl_rabbit [-] AMQP server rabbitmq01:5672 closed the connection. Check login credentials: Socket closed neutron-server.log: 2016-03-24 12:27:40.140 25737 ERROR oslo.messaging._drivers.impl_rabbit [-] AMQP server on rabbitmq01:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds. 2016-03-24 12:27:41.156 25737 ERROR oslo.messaging._drivers.impl_rabbit [-] AMQP server on rabbitmq01:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. 2016-03-24 12:27:42.202 25737 INFO oslo.messaging._drivers.impl_rabbit [-] Reconnected to AMQP server on rabbitmq02:5672 2016-03-24 12:27:48.786 25737 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer 2016-03-24 12:28:03.580 25737 ERROR oslo.messaging._drivers.impl_rabbit [-] AMQP server on rabbitmq01:5672 is unreachable: [Errno 32] Broken pipe. Trying again in 1 seconds. 2016-03-24 12:28:04.606 25737 INFO oslo.messaging._drivers.impl_rabbit [-] Reconnected to AMQP server on rabbitmq01:5672 [email protected]<mailto:[email protected]>: =INFO REPORT==== 24-Mar-2016::14:18:01 === accepting AMQP connection <0.2831.0> (10.1.1.101:50740<http://10.1.1.101:50740> -> 10.1.1.41:5672<http://10.1.1.41:5672>) =INFO REPORT==== 24-Mar-2016::14:18:01 === accepting AMQP connection <0.2828.0> (10.1.1.101:50738<http://10.1.1.101:50738> -> 10.1.1.41:5672<http://10.1.1.41:5672>) =INFO REPORT==== 24-Mar-2016::14:18:01 === accepting AMQP connection <0.2825.0> (10.1.1.101:50736<http://10.1.1.101:50736> -> 10.1.1.41:5672<http://10.1.1.41:5672>) =INFO REPORT==== 24-Mar-2016::14:18:01 === accepting AMQP connection <0.2834.0> (10.1.1.101:50742<http://10.1.1.101:50742> -> 10.1.1.41:5672<http://10.1.1.41:5672>) =INFO REPORT==== 24-Mar-2016::14:18:01 === Mirrored queue 'reply_6a400f775e434bafb05db9b8b1976c29' in vhost '/': Adding mirror on node rabbit@rabbitmq02: <7323.17821.0> =INFO REPORT==== 24-Mar-2016::14:18:01 === Mirrored queue 'reply_6a400f775e434bafb05db9b8b1976c29' in vhost '/': Adding mirror on node rabbit@rabbitmq03: <7324.18726.0> =INFO REPORT==== 24-Mar-2016::14:18:01 === Mirrored queue 'compute_fanout_aa803659695f48a2ab997e6bb8bf28bf' in vhost '/': Adding mirror on node rabbit@rabbitmq02: <7323.17827.0> =INFO REPORT==== 24-Mar-2016::14:18:01 === Mirrored queue 'compute_fanout_aa803659695f48a2ab997e6bb8bf28bf' in vhost '/': Adding mirror on node rabbit@rabbitmq03: <7324.18732.0> =ERROR REPORT==== 24-Mar-2016::14:19:41 === closing AMQP connection <0.2825.0> (10.1.1.101:50736<http://10.1.1.101:50736> -> 10.1.1.41:5672<http://10.1.1.41:5672>): Missed heartbeats from client, timeout: 20s =ERROR REPORT==== 24-Mar-2016::14:19:41 === closing AMQP connection <0.2834.0> (10.1.1.101:50742<http://10.1.1.101:50742> -> 10.1.1.41:5672<http://10.1.1.41:5672>): Missed heartbeats from client, timeout: 20s =ERROR REPORT==== 24-Mar-2016::14:20:01 === closing AMQP connection <0.2828.0> (10.1.1.101:50738<http://10.1.1.101:50738> -> 10.1.1.41:5672<http://10.1.1.41:5672>): Missed heartbeats from client, timeout: 20s =ERROR REPORT==== 24-Mar-2016::14:20:01 === closing AMQP connection <0.2831.0> (10.1.1.101:50740<http://10.1.1.101:50740> -> 10.1.1.41:5672<http://10.1.1.41:5672>): Missed heartbeats from client, timeout: 20s I reinstalled rabbitmq-server on dedicated server without success. Someone help me pls to fix the problem. I could provide any additional info.
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : [email protected] Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
