Am Mittwoch, 30. Oktober 2013, 19:02:33 schrieb Dheerendra: > also look at the logs /var/log/nova/nova-network.log on compute node. It > should tell whether it is able to connect to rabbit on controller.
I got now! The problem was a missing database entry on the compute node. Sorry guys for this stupid error, though the docs don't mention creating the entry. Thanks for your help! Florian > > -Dheerendra > > > On Wed, Oct 30, 2013 at 6:41 PM, Dheerendra < > > [email protected]> wrote: > > Hi Florian > > > > I observed this kind of issue when there is race condition. > > > > 1. RabbitMQ is NOT running on controller-node > > 2. You issue nova-network from compute-node. Now this step hangs. > > 3. Just see what is happening in nova-api.log on controller(tail -f > > /var/log/nova/nova-api.log) > > 4. Now you start the RabbitMQ on the controller node. > > 5. nova-network command compute returns the error. Also you will see > > 'timeout' error in nova-api.log. > > > > From your log I suspect this because initially RabitMQ was not up. Looks > > like you started it afterwards. > > Did you try to run the same 'nova-network' command for the second time ? > > i.e after restarting the rabbitMQ ? > > > > -Dheerendra > > > > On Wed, Oct 30, 2013 at 3:47 PM, Florian Lindner <[email protected]>wrote: > >> Am Dienstag, 29. Oktober 2013, 14:32:00 schrieb Razique Mahroua: > >> > You are using Neutron? > >> > Looks like the Neutron server is unable to perform the request actually > >> > >> Actually no, I don't think so, I think I use the nova-network. > >> > >> The documentation > >> http://docs.openstack.org/havana/install-guide/install/apt/content/ says: > >> > >> Configuring Networking can be one of the most bewildering experiences you > >> will > >> encounter when working with OpenStack. To assist in this we have chosen > >> the > >> simplest production-ready configuration for this guide: the legacy > >> networking > >> in OpenStack Compute, with a flat network, that takes care of DHCP. > >> > >> This setup uses "multi-host" functionality: the networking is configured > >> to be > >> highly available by splitting networking functionality across multiple > >> hosts. > >> As a result, there is no single network controller that acts as a single > >> point > >> of failure. Because each compute node is configured for networking in > >> this > >> setup, no additional networking configuration is required on the > >> controller. > >> > >> (Chapter 5: Enabling Networking) > >> > >> and I have installed the nova-network package. > >> > >> I've downloaded my entire /etc to http://xgm.de/upload/etc/ if you or > >> anyone > >> else wants to have a look! Yes, these "123" are the actual passwords, > >> it's > >> just from a VM on my local computer. ;-) > >> > >> Thanks a lot! > >> > >> Florian > >> > >> > On October 29, 2013 at 12:00:18, Florian Lindner ([email protected]) > >> > wrote: > >> > > >> > Am Montag, 28. Oktober 2013, 13:44:00 schrieb Craig E. Ward: > >> > > On the RabbitMQ server, are vhost and user id setup correctly? > >> > > >> > Ok, part of the problem seemed to be that the documentation assumed I > >> > >> set > >> > >> > the password guest for the rabbit guest account. Did that now and the > >> > problem changed: > >> > > >> > root@controller:~# nova network-create vmnet --fixed-range-v4= > >> > >> 10.0.0.0/24 -- > >> > >> > bridge-interface=br100 --multi-host=T > >> > ERROR: The server has either erred or is incapable of performing the > >> > requested operation. (HTTP 500) (Request-ID: > >> > req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3) > >> > > >> > nova-api.log shows: > >> > > >> > http://pastebin.com/raw.php?i=13U8a4Na > >> > > >> > > >> > 2013-10-29 19:14:18.587 1272 INFO nova.osapi_compute.wsgi.server [-] > >> > >> (1272) > >> > >> > accepted ('192.168.122.10', 53116) > >> > 2013-10-29 19:14:19.207 1272 INFO nova.openstack.common.rpc.common > >> > [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3 > >> > >> a1dd41c156c64810b41d46d4062a2b17 > >> > >> > 68d6cb4e2ceb4700931b6489ad25fb6f] Connected to AMQP server on > >> > controller:5672 2013-10-29 19:14:19.218 1272 INFO > >> > nova.openstack.common.rpc.common > >> > >> [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3 > >> > >> > a1dd41c156c64810b41d46d4062a2b17 68d6cb4e2ceb4700931b6489ad25fb6f] > >> > Connected to AMQP server on controller:5672 2013-10-29 19:15:19.238 > >> > 1272 > >> > ERROR nova.api.openstack [req-900a80bd- dfc1-49fc-9599-239fd5b9b7e3 > >> > a1dd41c156c64810b41d46d4062a2b17 > >> > 68d6cb4e2ceb4700931b6489ad25fb6f] Caught error: Timeout while waiting > >> > >> on RPC > >> > >> > response - topic: "network", RPC method: "create_networks" info: > >> > "<unknown>" 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack > >> > >> Traceback > >> > >> > (most recent call last): > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line > >> > >> 119, > >> > >> > in __call__ > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > req.get_response(self.application) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack application, > >> > catch_exc_info=False) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in > >> > call_application > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack app_iter = > >> > application(self.environ, start_response) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > >> resp(environ, > >> > >> > start_response) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > >> "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py > >> ", > >> > >> > line 539, in __call__ > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > >> self.app(env, > >> > >> > start_response) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > >> resp(environ, > >> > >> > start_response) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > >> resp(environ, > >> > >> > start_response) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in > >> > __call__ 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack response > >> > >> = > >> > >> > self.app(environ, start_response) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > >> resp(environ, > >> > >> > start_response) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__ > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack resp = > >> > self.call_func(req, *args, **self.kwargs) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > self.func(req, *args, **kwargs) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line > >> > >> 917, in > >> > >> > __call__ > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack content_type, > >> > >> body, > >> > >> > accept) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line > >> > >> 976, in > >> > >> > _process_stack > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack action_result = > >> > self.dispatch(meth, request, action_args) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line > >> > >> 1057, in > >> > >> > dispatch > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > method(req=request, **action_args) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist- > >> > packages/nova/api/openstack/compute/contrib/os_networks.py", line 126, > >> > >> in > >> > >> > create > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack network = > >> > self.network_api.create(context, **params)[0] > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 93, in > >> > >> wrapped > >> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return func(self, > >> > context, *args, **kwargs) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 142, in > >> > >> create > >> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > self.network_rpcapi.create_networks(context, **kwargs) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line 102, in > >> > create_networks > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > self.client.call(ctxt, 'create_networks', **kwargs) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/nova/rpcclient.py", line 85, in call > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > self._invoke(self.proxy.call, ctxt, method, **kwargs) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/nova/rpcclient.py", line 63, in > >> > >> _invoke > >> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return > >> > cast_or_call(ctxt, msg, **self.kwargs) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File > >> > "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py", > >> > >> line > >> > >> > 130, in call > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack exc.info, > >> > real_topic, msg.get('method')) > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack Timeout: Timeout > >> > >> while > >> > >> > waiting on RPC response - topic: "network", RPC method: > >> "create_networks" > >> > >> > info: "<unknown>" > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack > >> > 2013-10-29 19:15:19.249 1272 INFO nova.api.openstack [req-900a80bd- > >> > dfc1-49fc-9599-239fd5b9b7e3 a1dd41c156c64810b41d46d4062a2b17 > >> > 68d6cb4e2ceb4700931b6489ad25fb6f] > >> > http://controller:8774/v2/68d6cb4e2ceb4700931b6489ad25fb6f/os-networks > >> > returned with HTTP 500 > >> > 2013-10-29 19:15:19.255 1272 INFO nova.osapi_compute.wsgi.server > >> > [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3 > >> > >> a1dd41c156c64810b41d46d4062a2b17 > >> > >> > 68d6cb4e2ceb4700931b6489ad25fb6f] 192.168.122.10 "POST > >> > /v2/68d6cb4e2ceb4700931b6489ad25fb6f/os-networks HTTP/1.1" status: 500 > >> > >> len: > >> > 335 time: 60.6673729 > >> > > >> > Sorry, I got not idea... > >> > > >> > Thanks! > >> > Florian > >> > > >> > > % rabbitmqctl list_users > >> > > > >> > > % rabbitmqctl list_vhosts > >> > > > >> > > The user and vhost should match what is in the OpenStack > >> > >> configurations. > >> > >> > > On 10/28/13 1:14 PM, Florian Lindner wrote: > >> > > > Am Montag, 28. Oktober 2013, 12:04:02 schrieb Razique Mahroua: > >> > > >> hey > >> > > >> is «controller» a hostame all ur servers can resolve? > >> > > > > >> > > > Yes, set in /etc/hosts. > >> > > > > >> > > > Regards, > >> > > > Florian > >> > > > > >> > > >> - Razique > >> > > >> > >> > > >> On Oct 28, 2013, at 11:40, Florian Lindner <[email protected]> > >> > >> wrote: > >> > > >>> Hello, > >> > > >>> > >> > > >>> I've installed OpenStack on a Ubuntu 12.04 VM according to the > >> > >> install > >> > >> > > >>> guide. It worked fine until I created the compute node on a > >> > >> second VM > >> > >> > > >>> and > >> > >> > > >>> tried to set up nova networking: > >> http://docs.openstack.org/havana/install-guide/install/apt/content/nov > >> > >> > > >>> a-> >>> ne twork.html > >> > > >>> > >> > > >>> # nova network-create vmnet --fixed-range-v4=10.0.0.0/24--bridge- > >> > > >>> interface=br100 --multi-host=T > >> > > >>> > >> > > >>> just does nothing until canceled. I tried on the controller and > >> > > >>> compute1 > >> > > >>> node. > >> > > >>> > >> > > >>> nova-api.log shows: > >> > > >>> > >> > > >>> ERROR nova.openstack.common.rpc.common AMQP server on > >> > >> controller:5672 > >> > >> > > >>> is > >> > > >>> unreachable: Socket closed. Trying again in 1 seconds. > >> > > >>> INFO nova.openstack.common.rpc.common Reconnecting to AMQP server > >> > >> on > >> > >> > > >>> controller:5672 > >> > > >>> > >> > > >>> controller:~# netstat -lpn | grep beam > >> > > >>> tcp 0 0 0.0.0.0:36595 0.0.0.0:* > >> > > >>> LISTEN > >> > > >>> 1045/beam > >> > > >>> tcp6 0 0 :::5672 :::* > >> > > >>> LISTEN > >> > > >>> 1045/beam > >> > > >>> > >> > > >>> It listens on this port, but IPv6. IMO on linux listing on an > >> > > >>> IPv6 > >> > > >>> port > >> > > >>> also means listening on the same v4 port. > >> > > >>> > >> > > >>> What's wrong with the installation? > >> > >> _______________________________________________ > >> Mailing list: > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > >> Post to : [email protected] > >> Unsubscribe : > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : [email protected] Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
