Hello guys, Last saturday I needed to turn off my controller node because it was needed to perform a energy maintenance at my lab. After turning on the controller again I could fix the network problems, but I can't fix the cinder one, I'm using Grizzly at Ubuntu Server 12.04 and when I try to attach my storage I got this error:
cinder-api.log: 2013-11-05 11:23:49 INFO [cinder.api.middleware.fault] http://192.168.3.1:8776/v1/d13839320f5d4194a4a3fe3b723d6144/volumes/954d2f1b-837b-4ba5-abfd-b3610597be5e/actionreturned with HTTP 500 2013-11-05 11:23:53 INFO [cinder.api.openstack.wsgi] POST http://192.168.3.1:8776/v1/d13839320f5d4194a4a3fe3b723d6144/volumes/954d2f1b-837b-4ba5-abfd-b3610597be5e/action 2013-11-05 11:24:29 INFO [cinder.api.openstack.wsgi] GET http://192.168.3.1:8776/v1/d13839320f5d4194a4a3fe3b723d6144/volumes/detail 2013-11-05 11:24:29 AUDIT [cinder.api.v1.volumes] vol=<cinder.db.sqlalchemy.models.Volume object at 0x38a2250> 2013-11-05 11:24:29 AUDIT [cinder.api.v1.volumes] vol=<cinder.db.sqlalchemy.models.Volume object at 0x373f5d0> 2013-11-05 11:24:29 AUDIT [cinder.api.v1.volumes] vol=<cinder.db.sqlalchemy.models.Volume object at 0x37a9890> 2013-11-05 11:24:29 INFO [cinder.api.openstack.wsgi] http://192.168.3.1:8776/v1/d13839320f5d4194a4a3fe3b723d6144/volumes/detailreturned with HTTP 200 2013-11-05 11:24:53 ERROR [cinder.api.middleware.fault] Caught error: Timeout while waiting on RPC response. Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cinder/api/middleware/fault.py", line 73, in __call__ return req.get_response(self.application) File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send application, catch_exc_info=False) File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_application app_iter = application(self.environ, start_response) File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ return resp(environ, start_response) File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 450, in __call__ return self.app(env, start_response) File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ return resp(environ, start_response) File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ return resp(environ, start_response) File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__ response = self.app(environ, start_response) File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ return resp(environ, start_response) File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__ resp = self.call_func(req, *args, **self.kwargs) File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func return self.func(req, *args, **kwargs) File "/usr/lib/python2.7/dist-packages/cinder/api/openstack/wsgi.py", line 803, in __call__ content_type, body, accept) File "/usr/lib/python2.7/dist-packages/cinder/api/openstack/wsgi.py", line 851, in _process_stack action_result = self.dispatch(meth, request, action_args) File "/usr/lib/python2.7/dist-packages/cinder/api/openstack/wsgi.py", line 927, in dispatch return method(req=request, **action_args) File "/usr/lib/python2.7/dist-packages/cinder/api/contrib/volume_actions.py", line 137, in _initialize_connection connector) File "/usr/lib/python2.7/dist-packages/cinder/volume/api.py", line 63, in wrapped return func(self, context, target_obj, *args, **kwargs) File "/usr/lib/python2.7/dist-packages/cinder/volume/api.py", line 493, in initialize_connection connector) File "/usr/lib/python2.7/dist-packages/cinder/volume/rpcapi.py", line 117, in initialize_connection volume['host'])) File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/proxy.py", line 80, in call return rpc.call(context, self._get_topic(topic), msg, timeout) File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/__init__.py", line 140, in call return _get_impl().call(CONF, context, topic, msg, timeout) File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/impl_kombu.py", line 798, in call rpc_amqp.get_connection_pool(conf, Connection)) File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/amqp.py", line 613, in call rv = list(rv) File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/amqp.py", line 555, in __iter__ self.done() File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ self.gen.next() File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/amqp.py", line 552, in __iter__ self._iterator.next() File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/impl_kombu.py", line 648, in iterconsume yield self.ensure(_error_callback, _consume) File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/impl_kombu.py", line 566, in ensure error_callback(e) File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/impl_kombu.py", line 629, in _error_callback raise rpc_common.Timeout() Timeout: Timeout while waiting on RPC response. 2013-11-05 11:24:53 INFO [cinder.api.middleware.fault] http://192.168.3.1:8776/v1/d13839320f5d4194a4a3fe3b723d6144/volumes/954d2f1b-837b-4ba5-abfd-b3610597be5e/actionreturned with HTTP 500 2013-11-05 11:24:53 INFO [cinder.api.openstack.wsgi] POST http://192.168.3.1:8776/v1/d13839320f5d4194a4a3fe3b723d6144/volumes/954d2f1b-837b-4ba5-abfd-b3610597be5e/action 2013-11-05 11:24:53 INFO [cinder.api.openstack.wsgi] http://192.168.3.1:8776/v1/d13839320f5d4194a4a3fe3b723d6144/volumes/954d2f1b-837b-4ba5-abfd-b3610597be5e/actionreturned with HTTP 202 My cinder-volume.log and cinder-scheduler.log are empty. The attachment was working before rebooting the controller node. Can anybody please help me to fix it? Thank you all. Guilherme.
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : [email protected] Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
