Public bug reported:
This is using the latest nova from trunk. In our deployment, we had a
hypervisor go down and the tenant issued a hard reboot prior. When
attempting a reboot on a guest with the state HARD_REBOOT, nova
controller throws this error in it's logs and returns 'ERROR: The server
has either erred or is incapable of performing the requested operation.
(HTTP 500)' to the user:
2014-03-06 18:21:00,535 (routes.middleware): DEBUG middleware __call__ Matched
POST /tenant1/servers/778032b2-469d-445e-abde-7b9b0b673324/action
2014-03-06 18:21:00,536 (routes.middleware): DEBUG middleware __call__ Route
path: '/{project_id}/servers/:(id)/action', defaults: {'action': u'action',
'controller': <nova.api.openstack.wsgi.Resource object at 0x5242c90>}
2014-03-06 18:21:00,536 (routes.middleware): DEBUG middleware __call__ Match
dict: {'action': u'action', 'controller': <nova.api.openstack.wsgi.Resource
object at 0x5242c90>, 'project_id': u'tenant1', 'id':
u'778032b2-469d-445e-abde-7b9b0b673324'}
2014-03-06 18:21:00,537 (nova.api.openstack.wsgi): DEBUG wsgi _process_stack
Action: 'action', body: {"reboot": {"type": "SOFT"}}
2014-03-06 18:21:00,538 (nova.api.openstack.wsgi): DEBUG wsgi _process_stack
Calling method <bound method Controller._action_reboot of
<nova.api.openstack.compute.contrib.keypairs.Controller object at 0x4c35a50>>
2014-03-06 18:21:00,747 (nova.api.openstack): ERROR __init__ _error Caught
error: Unexpected task state: expecting [None, 'rebooting'] but the actual
state is rebooting_hard
Traceback (most recent call last):
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/api/openstack/__init__.py",
line 125, in __call__
return req.get_response(self.application)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/request.py",
line 1320, in send
application, catch_exc_info=False)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/request.py",
line 1284, in call_application
app_iter = application(self.environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 144, in __call__
return resp(environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 598, in __call__
return self.app(env, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 144, in __call__
return resp(environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 144, in __call__
return resp(environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/routes/middleware.py",
line 131, in __call__
response = self.app(environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 144, in __call__
return resp(environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 130, in __call__
resp = self.call_func(req, *args, **self.kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 195, in call_func
return self.func(req, *args, **kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/api/openstack/wsgi.py",
line 925, in __call__
content_type, body, accept)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/api/openstack/wsgi.py",
line 987, in _process_stack
action_result = self.dispatch(meth, request, action_args)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/api/openstack/wsgi.py",
line 1074, in dispatch
return method(req=request, **action_args)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/api/openstack/compute/servers.py",
line 1145, in _action_reboot
self.compute_api.reboot(context, instance, reboot_type)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/compute/api.py",
line 199, in wrapped
return func(self, context, target, *args, **kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/compute/api.py",
line 189, in inner
return function(self, context, instance, *args, **kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/compute/api.py",
line 170, in inner
return f(self, context, instance, *args, **kw)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/compute/api.py",
line 2073, in reboot
instance.save(expected_task_state=[None, task_states.REBOOTING])
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/objects/base.py",
line 151, in wrapper
return fn(self, ctxt, *args, **kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/objects/instance.py",
line 472, in save
columns_to_join=_expected_cols(expected_attrs))
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/db/api.py",
line 739, in instance_update_and_get_original
columns_to_join=columns_to_join)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py",
line 128, in wrapper
return f(*args, **kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py",
line 2164, in instance_update_and_get_original
columns_to_join=columns_to_join)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py",
line 2215, in _instance_update
actual=actual_state, expected=expected)
UnexpectedTaskStateError: Unexpected task state: expecting [None, 'rebooting']
but the actual state is rebooting_hard
2014-03-06 18:21:00,750 (nova.api.openstack): INFO __init__ _error
http://nova-controller.isg.apple.com:8774/v2/tenant1/servers/778032b2-469d-445e-abde-7b9b0b673324/action
returned with HTTP 500
The actual error message back to the user must be something along the lines of
'Unexpected task state: expecting [None, 'rebooting'] but the actual state is
rebooting_hard' with a 4xx HTTP code.
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1288926
Title:
incorrect error code when rebooting a rebooting_hard guest
Status in OpenStack Compute (Nova):
New
Bug description:
This is using the latest nova from trunk. In our deployment, we had a
hypervisor go down and the tenant issued a hard reboot prior. When
attempting a reboot on a guest with the state HARD_REBOOT, nova
controller throws this error in it's logs and returns 'ERROR: The
server has either erred or is incapable of performing the requested
operation. (HTTP 500)' to the user:
2014-03-06 18:21:00,535 (routes.middleware): DEBUG middleware __call__
Matched POST /tenant1/servers/778032b2-469d-445e-abde-7b9b0b673324/action
2014-03-06 18:21:00,536 (routes.middleware): DEBUG middleware __call__ Route
path: '/{project_id}/servers/:(id)/action', defaults: {'action': u'action',
'controller': <nova.api.openstack.wsgi.Resource object at 0x5242c90>}
2014-03-06 18:21:00,536 (routes.middleware): DEBUG middleware __call__ Match
dict: {'action': u'action', 'controller': <nova.api.openstack.wsgi.Resource
object at 0x5242c90>, 'project_id': u'tenant1', 'id':
u'778032b2-469d-445e-abde-7b9b0b673324'}
2014-03-06 18:21:00,537 (nova.api.openstack.wsgi): DEBUG wsgi _process_stack
Action: 'action', body: {"reboot": {"type": "SOFT"}}
2014-03-06 18:21:00,538 (nova.api.openstack.wsgi): DEBUG wsgi _process_stack
Calling method <bound method Controller._action_reboot of
<nova.api.openstack.compute.contrib.keypairs.Controller object at 0x4c35a50>>
2014-03-06 18:21:00,747 (nova.api.openstack): ERROR __init__ _error Caught
error: Unexpected task state: expecting [None, 'rebooting'] but the actual
state is rebooting_hard
Traceback (most recent call last):
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/api/openstack/__init__.py",
line 125, in __call__
return req.get_response(self.application)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/request.py",
line 1320, in send
application, catch_exc_info=False)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/request.py",
line 1284, in call_application
app_iter = application(self.environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 144, in __call__
return resp(environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/keystoneclient/middleware/auth_token.py",
line 598, in __call__
return self.app(env, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 144, in __call__
return resp(environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 144, in __call__
return resp(environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/routes/middleware.py",
line 131, in __call__
response = self.app(environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 144, in __call__
return resp(environ, start_response)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 130, in __call__
resp = self.call_func(req, *args, **self.kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/webob/dec.py",
line 195, in call_func
return self.func(req, *args, **kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/api/openstack/wsgi.py",
line 925, in __call__
content_type, body, accept)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/api/openstack/wsgi.py",
line 987, in _process_stack
action_result = self.dispatch(meth, request, action_args)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/api/openstack/wsgi.py",
line 1074, in dispatch
return method(req=request, **action_args)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/api/openstack/compute/servers.py",
line 1145, in _action_reboot
self.compute_api.reboot(context, instance, reboot_type)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/compute/api.py",
line 199, in wrapped
return func(self, context, target, *args, **kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/compute/api.py",
line 189, in inner
return function(self, context, instance, *args, **kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/compute/api.py",
line 170, in inner
return f(self, context, instance, *args, **kw)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/compute/api.py",
line 2073, in reboot
instance.save(expected_task_state=[None, task_states.REBOOTING])
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/objects/base.py",
line 151, in wrapper
return fn(self, ctxt, *args, **kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/objects/instance.py",
line 472, in save
columns_to_join=_expected_cols(expected_attrs))
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/db/api.py",
line 739, in instance_update_and_get_original
columns_to_join=columns_to_join)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py",
line 128, in wrapper
return f(*args, **kwargs)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py",
line 2164, in instance_update_and_get_original
columns_to_join=columns_to_join)
File
"/usr/local/gshare/csi-nova.venv/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py",
line 2215, in _instance_update
actual=actual_state, expected=expected)
UnexpectedTaskStateError: Unexpected task state: expecting [None,
'rebooting'] but the actual state is rebooting_hard
2014-03-06 18:21:00,750 (nova.api.openstack): INFO __init__ _error
http://nova-controller.isg.apple.com:8774/v2/tenant1/servers/778032b2-469d-445e-abde-7b9b0b673324/action
returned with HTTP 500
The actual error message back to the user must be something along the lines
of 'Unexpected task state: expecting [None, 'rebooting'] but the actual state
is rebooting_hard' with a 4xx HTTP code.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1288926/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
