The bug has been reproduced several times by different people:
https://bugs.launchpad.net/nova/+bug/1611447
** Changed in: nova
Status: Expired => New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1312002
Title:
nova cell-show <cell name> causes ValueError: Circular reference
detected
Status in OpenStack Compute (nova):
New
Bug description:
I am using devstack development environment with n-cell enabled in localrc.
When I call API to list all cells a cell with name "child" is explored.
Again when I try to "cell-show" this cell "child" call is failed with an
error response 500.
following is the operations followed -
Setting in /etc/nova/nova.conf
[cells]
name = region
cell_type = api
enable = True
When API call to get cells -
REQUEST -
curl -i
'http://10.0.9.40:8774/v2/dcab4e3fc2734bad97c43d46bb77d076/os-cells' -X GET -H
"X-Auth-Project-Id: demo" -H "User-Agent: python-novaclient" -H "Accept:
application/json" -H "X-Auth-Token:<token>"
RESPONSE -
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 111
X-Compute-Request-Id: req-e83c337a-a7d9-47e5-a472-72cd41126f21
Date: Thu, 24 Apr 2014 04:12:34 GMT
{"cells": [{"username": "guest", "rpc_host": "10.0.9.40", "type":
"child", "name": "child", "rpc_port": 5672}]}
During particular cell show (using cell name as "child")-
REQUEST -
curl -i
'http://10.0.9.40:8774/v2/dcab4e3fc2734bad97c43d46bb77d076/os-cells/child' -X
GET -H "X-Auth-Project-Id: demo" -H "User-Agent: python-novaclient" -H "Accept:
application/json" -H "X-Auth-Token: <token>"
RESPONSE -
HTTP/1.1 500 Internal Server Error
Content-Length: 128
Content-Type: application/json; charset=UTF-8
X-Compute-Request-Id: req-ba2503bd-3948-4cce-8ee6-543aa597497b
Date: Thu, 24 Apr 2014 04:10:33 GMT
{"computeFault": {"message": "The server has either erred or is
incapable of performing the requested operation.", "code": 500}}
Note: when using cell name as "api" or "region" 404 is returned.
nova-api logs during show call is traced as below -
2014-04-24 11:15:20.219 DEBUG keystoneclient.middleware.auth_token [-]
Authenticating user token from (pid=705) __call__
/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py:603
2014-04-24 11:15:20.220 DEBUG keystoneclient.middleware.auth_token [-]
Removing headers from request environment:
X-Identity-Status,X-Domain-Id,X-Domain-Name,X-Project-Id,X-Project-Name,X-Project-Domain-Id,X-Project-Domain-Name,X-User-Id,X-User-Name,X-User-Domain-Id,X-User-Domain-Name,X-Roles,X-Service-Catalog,X-User,X-Tenant-Id,X-Tenant-Name,X-Tenant,X-Role
from (pid=705) _remove_auth_headers
/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py:662
2014-04-24 11:15:20.232 DEBUG keystoneclient.middleware.auth_token [-]
Storing token in cache from (pid=705) _cache_put
/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py:1121
2014-04-24 11:15:20.233 DEBUG keystoneclient.middleware.auth_token [-]
Received request from user: 6cf0c59310fb4b189fc157b19d0e1026 with project_id :
2c3857a83f454b7cb073891ef47acd11 and roles: heat_stack_owner,admin from
(pid=705) _build_user_headers
/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py:910
2014-04-24 11:15:20.238 DEBUG routes.middleware
[req-3ddd31fe-cd31-4c81-b960-81c753566b4a admin demo] Matched GET
/2c3857a83f454b7cb073891ef47acd11/os-cells/child from (pid=705) __call__
/usr/lib/python2.7/dist-packages/routes/middleware.py:100
2014-04-24 11:15:20.239 DEBUG routes.middleware
[req-3ddd31fe-cd31-4c81-b960-81c753566b4a admin demo] Route path:
'/{project_id}/os-cells/:(id)', defaults: {'action': u'show', 'controller':
<nova.api.openstack.wsgi.Resource object at 0x38e8490>} from (pid=705) __call__
/usr/lib/python2.7/dist-packages/routes/middleware.py:102
2014-04-24 11:15:20.239 DEBUG routes.middleware
[req-3ddd31fe-cd31-4c81-b960-81c753566b4a admin demo] Match dict: {'action':
u'show', 'controller': <nova.api.openstack.wsgi.Resource object at 0x38e8490>,
'project_id': u'2c3857a83f454b7cb073891ef47acd11', 'id': u'child'} from
(pid=705) __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:103
2014-04-24 11:15:20.240 DEBUG nova.api.openstack.wsgi
[req-3ddd31fe-cd31-4c81-b960-81c753566b4a admin demo] Calling method '<bound
method Controller.show of <nova.api.openstack.compute.contrib.cells.Controller
object at 0x3789c10>>' (Content-type='None', Accept='application/json') from
(pid=705) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:945
2014-04-24 11:15:20.253 ERROR nova.api.openstack
[req-3ddd31fe-cd31-4c81-b960-81c753566b4a admin demo] Caught error: Circular
reference detected
Traceback (most recent call last):
File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
133, in _dispatch_and_reply
incoming.message))
File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py",
line 67, in reply
self._send_reply(conn, reply, failure, log_failure=log_failure)
File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py",
line 61, in _send_reply
conn.direct_send(self.reply_q, rpc_common.serialize_msg(msg))
File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/common.py",
line 317, in serialize_msg
_MESSAGE_KEY: jsonutils.dumps(raw_msg)}
File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/openstack/common/jsonutils.py",
line 164, in dumps
return json.dumps(value, default=default, **kwargs)
File "/usr/lib/python2.7/json/__init__.py", line 238, in dumps
**kw).encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
return _iterencode(o, 0)
ValueError: Circular reference detected
2014-04-24 11:15:20.253 TRACE nova.api.openstack Traceback (most recent
call last):
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/opt/stack/nova/nova/api/openstack/__init__.py", line 125, in __call__
2014-04-24 11:15:20.253 TRACE nova.api.openstack return
req.get_response(self.application)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
2014-04-24 11:15:20.253 TRACE nova.api.openstack application,
catch_exc_info=False)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1284, in
call_application
2014-04-24 11:15:20.253 TRACE nova.api.openstack app_iter =
application(self.environ, start_response)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-04-24 11:15:20.253 TRACE nova.api.openstack return resp(environ,
start_response)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py",
line 616, in __call__
2014-04-24 11:15:20.253 TRACE nova.api.openstack return self.app(env,
start_response)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-04-24 11:15:20.253 TRACE nova.api.openstack return resp(environ,
start_response)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-04-24 11:15:20.253 TRACE nova.api.openstack return resp(environ,
start_response)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
2014-04-24 11:15:20.253 TRACE nova.api.openstack response =
self.app(environ, start_response)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-04-24 11:15:20.253 TRACE nova.api.openstack return resp(environ,
start_response)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
2014-04-24 11:15:20.253 TRACE nova.api.openstack resp =
self.call_func(req, *args, **self.kwargs)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
2014-04-24 11:15:20.253 TRACE nova.api.openstack return
self.func(req, *args, **kwargs)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/opt/stack/nova/nova/api/openstack/wsgi.py", line 917, in __call__
2014-04-24 11:15:20.253 TRACE nova.api.openstack content_type, body,
accept)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/opt/stack/nova/nova/api/openstack/wsgi.py", line 983, in _process_stack
2014-04-24 11:15:20.253 TRACE nova.api.openstack action_result =
self.dispatch(meth, request, action_args)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/opt/stack/nova/nova/api/openstack/wsgi.py", line 1070, in dispatch
2014-04-24 11:15:20.253 TRACE nova.api.openstack return
method(req=request, **action_args)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/opt/stack/nova/nova/api/openstack/common.py", line 602, in inner
2014-04-24 11:15:20.253 TRACE nova.api.openstack return
function(*args, **kwargs)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/opt/stack/nova/nova/api/openstack/compute/contrib/cells.py", line 264, in show
2014-04-24 11:15:20.253 TRACE nova.api.openstack cell =
self.cells_rpcapi.cell_get(context, id)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/opt/stack/nova/nova/cells/rpcapi.py", line 450, in cell_get
2014-04-24 11:15:20.253 TRACE nova.api.openstack return
cctxt.call(ctxt, 'cell_get', cell_name=cell_name)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/client.py", line
150, in call
2014-04-24 11:15:20.253 TRACE nova.api.openstack wait_for_reply=True,
timeout=timeout)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/transport.py", line 90,
in _send
2014-04-24 11:15:20.253 TRACE nova.api.openstack timeout=timeout)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py",
line 412, in send
2014-04-24 11:15:20.253 TRACE nova.api.openstack return
self._send(target, ctxt, message, wait_for_reply, timeout)
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py",
line 405, in _send
2014-04-24 11:15:20.253 TRACE nova.api.openstack raise result
2014-04-24 11:15:20.253 TRACE nova.api.openstack ValueError: Circular
reference detected
2014-04-24 11:15:20.253 TRACE nova.api.openstack Traceback (most recent
call last):
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
133, in _dispatch_and_reply
2014-04-24 11:15:20.253 TRACE nova.api.openstack incoming.message))
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py",
line 67, in reply
2014-04-24 11:15:20.253 TRACE nova.api.openstack
self._send_reply(conn, reply, failure, log_failure=log_failure)
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py",
line 61, in _send_reply
2014-04-24 11:15:20.253 TRACE nova.api.openstack
conn.direct_send(self.reply_q, rpc_common.serialize_msg(msg))
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/common.py",
line 317, in serialize_msg
2014-04-24 11:15:20.253 TRACE nova.api.openstack _MESSAGE_KEY:
jsonutils.dumps(raw_msg)}
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/local/lib/python2.7/dist-packages/oslo/messaging/openstack/common/jsonutils.py",
line 164, in dumps
2014-04-24 11:15:20.253 TRACE nova.api.openstack return
json.dumps(value, default=default, **kwargs)
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/lib/python2.7/json/__init__.py", line 238, in dumps
2014-04-24 11:15:20.253 TRACE nova.api.openstack **kw).encode(obj)
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/lib/python2.7/json/encoder.py", line 201, in encode
2014-04-24 11:15:20.253 TRACE nova.api.openstack chunks =
self.iterencode(o, _one_shot=True)
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.253 TRACE nova.api.openstack File
"/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
2014-04-24 11:15:20.253 TRACE nova.api.openstack return
_iterencode(o, 0)
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.253 TRACE nova.api.openstack ValueError: Circular
reference detected
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.253 TRACE nova.api.openstack
2014-04-24 11:15:20.259 INFO nova.api.openstack
[req-3ddd31fe-cd31-4c81-b960-81c753566b4a admin demo]
http://10.0.9.40:8774/v2/2c3857a83f454b7cb073891ef47acd11/os-cells/child
returned with HTTP 500
2014-04-24 11:15:20.259 DEBUG nova.api.openstack.wsgi
[req-3ddd31fe-cd31-4c81-b960-81c753566b4a admin demo] Returning 500 to user:
The server has either erred or is incapable of performing the requested
operation. from (pid=705) __call__
/opt/stack/nova/nova/api/openstack/wsgi.py:1215
2014-04-24 11:15:20.260 INFO nova.osapi_compute.wsgi.server
[req-3ddd31fe-cd31-4c81-b960-81c753566b4a admin demo] 10.0.9.40 "GET
/v2/2c3857a83f454b7cb073891ef47acd11/os-cells/child HTTP/1.1" status: 500 len:
335 time: 0.0411668
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1312002/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
