** Changed in: nova/havana
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1274341
Title:
reservation_commit can lead to deadlock
Status in OpenStack Compute (Nova):
Fix Released
Status in OpenStack Compute (nova) havana series:
Fix Released
Bug description:
reservation_commit can deadlock under load:
2014-01-29 18:51:00.470 ERROR nova.quota
[req-659db2df-6124-4e6a-b86b-700aae1de805 183137d95c584efb84e773a21f2ef7a1
d8d5b06deffc45e7b258eb65ea04017c] Failed to commit reservations
['5bea6421-1648-4fe1-9d21-4232f536e031',
'8b27edda-f40e-476c-a9b3-d007aa3f6aac', '58e357df-b45d-4008-9ef3-0da3d8daebbb']
2014-01-29 18:51:00.470 4380 TRACE nova.quota Traceback (most recent call
last):
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/nova/quota.py", line 982, in commit
2014-01-29 18:51:00.470 4380 TRACE nova.quota
self._driver.commit(context, reservations, project_id=project_id)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/nova/quota.py", line 370, in commit
2014-01-29 18:51:00.470 4380 TRACE nova.quota
db.reservation_commit(context, reservations, project_id=project_id)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/nova/db/api.py", line 970, in
reservation_commit
2014-01-29 18:51:00.470 4380 TRACE nova.quota project_id=project_id)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 114, in
wrapper
2014-01-29 18:51:00.470 4380 TRACE nova.quota return f(*args, **kwargs)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 2786, in
reservation_commit
2014-01-29 18:51:00.470 4380 TRACE nova.quota for reservation in
reservation_query.all():
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2115, in all
2014-01-29 18:51:00.470 4380 TRACE nova.quota return list(self)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2227, in
__iter__
2014-01-29 18:51:00.470 4380 TRACE nova.quota return
self._execute_and_instances(context)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2242, in
_execute_and_instances
2014-01-29 18:51:00.470 4380 TRACE nova.quota result =
conn.execute(querycontext.statement, self._params)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1449, in
execute
2014-01-29 18:51:00.470 4380 TRACE nova.quota params)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1584, in
_execute_clauseelement
2014-01-29 18:51:00.470 4380 TRACE nova.quota compiled_sql,
distilled_params
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1698, in
_execute_context
2014-01-29 18:51:00.470 4380 TRACE nova.quota context)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1691, in
_execute_context
2014-01-29 18:51:00.470 4380 TRACE nova.quota context)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 331, in
do_execute
2014-01-29 18:51:00.470 4380 TRACE nova.quota cursor.execute(statement,
parameters)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
2014-01-29 18:51:00.470 4380 TRACE nova.quota self.errorhandler(self,
exc, value)
2014-01-29 18:51:00.470 4380 TRACE nova.quota File
"/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in
defaulterrorhandler
2014-01-29 18:51:00.470 4380 TRACE nova.quota raise errorclass, errorvalue
2014-01-29 18:51:00.470 4380 TRACE nova.quota OperationalError:
(OperationalError) (1213, 'Deadlock found when trying to get lock; try
restarting transaction') 'SELECT reservations.created_at AS
reservations_created_at, reservations.updated_at AS reservations_updated_at,
reservations.deleted_at AS reservations_deleted_at, reservations.deleted AS
reservations_deleted, reservations.id AS reservations_id, reservations.uuid AS
reservations_uuid, reservations.usage_id AS reservations_usage_id,
reservations.project_id AS reservations_project_id, reservations.resource AS
reservations_resource, reservations.delta AS reservations_delta,
reservations.expire AS reservations_expire \nFROM reservations \nWHERE
reservations.deleted = %s AND reservations.uuid IN (%s, %s, %s) FOR UPDATE' (0,
'5bea6421-1648-4fe1-9d21-4232f536e031', '8b27edda-f40e-476c-a9b3-d007aa3f6aac',
'58e357df-b45d-4008-9ef3-0da3d8daebbb')
This can be fixed with our @_retry_on_deadlock wrapper
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1274341/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
