On 10/19/2016 08:36 AM, Ian Cordasco wrote:
Hey Kevin, So just looking at the pastes you have here, I'm inclined to believe this is actually a bug in oslo_db/sqlalchemy. If you follow the trace, there's a PyMySQL InternalError not being handled inside of sqlalchemy. I'm not sure if SQLAlchemy considers InternalErrors to be something it cannot retry, or something that the user should decide how to handle, but I would start chatting with the folk who work on oslo_db and SQLAlchemy in the community.
SQLAlchemy itself does not retry transactions. A retry is typically at the method level where the calling application (nova in this case) would make use of the oslo retry decorator, seen here: https://github.com/openstack/oslo.db/blob/master/oslo_db/api.py#L85 . This decorator is configured to retry based on specific oslo-level exceptions being intercepted, of which DBDeadlock is the primary exception this function was written for.
In this case, both stack traces illustrate the error being thrown is DBDeadlock, which is an oslo-db-specific error that is the result of the correct handling of this PyMySQL error code. The original error object is maintained as a data member of DBDeadlock so that the source of the DBDeadlock can be seen. The declaration of this interception is here: https://github.com/openstack/oslo.db/blob/master/oslo_db/sqlalchemy/exc_filters.py#L56 . SQLAlchemy re-throws this user-generated exception in the context of the original, so in Python 2 where stack traces are still a confusing affair, it's hard to see that this interception occurred, but DBDeadlock indicates that it has.
That said, this also looks like something that should be reported to Nova. Something causing an unhandled exception is definitely bug worth (even if the fix belongs somewhere in one of its dependencies). -- Ian Cordasco __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
