Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-11 Thread Joe Gordon
Nova is having a related issue, where we are hitting issues in our unit tests. https://bugs.launchpad.net/nova/+bug/1328997 Stderr: 'ERROR: database openstack_citest is being accessed by other users\\nDETAIL: There are 1 other session(s) using the database.\\n\''

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-09 Thread Devananda van der Veen
There may be some problems with MySQL when testing parallel writes in different non-committing transactions, even in READ COMMITTED mode, due to InnoDB locking, if the queries use non-unique secondary indexes for UPDATE or SELECT..FOR UPDATE queries. This is done by the with_lockmode('update')

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-09 Thread Mike Bayer
On Jun 9, 2014, at 12:50 PM, Devananda van der Veen devananda@gmail.com wrote: There may be some problems with MySQL when testing parallel writes in different non-committing transactions, even in READ COMMITTED mode, due to InnoDB locking, if the queries use non-unique secondary indexes

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-09 Thread Jay Pipes
On 06/09/2014 12:50 PM, Devananda van der Veen wrote: There may be some problems with MySQL when testing parallel writes in different non-committing transactions, even in READ COMMITTED mode, due to InnoDB locking, if the queries use non-unique secondary indexes for UPDATE or SELECT..FOR UPDATE

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-09 Thread Devananda van der Veen
On Mon, Jun 9, 2014 at 10:49 AM, Jay Pipes jaypi...@gmail.com wrote: On 06/09/2014 12:50 PM, Devananda van der Veen wrote: There may be some problems with MySQL when testing parallel writes in different non-committing transactions, even in READ COMMITTED mode, due to InnoDB locking, if the

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-09 Thread Jay Pipes
On 06/09/2014 02:57 PM, Devananda van der Veen wrote: On Mon, Jun 9, 2014 at 10:49 AM, Jay Pipes jaypi...@gmail.com wrote: On 06/09/2014 12:50 PM, Devananda van der Veen wrote: There may be some problems with MySQL when testing parallel writes in different non-committing transactions, even in

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-09 Thread Mike Bayer
On Jun 9, 2014, at 1:08 PM, Mike Bayer mba...@redhat.com wrote: On Jun 9, 2014, at 12:50 PM, Devananda van der Veen devananda@gmail.com wrote: There may be some problems with MySQL when testing parallel writes in different non-committing transactions, even in READ COMMITTED mode,

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-09 Thread Devananda van der Veen
Mike, For the typical case, your proposal sounds reasonable to me. That should protect against cross-session locking while still getting the benefits of testing DML without committing to disk. The issue I was originally raising is, of course, the special case -- testing of migrations -- which, I

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-08 Thread Roman Podoliaka
Hi Deva, I haven't actually touched Ironic db migrations tests code yet, but your feedback is very valuable for oslo.db maintainers, thank you! So currently, there are two ways to run migrations tests: 1. Opportunistically (using openstack_citest user credentials; this is how we test migrations

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-08 Thread Roman Podoliaka
Hi Mike, However, when testing an application that uses a fixed set of tables, as should be the case for the majority if not all Openstack apps, there’s no reason that these tables need to be recreated for every test. This is a very good point. I tried to use the recipe from SQLAlchemy docs

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-08 Thread Mike Bayer
On Jun 8, 2014, at 11:46 AM, Roman Podoliaka rpodoly...@mirantis.com wrote: Overall, the approach with executing a test within a transaction and then emitting ROLLBACK worked quite well. The only problem I ran into were tests doing ROLLBACK on purpose. But you've updated the recipe since

Re: [openstack-dev] [Oslo] [Ironic] DB migration woes

2014-06-07 Thread Mike Bayer
On Jun 6, 2014, at 8:12 PM, Devananda van der Veen devananda@gmail.com wrote: I think some things are broken in the oslo-incubator db migration code. Ironic moved to this when Juno opened and things seemed fine, until recently when Lucas tried to add a DB migration and noticed that it