> there is ongoing work to add the revision number Thanks for pointing out. Then I’ll stick with opt.2.
> Then, the object should before update() fetch the current state and try to > merge the requested changes to the current state of the object I don’t think merging states automatically could be possible without knowledge of business logic (the ab = a + b test case clearly shows that I believe). So I was thinking about throwing the Retry exception and let the user take care of that (e.g. with_db_retry). > On 13 May 2016, at 11:22, Korzeniewski, Artur <[email protected]> > wrote: > > Hi Ilya, > Thanks for investigating the concurrency issue. It is indeed a problem from > multithreaded neutron-server point of view. > > Regarding the opt.2, there is ongoing work to add the revision number[1] to > main neutron resources (port, network, subnet...) This is connected to spec > [2] and Launchpad bug [3]: push all object changes as AMQP notifications. I > think [1] is implementing part of your idea about version number. If version > number will be added to standard attributes table, it will automagically > appear as a new field in object. > > What is left to be done in opt.2 is to handle situation when object is trying > to update the state and the revision is not the latest. Then, the object > should before update() fetch the current state and try to merge the requested > changes to the current state of the object, and then apply it in the DB. Also > it would send the newest object state via AMQP notification, obsoleting the > previous configuration. > > We can also check how nova is handling the concurrency issue in their OVO > usage model. > > Regards, > Artur > IRC: korzen > > [1] https://review.openstack.org/#/c/303966/11 > [2] https://review.openstack.org/#/c/225995 > [3] https://bugs.launchpad.net/neutron/+bug/1516195 > > -----Original Message----- > From: Ilya Chukhnakov [mailto:[email protected]] > Sent: Thursday, May 12, 2016 8:26 PM > To: OpenStack Development Mailing List <[email protected]> > Subject: [openstack-dev] [neutron][ovo] NeutronDbObject concurrency issues > > Hi everyone. > > I’ve recently found that straightforward use of NeutronDbObject is prone to > concurrency-related problems. > > I’ve submitted a patch set [3] with some tests to show that without special > treatment using NeutronDbObject could lead to unexpected results. > > Further patch sets will provide acquire_object/acquire_objects contextmanager > methods to the NeutronDbObject class. These methods are to be used in place > of get_object/get_objects whenever the user intends to make changes to the > object. > These methods would start an autonested_transaction. > > There are (at least) two potential options for the implementation: > > 1. Based on the DB locks (e.g. SELECT FOR UPDATE/SqlAlchemy with_for_update). > > pros: > - the object is guaranteed to not be changed while within the context > > cons: > - prone to deadlocks ([1] and potentially when locking multiple objects) > > 2. Lock-free CAS based on object version counter. Can use SqlAlchemy version > counter [2] or add our own. If conflicting changes are detected upon exiting > the context (i.e. version counter held differs from the one in the DB), will > raise OSLO RetryRequest exception. > > pros: > - does not require locking > > cons: > - require an additional field in the models > > While opt.2 only prevents the conflicting changes, but does not guarantee > that the object does not change while within the context, opt.1 may seem > preferential. But even with opt.1 the user should not expect that the changes > made to the object while within the context will get to the database as the > autonested_transaction could fail on flush/commit. > > So I’d like to hear others’ opinion on the problem and which of the two > implementation options would be preferred? Or maybe someone has a better idea. > > [1] > https://wiki.openstack.org/wiki/OpenStack_and_SQLAlchemy#MySQLdb_.2B_eventlet_.3D_sad > [2] http://docs.sqlalchemy.org/en/rel_0_9/orm/versioning.html > > [3] https://review.openstack.org/#/c/315705/ > > -- > Thanks, > Ilya > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
