Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-14 Thread Matthew Booth
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/11/14 19:19, Dan Smith wrote: Unfortunately this model doesn't apply to Nova objects, which are persisted remotely. Unless I've missed something, SQLA doesn't run on Nova Compute at all. Instead, when Nova Compute calls object.save() this

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-14 Thread Matthew Booth
On 13/11/14 18:26, Mike Bayer wrote: On Nov 13, 2014, at 3:52 AM, Nikola Đipanov ndipa...@redhat.com wrote: On 11/13/2014 02:45 AM, Dan Smith wrote: I’m not sure if I’m seeing the second SELECT here either but I’m less familiar with what I’m looking at. compute_node_update() does the one

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-14 Thread Dan Smith
I don't think it's a problem. It puts a practical limit on the scope of an 'api call' which can be covered by a single database transaction though, because it would be difficult to arrange for 2 RPC calls to both use the same DB transaction on the remote end. I think we agree on this. Sure,

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-14 Thread Matthew Booth
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14/11/14 14:40, Dan Smith wrote: I don't think it's a problem. It puts a practical limit on the scope of an 'api call' which can be covered by a single database transaction though, because it would be difficult to arrange for 2 RPC calls to

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Nikola Đipanov
On 11/13/2014 02:45 AM, Dan Smith wrote: I’m not sure if I’m seeing the second SELECT here either but I’m less familiar with what I’m looking at. compute_node_update() does the one SELECT as we said, then it doesn’t look like self._from_db_object() would emit any further SQL specific to

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Matthew Booth
On 13/11/14 08:52, Nikola Đipanov wrote: On 11/13/2014 02:45 AM, Dan Smith wrote: I’m not sure if I’m seeing the second SELECT here either but I’m less familiar with what I’m looking at. compute_node_update() does the one SELECT as we said, then it doesn’t look like self._from_db_object()

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Matthew Booth
On 12/11/14 23:23, Mike Bayer wrote: On Nov 12, 2014, at 10:56 AM, Matthew Booth mbo...@redhat.com wrote: For brevity, I have conflated what happens in object.save() with what happens in db.api. Where the code lives isn't relevant here: I'm only looking at what happens. Specifically, the

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Matthew Booth
On 12/11/14 19:39, Mike Bayer wrote: On Nov 12, 2014, at 12:45 PM, Dan Smith d...@danplanet.com wrote: I personally favour having consistent behaviour across the board. How about updating them all to auto-refresh by default for consistency, but adding an additional option to save() to

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Matthew Booth
On 12/11/14 19:39, Mike Bayer wrote: lets keep in mind my everyone-likes-it-so-far proposal for reader() and writer(): https://review.openstack.org/#/c/125181/ (this is where it’s going to go as nobody has -1’ed it, so in absence of any “no way!” votes I have to assume this is what we’re

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Dan Smith
On 12/11/14 19:39, Mike Bayer wrote: lets keep in mind my everyone-likes-it-so-far proposal for reader() and writer(): https://review.openstack.org/#/c/125181/ (this is where it’s going to go as nobody has -1’ed it, so in absence of any “no way!” votes I have to assume this is what we’re

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Matthew Booth
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/11/14 14:26, Dan Smith wrote: On 12/11/14 19:39, Mike Bayer wrote: lets keep in mind my everyone-likes-it-so-far proposal for reader() and writer(): https://review.openstack.org/#/c/125181/ (this is where it’s going to go as nobody has -1’ed

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Dan Smith
Can we guarantee that the lifetime of a context object in conductor is a single rpc call, and that the object cannot be referenced from any other thread? Yes, without a doubt. --Dan signature.asc Description: OpenPGP digital signature ___

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Mike Bayer
On Nov 13, 2014, at 3:52 AM, Nikola Đipanov ndipa...@redhat.com wrote: On 11/13/2014 02:45 AM, Dan Smith wrote: I’m not sure if I’m seeing the second SELECT here either but I’m less familiar with what I’m looking at. compute_node_update() does the one SELECT as we said, then it doesn’t

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Mike Bayer
On Nov 13, 2014, at 5:47 AM, Matthew Booth mbo...@redhat.com wrote: Unfortunately this model doesn't apply to Nova objects, which are persisted remotely. Unless I've missed something, SQLA doesn't run on Nova Compute at all. Instead, when Nova Compute calls object.save() this results in an

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-13 Thread Dan Smith
Unfortunately this model doesn't apply to Nova objects, which are persisted remotely. Unless I've missed something, SQLA doesn't run on Nova Compute at all. Instead, when Nova Compute calls object.save() this results in an RPC call to Nova Conductor, which persists the object in the DB using

[openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-12 Thread Matthew Booth
I'm currently investigating the feasibility of a generic compare-and-swap feature for NovaObject.save(). This post isn't about that, but that's the larger context. As a preliminary step towards that goal, I've started by investigating how Nova objects are saved today. Ideally there would be some

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-12 Thread Dan Smith
An initial inconsistency I have noticed is that some objects refresh themselves from the database when calling save(), but others don't. I agree that it would be ideal for all objects to behave the same in this regard. I expect that in practice, it's not necessary for all objects to do this,

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-12 Thread Matthew Booth
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/11/14 16:22, Dan Smith wrote: An initial inconsistency I have noticed is that some objects refresh themselves from the database when calling save(), but others don't. I agree that it would be ideal for all objects to behave the same in

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-12 Thread Dan Smith
I personally favour having consistent behaviour across the board. How about updating them all to auto-refresh by default for consistency, but adding an additional option to save() to disable it for particular calls? I think these should be two patches: one to make them all auto-refresh, and

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-12 Thread Mike Bayer
On Nov 12, 2014, at 12:45 PM, Dan Smith d...@danplanet.com wrote: I personally favour having consistent behaviour across the board. How about updating them all to auto-refresh by default for consistency, but adding an additional option to save() to disable it for particular calls? I

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-12 Thread Mike Bayer
On Nov 12, 2014, at 10:56 AM, Matthew Booth mbo...@redhat.com wrote: For brevity, I have conflated what happens in object.save() with what happens in db.api. Where the code lives isn't relevant here: I'm only looking at what happens. Specifically, the following objects refresh themselves

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-12 Thread Mike Bayer
On Nov 12, 2014, at 6:23 PM, Mike Bayer mba...@redhat.com wrote: If I may inquire as to the irrelevant complexity, I’m trying to pinpoint where you see this happening. When we talk about updating a ComputeNode, because I’m only slightly familiar with Nova’s codebase, I assume we are

Re: [openstack-dev] [nova] Consistency, efficiency, and safety of NovaObject.save()

2014-11-12 Thread Dan Smith
I’m not sure if I’m seeing the second SELECT here either but I’m less familiar with what I’m looking at. compute_node_update() does the one SELECT as we said, then it doesn’t look like self._from_db_object() would emit any further SQL specific to that row. I don't think you're missing