Re: [openstack-dev] [nova] Distributed locking

2014-06-25 Thread John Garbutt
So just to keep the ML up with some of the discussion we had in IRC the other day... Most resources in Nova are owned by a particular nova-compute. So the locks on the resources are effectively held by the nova-compute that owns the resource. We already effectively have a cross nova-compute lock

Re: [openstack-dev] [nova] Distributed locking

2014-06-25 Thread Joshua Harlow
: [openstack-dev] [nova] Distributed locking So just to keep the ML up with some of the discussion we had in IRC the other day... Most resources in Nova are owned by a particular nova-compute. So the locks on the resources are effectively held by the nova-compute that owns the resource. We already

Re: [openstack-dev] [nova] Distributed locking

2014-06-17 Thread Matthew Booth
Organization: Red Hat Date: Friday, June 13, 2014 at 1:40 AM To: Joshua Harlow harlo...@yahoo-inc.com, OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [nova] Distributed locking On 12/06/14 21:38, Joshua Harlow wrote: So

Re: [openstack-dev] [nova] Distributed locking

2014-06-17 Thread Doug Hellmann
-inc.com, OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [nova] Distributed locking On 12/06/14 21:38, Joshua Harlow wrote: So just a few thoughts before going to far down this path, Can we make sure we really really

Re: [openstack-dev] [nova] Distributed locking

2014-06-17 Thread Clint Byrum
Excerpts from Matthew Booth's message of 2014-06-17 01:36:11 -0700: On 17/06/14 00:28, Joshua Harlow wrote: So this is a reader/write lock then? I have seen https://github.com/python-zk/kazoo/pull/141 come up in the kazoo (zookeeper python library) but there was a lack of a maintainer

Re: [openstack-dev] [nova] Distributed locking

2014-06-16 Thread Jay Pipes
On 06/13/2014 05:01 AM, Julien Danjou wrote: On Thu, Jun 12 2014, Jay Pipes wrote: This is news to me. When was this decided and where can I read about it? Originally https://wiki.openstack.org/wiki/Oslo/blueprints/service-sync has been proposed, presented and accepted back at the Icehouse

Re: [openstack-dev] [nova] Distributed locking

2014-06-16 Thread Joshua Harlow
...@redhat.com Organization: Red Hat Date: Friday, June 13, 2014 at 1:40 AM To: Joshua Harlow harlo...@yahoo-inc.com, OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [nova] Distributed locking On 12/06/14 21:38, Joshua Harlow

Re: [openstack-dev] [nova] Distributed locking

2014-06-15 Thread Clint Byrum
Excerpts from Matthew Booth's message of 2014-06-13 01:40:30 -0700: On 12/06/14 21:38, Joshua Harlow wrote: So just a few thoughts before going to far down this path, Can we make sure we really really understand the use-case where we think this is needed. I think it's fine that this

Re: [openstack-dev] [nova] Distributed locking

2014-06-15 Thread Angus Lees
On Fri, 13 Jun 2014 09:40:30 AM Matthew Booth wrote: On 12/06/14 21:38, Joshua Harlow wrote: So just a few thoughts before going to far down this path, Can we make sure we really really understand the use-case where we think this is needed. I think it's fine that this use-case exists,

Re: [openstack-dev] [nova] Distributed locking

2014-06-14 Thread Robert Collins
On 13 June 2014 02:30, Matthew Booth mbo...@redhat.com wrote: We have a need for a distributed lock in the VMware driver, which I suspect isn't unique. Specifically it is possible for a VMware datastore to be accessed via multiple nova nodes if it is shared between clusters[1]. Unfortunately

Re: [openstack-dev] [nova] Distributed locking

2014-06-14 Thread Joshua Harlow
Are the details of that implementation described on wiki or elsewhere? (Partially for my own curiosity). I think I understand how it works but write ups usually clear that right up. Sent from my really tiny device... On Jun 14, 2014, at 12:15 AM, Robert Collins robe...@robertcollins.net

Re: [openstack-dev] [nova] Distributed locking

2014-06-13 Thread Matthew Booth
: Thursday, June 12, 2014 at 7:30 AM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Subject: [openstack-dev] [nova] Distributed locking We have a need for a distributed lock in the VMware driver, which I suspect isn't unique. Specifically

[openstack-dev] [nova] Distributed locking

2014-06-12 Thread Matthew Booth
We have a need for a distributed lock in the VMware driver, which I suspect isn't unique. Specifically it is possible for a VMware datastore to be accessed via multiple nova nodes if it is shared between clusters[1]. Unfortunately the vSphere API doesn't provide us with the primitives to implement

Re: [openstack-dev] [nova] Distributed locking

2014-06-12 Thread Julien Danjou
On Thu, Jun 12 2014, Matthew Booth wrote: We have a need for a distributed lock in the VMware driver, which I suspect isn't unique. Specifically it is possible for a VMware datastore to be accessed via multiple nova nodes if it is shared between clusters[1]. Unfortunately the vSphere API

Re: [openstack-dev] [nova] Distributed locking

2014-06-12 Thread Matthew Booth
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/06/14 15:35, Julien Danjou wrote: On Thu, Jun 12 2014, Matthew Booth wrote: We have a need for a distributed lock in the VMware driver, which I suspect isn't unique. Specifically it is possible for a VMware datastore to be accessed via

Re: [openstack-dev] [nova] Distributed locking

2014-06-12 Thread Julien Danjou
On Thu, Jun 12 2014, Matthew Booth wrote: This looks interesting. It doesn't have hooks for fencing, though. What's the status of tooz? Would you be interested in adding fencing hooks? It's maintained and developer, we have plan to use it in Ceilometer and others projects. Joshua also wants

Re: [openstack-dev] [nova] Distributed locking

2014-06-12 Thread Devananda van der Veen
Ironic has a simple lock mechanism for nodes to ensure that, if the hash ring rebalances while an operation is in progress, the second conductor doesn't trample on the work that the first conductor is doing until it's finished (and releases the lock). Right now, it's got a simple DB backing.

Re: [openstack-dev] [nova] Distributed locking

2014-06-12 Thread Angus Lees
On Thu, 12 Jun 2014 05:06:38 PM Julien Danjou wrote: On Thu, Jun 12 2014, Matthew Booth wrote: This looks interesting. It doesn't have hooks for fencing, though. What's the status of tooz? Would you be interested in adding fencing hooks? It's maintained and developer, we have plan to