Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

On Mon, Jun 29, 2015 at 03:45:56PM +0300, Duncan Thomas wrote: > On 29 June 2015 at 15:23, Dulko, Michal wrote: > > > There’s also some similar situations when we actually don’t lock on > > resources. For example – a cgsnapshot may get deleted while creating a > > consistencygroup from it. > >

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

Clint Byrum wrote: Excerpts from Duncan Thomas's message of 2015-06-29 07:54:27 -0700: Do we know what is so hated about the glance task API? Tasks and entity queues give the required exclusion, if you accept that tasks can fail if previous tasks in the queue can cause things to be pulled out fr

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

Excerpts from Duncan Thomas's message of 2015-06-29 07:54:27 -0700: > Do we know what is so hated about the glance task API? Tasks and entity > queues give the required exclusion, if you accept that tasks can fail if > previous tasks in the queue can cause things to be pulled out from under it. >

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

:46 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot On 29 June 2015 at 15:23, Dulko, Michal mailto:michal.du...@intel.com> <mailto:michal.du...@intel.com <mailto:michal.du...@intel.c

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

ilto:duncan.tho...@gmail.com>] *Sent:* Monday, June 29, 2015 2:46 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot On 29 June 2015 at 15:

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

> procedure scheduled? >> >> *From:*Duncan Thomas [mailto:duncan.tho...@gmail.com] >> *Sent:* Monday, June 29, 2015 2:46 PM >> *To:* OpenStack Development Mailing List (not for usage questions) >> *Subject:* Re: [openstack-dev] [cinder][oslo] Locks for create from >

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

s [mailto:duncan.tho...@gmail.com] *Sent:* Monday, June 29, 2015 2:46 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot On 29 June 2015 at 15:23, Dulko, Michal mailto:michal.du...@intel.com>>

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

: Monday, June 29, 2015 2:46 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot On 29 June 2015 at 15:23, Dulko, Michal mailto:michal.du...@intel.com>> wrote: There’s also some similar situation

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

On Sun, Jun 28, 2015 at 1:16 PM, Duncan Thomas wrote: > We need mutual exclusion for several operations. Whether that is done by > entity queues, locks, state based locking at the api later, or something > else, we need mutual exclusion. > > Our current api does not lend itself to looser consiste

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

On 29 June 2015 at 15:23, Dulko, Michal wrote: > There’s also some similar situations when we actually don’t lock on > resources. For example – a cgsnapshot may get deleted while creating a > consistencygroup from it. > > > > From my perspective it seems best to have atomic state changes and >

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

Development Mailing List Subject: Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot We need mutual exclusion for several operations. Whether that is done by entity queues, locks, state based locking at the api later, or something else, we need mutual exclusion. Our current

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

We need mutual exclusion for several operations. Whether that is done by entity queues, locks, state based locking at the api later, or something else, we need mutual exclusion. Our current api does not lend itself to looser consistency, and I struggle to come up with a sane api that does - nobody

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

Do we really need any of these locks? I'm sure we could come up with some way to remove them, rather than make them distributed. On Sun, Jun 28, 2015 at 5:07 AM, Joshua Harlow wrote: > John Griffith wrote: > >> >> >> On Sat, Jun 27, 2015 at 11:47 AM, Joshua Harlow >

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

John Griffith wrote: On Sat, Jun 27, 2015 at 11:47 AM, Joshua Harlow mailto:harlo...@outlook.com>> wrote: Duncan Thomas wrote: We are working on some sort of distributed replacement for the locks in cinder, since file locks are limiting our ability to do HA. I'm

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

On Sat, Jun 27, 2015 at 11:47 AM, Joshua Harlow wrote: > Duncan Thomas wrote: > >> We are working on some sort of distributed replacement for the locks in >> cinder, since file locks are limiting our ability to do HA. I'm afraid >> you're unlikely to get any traction until that work is done. >> >

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

Duncan Thomas wrote: We are working on some sort of distributed replacement for the locks in cinder, since file locks are limiting our ability to do HA. I'm afraid you're unlikely to get any traction until that work is done. I also have a concern that some backend do not handle load well, and so

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

Something that I would prefer is to get the following merged: https://github.com/python-zk/kazoo/pull/306 Then we can have tooz (http://docs.openstack.org/developer/tooz/) have a implementation of that (using the above PR #306 for the kazoo/zookeeper impl) and providing impls of it for the var

Re: [openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

We are working on some sort of distributed replacement for the locks in cinder, since file locks are limiting our ability to do HA. I'm afraid you're unlikely to get any traction until that work is done. I also have a concern that some backend do not handle load well, and so benefit from the curre

[openstack-dev] [cinder][oslo] Locks for create from volume/snapshot

Hi folks, Currently we use a lockfile to protect the create operations from concurrent delete the source volume/snapshot, we use exclusive locks on both delete and create sides which will ensure that: 1. If a create of VolA from snap/VolB is in progress, any delete requests for snap/VolB