Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-30 Thread Davanum Srinivas
Julien, Right. the idea is to expand where we use etcd in openstack for other things as well. example the os-lively effort from Jay, another one is the oslo.config calling etcd3 fetch config entries. Thanks, Dims On Thu, Mar 30, 2017 at 1:10 PM, Julien Danjou wrote: > On Thu, Mar 30 2017, Davan

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-30 Thread Julien Danjou
On Thu, Mar 30 2017, Davanum Srinivas wrote: > Julien, > > Yes, Jay is adding this driver for "etcd3://" using the > https://pypi.python.org/pypi/etcd3 library > https://review.openstack.org/#/c/447223/ > > and i'll followup with a "etcd3+http://"; driver using the > https://pypi.python.org/pypi/e

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-30 Thread Davanum Srinivas
Julien, Yes, Jay is adding this driver for "etcd3://" using the https://pypi.python.org/pypi/etcd3 library https://review.openstack.org/#/c/447223/ and i'll followup with a "etcd3+http://"; driver using the https://pypi.python.org/pypi/etcd3gw library Thanks, Dims On Thu, Mar 30, 2017 at 12:51

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-30 Thread Julien Danjou
On Thu, Mar 30 2017, Davanum Srinivas wrote: > Update.. So i just proposed a new library built off the [3] gist to be > added to g-r and u-c. > > https://review.openstack.org/#/c/450967/ > > The alternative was to add support for custom backends in etcd3 > library itself with grpc and v3alpha HTTP

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-30 Thread Davanum Srinivas
Update.. So i just proposed a new library built off the [3] gist to be added to g-r and u-c. https://review.openstack.org/#/c/450967/ The alternative was to add support for custom backends in etcd3 library itself with grpc and v3alpha HTTP API as implementations. Since tooz is already an abstract

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-19 Thread Jay Pipes
Also, I gave a stab at an etcd3 tooz coordination driver: https://review.openstack.org/#/c/447223/ Can't figure out how to get the tests to run properly, but meh, I'll get to it in a bit :) -jay On 03/19/2017 04:34 PM, Davanum Srinivas wrote: Quick update: We have 3 options to talk to etcd:

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-19 Thread Davanum Srinivas
Quick update: We have 3 options to talk to etcd: 1) existing V2 HTTP API - still not deprecated in etcd 3.1.x, so existing code in tooz still works. 2) grpc based V3 API - We can use the etcd3 package, discussion in review[1], problem is that this will not work currently with eventlet based servic

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-19 Thread Jay Pipes
On 03/18/2017 07:48 PM, Mike Perez wrote: On 12:35 Mar 14, Jay Pipes wrote: On 03/14/2017 08:57 AM, Julien Danjou wrote: On Tue, Mar 14 2017, Davanum Srinivas wrote: Let's do it!! (etcd v2-v3 in tooz) Hehe. I'll move that higher in my priority list, I swear. But anyone is free to beat me to

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-18 Thread Mike Perez
On 12:35 Mar 14, Jay Pipes wrote: > On 03/14/2017 08:57 AM, Julien Danjou wrote: > >On Tue, Mar 14 2017, Davanum Srinivas wrote: > > > >>Let's do it!! (etcd v2-v3 in tooz) > > > >Hehe. I'll move that higher in my priority list, I swear. But anyone is > >free to beat me to it in the meantime. ;) >

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-18 Thread Mike Perez
On 16:37 Mar 18, Mike Perez wrote: > On 16:02 Mar 15, Monty Taylor wrote: > > On 03/15/2017 02:52 PM, Louis Taylor wrote: > > > On Wed, Mar 15, 2017 at 1:44 PM, Julien Danjou wrote: > > >> On Wed, Mar 15 2017, Davanum Srinivas wrote: > > >> > > >>> Yep, jd__ and i confirmed that things work with 3

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-18 Thread Mike Perez
On 16:02 Mar 15, Monty Taylor wrote: > On 03/15/2017 02:52 PM, Louis Taylor wrote: > > On Wed, Mar 15, 2017 at 1:44 PM, Julien Danjou wrote: > >> On Wed, Mar 15 2017, Davanum Srinivas wrote: > >> > >>> Yep, jd__ and i confirmed that things work with 3.x > >> > >> Though to be clear, what's used in

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Clint Byrum
Excerpts from Clark Boylan's message of 2017-03-15 09:50:49 -0700: > On Wed, Mar 15, 2017, at 06:19 AM, Monty Taylor wrote: > > On 03/15/2017 11:37 AM, Davanum Srinivas wrote: > > > Monty, Team, > > > > > > Sorry for the top post: > > > > > > Support for etcd/tooz in devstack (with file driver as

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Clark Boylan
On Wed, Mar 15, 2017, at 06:19 AM, Monty Taylor wrote: > On 03/15/2017 11:37 AM, Davanum Srinivas wrote: > > Monty, Team, > > > > Sorry for the top post: > > > > Support for etcd/tooz in devstack (with file driver as default) - > > https://review.openstack.org/#/c/445432/ > > > > As of right now

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Julien Danjou
On Wed, Mar 15 2017, Monty Taylor wrote: > On 03/15/2017 02:44 PM, Julien Danjou wrote: >> On Wed, Mar 15 2017, Davanum Srinivas wrote: >> >>> Yep, jd__ and i confirmed that things work with 3.x >> >> Though to be clear, what's used in tooz is the v2 HTTP API, not the new >> v3 gRPC API. > > But

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Monty Taylor
On 03/15/2017 02:52 PM, Louis Taylor wrote: > On Wed, Mar 15, 2017 at 1:44 PM, Julien Danjou wrote: >> On Wed, Mar 15 2017, Davanum Srinivas wrote: >> >>> Yep, jd__ and i confirmed that things work with 3.x >> >> Though to be clear, what's used in tooz is the v2 HTTP API, not the new >> v3 gRPC AP

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Monty Taylor
On 03/15/2017 02:44 PM, Julien Danjou wrote: > On Wed, Mar 15 2017, Davanum Srinivas wrote: > >> Yep, jd__ and i confirmed that things work with 3.x > > Though to be clear, what's used in tooz is the v2 HTTP API, not the new > v3 gRPC API. But if it conceptually works with v3 server using v2 htt

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Monty Taylor
On 03/15/2017 02:26 PM, Davanum Srinivas wrote: > On Wed, Mar 15, 2017 at 9:19 AM, Monty Taylor wrote: >> On 03/15/2017 11:37 AM, Davanum Srinivas wrote: >>> Monty, Team, >>> >>> Sorry for the top post: >>> >>> Support for etcd/tooz in devstack (with file driver as default) - >>> https://review.op

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Louis Taylor
On Wed, Mar 15, 2017 at 1:44 PM, Julien Danjou wrote: > On Wed, Mar 15 2017, Davanum Srinivas wrote: > >> Yep, jd__ and i confirmed that things work with 3.x > > Though to be clear, what's used in tooz is the v2 HTTP API, not the new > v3 gRPC API. And just to be double clear: although etcd 3.x c

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Julien Danjou
On Wed, Mar 15 2017, Davanum Srinivas wrote: > Yep, jd__ and i confirmed that things work with 3.x Though to be clear, what's used in tooz is the v2 HTTP API, not the new v3 gRPC API. -- Julien Danjou # Free Software hacker # https://julien.danjou.info signature.asc Description: PGP signature

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Davanum Srinivas
On Wed, Mar 15, 2017 at 9:19 AM, Monty Taylor wrote: > On 03/15/2017 11:37 AM, Davanum Srinivas wrote: >> Monty, Team, >> >> Sorry for the top post: >> >> Support for etcd/tooz in devstack (with file driver as default) - >> https://review.openstack.org/#/c/445432/ >> >> As of right now both zookee

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Monty Taylor
On 03/15/2017 01:48 PM, John Garbutt wrote: > On 15 March 2017 at 12:33, Sean Dague wrote: >> On 03/15/2017 08:10 AM, John Garbutt wrote: >>> On 15 March 2017 at 11:58, Jay Pipes wrote: On 03/15/2017 07:44 AM, Sean Dague wrote: > > On 03/14/2017 11:00 PM, Monty Taylor wrote: > >

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Monty Taylor
On 03/15/2017 11:37 AM, Davanum Srinivas wrote: > Monty, Team, > > Sorry for the top post: > > Support for etcd/tooz in devstack (with file driver as default) - > https://review.openstack.org/#/c/445432/ > > As of right now both zookeeper driver and etcd driver is working fine: > https://review.

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Nicolas Trangez
On Wed, 2017-03-15 at 08:33 -0400, Sean Dague wrote: > On 03/15/2017 08:10 AM, John Garbutt wrote: > > On 15 March 2017 at 11:58, Jay Pipes wrote: > > > On 03/15/2017 07:44 AM, Sean Dague wrote: > > > > > > > > On 03/14/2017 11:00 PM, Monty Taylor wrote: > > > > > > > > > > > > > > a) awesome.

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread John Garbutt
On 15 March 2017 at 12:33, Sean Dague wrote: > On 03/15/2017 08:10 AM, John Garbutt wrote: >> On 15 March 2017 at 11:58, Jay Pipes wrote: >>> On 03/15/2017 07:44 AM, Sean Dague wrote: On 03/14/2017 11:00 PM, Monty Taylor wrote: > > a) awesome. when the rest of this dips mo

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Sean Dague
On 03/15/2017 08:10 AM, John Garbutt wrote: > On 15 March 2017 at 11:58, Jay Pipes wrote: >> On 03/15/2017 07:44 AM, Sean Dague wrote: >>> >>> On 03/14/2017 11:00 PM, Monty Taylor wrote: >>> a) awesome. when the rest of this dips momentarily into words that might sound negative, pl

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread John Garbutt
On 15 March 2017 at 11:58, Jay Pipes wrote: > On 03/15/2017 07:44 AM, Sean Dague wrote: >> >> On 03/14/2017 11:00 PM, Monty Taylor wrote: >> >>> >>> a) awesome. when the rest of this dips momentarily into words that might >>> sound negative, please hear it all wrapped in an "awesome" and know tha

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Jay Pipes
On 03/15/2017 07:44 AM, Sean Dague wrote: On 03/14/2017 11:00 PM, Monty Taylor wrote: a) awesome. when the rest of this dips momentarily into words that might sound negative, please hear it all wrapped in an "awesome" and know that my personal desire is to see the thing you're working on be suc

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Davanum Srinivas
+1000 Sean. Thanks, Dims On Wed, Mar 15, 2017 at 7:44 AM, Sean Dague wrote: > On 03/14/2017 11:00 PM, Monty Taylor wrote: > >> a) awesome. when the rest of this dips momentarily into words that might >> sound negative, please hear it all wrapped in an "awesome" and know that >> my personal desi

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Sean Dague
On 03/14/2017 11:00 PM, Monty Taylor wrote: > a) awesome. when the rest of this dips momentarily into words that might > sound negative, please hear it all wrapped in an "awesome" and know that > my personal desire is to see the thing you're working on be successful > without undue burden... > >

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-15 Thread Davanum Srinivas
Monty, Team, Sorry for the top post: Support for etcd/tooz in devstack (with file driver as default) - https://review.openstack.org/#/c/445432/ As of right now both zookeeper driver and etcd driver is working fine: https://review.openstack.org/#/c/445630/ https://review.openstack.org/#/c/445629/

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Clint Byrum
Excerpts from Jay Pipes's message of 2017-03-14 22:13:32 -0400: > On 03/14/2017 05:01 PM, Clint Byrum wrote: > > Excerpts from Jay Pipes's message of 2017-03-14 15:30:32 -0400: > >> On 03/14/2017 02:50 PM, Julien Danjou wrote: > >>> On Tue, Mar 14 2017, Jay Pipes wrote: > >>> > Not tooz, becau

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Monty Taylor
On 03/15/2017 03:13 AM, Jay Pipes wrote: > On 03/14/2017 05:01 PM, Clint Byrum wrote: >> Excerpts from Jay Pipes's message of 2017-03-14 15:30:32 -0400: >>> On 03/14/2017 02:50 PM, Julien Danjou wrote: On Tue, Mar 14 2017, Jay Pipes wrote: > Not tooz, because I'm not interested in a D

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Jay Pipes
On 03/14/2017 05:01 PM, Clint Byrum wrote: Excerpts from Jay Pipes's message of 2017-03-14 15:30:32 -0400: On 03/14/2017 02:50 PM, Julien Danjou wrote: On Tue, Mar 14 2017, Jay Pipes wrote: Not tooz, because I'm not interested in a DLM nor leader election library (that's what the underlying e

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Julien Danjou
On Tue, Mar 14 2017, Clint Byrum wrote: > I understand abstractions can seem pretty cumbersome when you're moving > fast. Considering the problem is at least 5 years old in Nova and tooz itself is at least 3 years old, let's say that "moving fast" has a… funny taste. > It's not something I want

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Clint Byrum
Excerpts from Jay Pipes's message of 2017-03-14 15:30:32 -0400: > On 03/14/2017 02:50 PM, Julien Danjou wrote: > > On Tue, Mar 14 2017, Jay Pipes wrote: > > > >> Not tooz, because I'm not interested in a DLM nor leader election library > >> (that's what the underlying etcd3 cluster handles for me),

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Jay Pipes
On 03/14/2017 04:22 PM, Joshua Harlow wrote: Jay Pipes wrote: On 03/14/2017 02:50 PM, Julien Danjou wrote: On Tue, Mar 14 2017, Jay Pipes wrote: Not tooz, because I'm not interested in a DLM nor leader election library (that's what the underlying etcd3 cluster handles for me), only a fast ser

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Joshua Harlow
Jay Pipes wrote: On 03/14/2017 02:50 PM, Julien Danjou wrote: On Tue, Mar 14 2017, Jay Pipes wrote: Not tooz, because I'm not interested in a DLM nor leader election library (that's what the underlying etcd3 cluster handles for me), only a fast service liveness/healthcheck system, but it shows

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Jay Pipes
On 03/14/2017 02:50 PM, Julien Danjou wrote: On Tue, Mar 14 2017, Jay Pipes wrote: Not tooz, because I'm not interested in a DLM nor leader election library (that's what the underlying etcd3 cluster handles for me), only a fast service liveness/healthcheck system, but it shows usage of etcd3 an

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Julien Danjou
On Tue, Mar 14 2017, Jay Pipes wrote: > Not tooz, because I'm not interested in a DLM nor leader election library > (that's what the underlying etcd3 cluster handles for me), only a fast service > liveness/healthcheck system, but it shows usage of etcd3 and Google Protocol > Buffers implementing a

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Fox, Kevin M
+1 From: Thierry Carrez [thie...@openstack.org] Sent: Tuesday, March 14, 2017 3:00 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM Fox, Kevin M wrote: > With my operator hat on

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Jay Pipes
On 03/14/2017 08:57 AM, Julien Danjou wrote: On Tue, Mar 14 2017, Davanum Srinivas wrote: Let's do it!! (etcd v2-v3 in tooz) Hehe. I'll move that higher in my priority list, I swear. But anyone is free to beat me to it in the meantime. ;) A weekend experiment: https://github.com/jaypipes/o

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Tomasz Pa
Etcd seems to be a better choice for performance reasons as well: https://coreos.com/blog/performance-of-etcd.html TP On 14 Mar 2017 12:45 am, "Davanum Srinivas" wrote: > On Tue, Mar 14 2017, Davanum Srinivas wrote: > > > Let's do it!! (etcd v2-v3 in tooz) > > Hehe. I'll move that higher in my

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Julien Danjou
On Tue, Mar 14 2017, Davanum Srinivas wrote: > Let's do it!! (etcd v2-v3 in tooz) Hehe. I'll move that higher in my priority list, I swear. But anyone is free to beat me to it in the meantime. ;) -- Julien Danjou -- Free Software hacker -- https://julien.danjou.info signature.asc Description:

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Davanum Srinivas
Julien, Let's do it!! (etcd v2-v3 in tooz) Thanks, Dims On Tue, Mar 14, 2017 at 5:24 AM, Julien Danjou wrote: > On Mon, Mar 13 2017, Joshua Harlow wrote: > >> Etcd I think is also switching to grpc sometime in the future (afaik); that >> feature is in alpha/?beta?/experimental right now. > > Ye

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Thierry Carrez
Fox, Kevin M wrote: > With my operator hat on, I would like to use the etcd backend, as I'm already > paying the cost of maintaining etcd clusters as part of Kubernetes. Adding > Zookeeper is a lot more work. +1 In the spirit of better operationally integrating with Kubernetes, I think we need

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-14 Thread Julien Danjou
On Mon, Mar 13 2017, Joshua Harlow wrote: > Etcd I think is also switching to grpc sometime in the future (afaik); that > feature is in alpha/?beta?/experimental right now. Yeah I think that's the main "problem" in tooz right now, it's that we still rely on etcd v2 and I think v3 is out with that

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-13 Thread Joshua Harlow
Clint Byrum wrote: Excerpts from Fox, Kevin M's message of 2017-03-14 00:09:55 +: With my operator hat on, I would like to use the etcd backend, as I'm already paying the cost of maintaining etcd clusters as part of Kubernetes. Adding Zookeeper is a lot more work. It would probably be

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-13 Thread Clint Byrum
Excerpts from Fox, Kevin M's message of 2017-03-14 00:09:55 +: > With my operator hat on, I would like to use the etcd backend, as I'm already > paying the cost of maintaining etcd clusters as part of Kubernetes. Adding > Zookeeper is a lot more work. > It would probably be a good idea to p

Re: [openstack-dev] [oslo][devstack][all] ZooKeeper vs etcd for Tooz/DLM

2017-03-13 Thread Fox, Kevin M
With my operator hat on, I would like to use the etcd backend, as I'm already paying the cost of maintaining etcd clusters as part of Kubernetes. Adding Zookeeper is a lot more work. Thanks, Kevin From: Davanum Srinivas [dava...@gmail.com] Sent: Monday, M