Re: [openstack-dev] [all] etcd3 as base service - update

[Mike Bayer]

On 06/09/2017 11:12 AM, Lance Bragstad wrote:



I should have clarified. The idea was to put the keys used to encrypt 
and decrypt the tokens in etcd so that synchronizing the repository 
across a cluster for keystone nodes is easier for operators (but not 
without other operator pain as Kevin pointed out). The tokens themselves 
will remain completely non-persistent. Fernet key creation is explicitly 
controlled by operators and isn't something that end users generate.


makes sense and I agree is entirely appropriate thanks!





[0] 
https://github.com/openstack/keystone/blob/c528539879e824b8e6d5654292a85ccbee6dcf89/keystone/conf/fernet_tokens.py#L44-L54

[1] https://launchpad.net/bugs/1649616






On Thu, Jun 8, 2017 at 11:37 AM, Mike Bayer  >> wrote:



 On 06/08/2017 12:47 AM, Joshua Harlow wrote:

 So just out of curiosity, but do people really even
know what
 etcd is good for? I am thinking that there should be some
 guidance from folks in the community as to where etcd
should be
 used and where it shouldn't (otherwise we just all end
up in a
 mess).


 So far I've seen a proposal of etcd3 as a replacement for
memcached
 in keystone, and a new dogpile connector was added to
oslo.cache to
 handle referring to etcd3 as a cache backend.  This is a really
 simplistic / minimal kind of use case for a key-store.

 But, keeping in mind I don't know anything about etcd3
other than
 "it's another key-store", it's the only database used by
Kubernetes
 as a whole, which suggests it's doing a better job than
Redis in
 terms of "durable".   So I wouldn't be surprised if new /
existing
 openstack applications express some gravitational pull
towards using
 it as their own datastore as well.I'll be trying to
hang onto
 the etcd3 track as much as possible so that if/when that
happens I
 still have a job :).





 Perhaps a good idea to actually give examples of how it
should
 be used, how it shouldn't be used, what it offers, what it
 doesn't... Or at least provide links for people to read
up on this.

 Thoughts?

 Davanum Srinivas wrote:

 One clarification: Since
https://pypi.python.org/pypi/etcd3gw

 > just
 uses the HTTP API (/v3alpha) it will work under both
 eventlet and
 non-eventlet environments.

 Thanks,
 Dims


 On Wed, Jun 7, 2017 at 6:47 AM, Davanum
 Srinivas >>  wrote:

 Team,

 Here's the update to the base services
resolution from
 the TC:
https://governance.openstack.org/tc/reference/base-services.html


>


 First request is to Distros, Packagers, Deployers,
 anyone who
 installs/configures OpenStack:
 Please make sure you have latest etcd 3.x
available in your
 environment for Services to use, Fedora already
does, we
 need help in
 making sure all distros and architectures are
covered.

 Any project who want to use etcd v3 API via
grpc, please
 use:
https://pypi.python.org/pypi/etcd3

 > (works only for
 non-eventlet services)

 Those that depend on eventlet, please use the etcd3
 v3alpha HTTP API using:
https://pypi.python.org/pypi/etcd3gw

 >

 If you use 

Re: [openstack-dev] [all] etcd3 as base service - update

[Lance Bragstad]

On Fri, Jun 9, 2017 at 11:17 AM, Clint Byrum  wrote:

> Excerpts from Lance Bragstad's message of 2017-06-08 16:10:00 -0500:
> > On Thu, Jun 8, 2017 at 3:21 PM, Emilien Macchi 
> wrote:
> >
> > > On Thu, Jun 8, 2017 at 7:34 PM, Lance Bragstad 
> > > wrote:
> > > > After digging into etcd a bit, one place this might be help deployer
> > > > experience would be the handling of fernet keys for token encryption
> in
> > > > keystone. Currently, all keys used to encrypt and decrypt tokens are
> > > kept on
> > > > disk for each keystone node in the deployment. While simple, it
> requires
> > > > operators to perform rotation on a single node and then push, or
> sync,
> > > the
> > > > new key set to the rest of the nodes. This must be done in lock step
> in
> > > > order to prevent early token invalidation and inconsistent token
> > > responses.
> > >
> > > This is what we discussed a few months ago :-)
> > >
> > > http://lists.openstack.org/pipermail/openstack-dev/2017-
> March/113943.html
> > >
> > > I'm glad it's coming back ;-)
> > >
> >
> > Yep! I've proposed a pretty basic spec to backlog [0] in an effort to
> > capture the discussion. I've also noted the point Kevin brought up about
> > authorization in etcd (thanks, Kevin!)
> >
> > If someone feels compelled to take that and run with it, they are more
> than
> > welcome to.
> >
> > [0] https://review.openstack.org/#/c/472385/
> >
>
> I commented on the spec. I think this is a misguided idea. etcd3 is a
> _coordination_ service. Not a key manager. It lacks the audit logging
> and access control one expects to protect and manage key material. I'd
> much rather see something like Hashicorp's Vault [1] implemented for
> Fernet keys than etcd3. We even have a library for such things called
> Castellan[2].
>

Great point, and thanks for leaving it in the spec. I'm glad we're getting
this documented since this specific discussion has cropped up a couple
times.


>
> [1] https://www.vaultproject.io/
> [2] https://docs.openstack.org/developer/castellan/
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Clint Byrum]

Excerpts from Lance Bragstad's message of 2017-06-08 16:10:00 -0500:
> On Thu, Jun 8, 2017 at 3:21 PM, Emilien Macchi  wrote:
> 
> > On Thu, Jun 8, 2017 at 7:34 PM, Lance Bragstad 
> > wrote:
> > > After digging into etcd a bit, one place this might be help deployer
> > > experience would be the handling of fernet keys for token encryption in
> > > keystone. Currently, all keys used to encrypt and decrypt tokens are
> > kept on
> > > disk for each keystone node in the deployment. While simple, it requires
> > > operators to perform rotation on a single node and then push, or sync,
> > the
> > > new key set to the rest of the nodes. This must be done in lock step in
> > > order to prevent early token invalidation and inconsistent token
> > responses.
> >
> > This is what we discussed a few months ago :-)
> >
> > http://lists.openstack.org/pipermail/openstack-dev/2017-March/113943.html
> >
> > I'm glad it's coming back ;-)
> >
> 
> Yep! I've proposed a pretty basic spec to backlog [0] in an effort to
> capture the discussion. I've also noted the point Kevin brought up about
> authorization in etcd (thanks, Kevin!)
> 
> If someone feels compelled to take that and run with it, they are more than
> welcome to.
> 
> [0] https://review.openstack.org/#/c/472385/
> 

I commented on the spec. I think this is a misguided idea. etcd3 is a
_coordination_ service. Not a key manager. It lacks the audit logging
and access control one expects to protect and manage key material. I'd
much rather see something like Hashicorp's Vault [1] implemented for
Fernet keys than etcd3. We even have a library for such things called
Castellan[2].

[1] https://www.vaultproject.io/
[2] https://docs.openstack.org/developer/castellan/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Lance Bragstad]

On Fri, Jun 9, 2017 at 9:57 AM, Mike Bayer  wrote:

>
>
> On 06/08/2017 01:34 PM, Lance Bragstad wrote:
>
>> After digging into etcd a bit, one place this might be help deployer
>> experience would be the handling of fernet keys for token encryption in
>> keystone. Currently, all keys used to encrypt and decrypt tokens are kept
>> on disk for each keystone node in the deployment. While simple, it requires
>> operators to perform rotation on a single node and then push, or sync, the
>> new key set to the rest of the nodes. This must be done in lock step in
>> order to prevent early token invalidation and inconsistent token responses.
>>
>> An alternative would be to keep the keys in etcd and make the fernet bits
>> pluggable so that it's possible to read keys from disk or etcd (pending
>> configuration). The advantage would be that operators could initiate key
>> rotations from any keystone node in the deployment (or using etcd directly)
>> and not have to worry about distributing the new key set. Since etcd
>> associates metadata to the key-value pairs, we might be able to simplify
>> the rotation strategy as well.
>>
>
> Interesting, I had the mis-conception that "fernet" keys no longer
> required any server-side storage (how is "kept-on-disk" now implemented?) .


Currently - the keys used to encrypt and decrypt fernet tokens are stored
as files on the keystone server. The repositories default location is in
`/etc/keystone/fernet-keys`. The size of this repository is regulated by
the rotation process we provide in keystone-manage tooling [0].


> We've had continuous issues with the pre-fernet Keystone tokens filling up
> databases, even when operators were correctly expunging old tokens; some
> environments just did so many requests that the keystone-token table still
> blew up to where MySQL can no longer delete from it without producing a
> too-large transaction for Galera.
>

Yep - we actually just fixed a bug related to this [1].


>
> So after all the "finally fernet solves this problem" we propose, hey lets
> put them *back* in the database :).  That's great.  But, lets please not
> leave "cleaning out old tokens" as some kind of cron/worry-about-it-later
> thing.  that was a terrible architectural decision, with apologies to
> whoever made it.if you're putting some kind of "we create an infinite,
> rapidly growing, turns-to-garbage-in-30-seconds" kind of data in a
> database, removing that data robustly and ASAP needs to be part of the
> process.
>
>
I should have clarified. The idea was to put the keys used to encrypt and
decrypt the tokens in etcd so that synchronizing the repository across a
cluster for keystone nodes is easier for operators (but not without other
operator pain as Kevin pointed out). The tokens themselves will remain
completely non-persistent. Fernet key creation is explicitly controlled by
operators and isn't something that end users generate.

[0]
https://github.com/openstack/keystone/blob/c528539879e824b8e6d5654292a85ccbee6dcf89/keystone/conf/fernet_tokens.py#L44-L54
[1] https://launchpad.net/bugs/1649616


>
>
>
>
>
>> On Thu, Jun 8, 2017 at 11:37 AM, Mike Bayer  mba...@redhat.com>> wrote:
>>
>>
>>
>> On 06/08/2017 12:47 AM, Joshua Harlow wrote:
>>
>> So just out of curiosity, but do people really even know what
>> etcd is good for? I am thinking that there should be some
>> guidance from folks in the community as to where etcd should be
>> used and where it shouldn't (otherwise we just all end up in a
>> mess).
>>
>>
>> So far I've seen a proposal of etcd3 as a replacement for memcached
>> in keystone, and a new dogpile connector was added to oslo.cache to
>> handle referring to etcd3 as a cache backend.  This is a really
>> simplistic / minimal kind of use case for a key-store.
>>
>> But, keeping in mind I don't know anything about etcd3 other than
>> "it's another key-store", it's the only database used by Kubernetes
>> as a whole, which suggests it's doing a better job than Redis in
>> terms of "durable".   So I wouldn't be surprised if new / existing
>> openstack applications express some gravitational pull towards using
>> it as their own datastore as well.I'll be trying to hang onto
>> the etcd3 track as much as possible so that if/when that happens I
>> still have a job :).
>>
>>
>>
>>
>>
>> Perhaps a good idea to actually give examples of how it should
>> be used, how it shouldn't be used, what it offers, what it
>> doesn't... Or at least provide links for people to read up on
>> this.
>>
>> Thoughts?
>>
>> Davanum Srinivas wrote:
>>
>> One clarification: Since
>> https://pypi.python.org/pypi/etcd3gw
>>  just
>> uses the HTTP API (/v3alpha) it will work under both
>> eventlet and
>>  

Re: [openstack-dev] [all] etcd3 as base service - update

[gordon chung]

On 09/06/17 10:57 AM, Mike Bayer wrote:
> Interesting, I had the mis-conception that "fernet" keys no longer
> required any server-side storage (how is "kept-on-disk" now
> implemented?) .  We've had continuous issues with the pre-fernet
> Keystone tokens filling up databases, even when operators were correctly
> expunging old tokens; some environments just did so many requests that
> the keystone-token table still blew up to where MySQL can no longer
> delete from it without producing a too-large transaction for Galera.

i feel your pain. had exact same "can't clean token table because it's 
too damn big" issue.

>
> So after all the "finally fernet solves this problem" we propose, hey
> lets put them *back* in the database :).  That's great.  But, lets
> please not leave "cleaning out old tokens" as some kind of
> cron/worry-about-it-later thing.  that was a terrible architectural
> decision, with apologies to whoever made it.if you're putting some
> kind of "we create an infinite, rapidly growing,
> turns-to-garbage-in-30-seconds" kind of data in a database, removing
> that data robustly and ASAP needs to be part of the process.

my very basic understanding is that only the key to generate token is 
stored. so it in theory will expire less often but more importantly, 
isn't affected by the number of requests.

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Mike Bayer]

On 06/08/2017 04:24 PM, Julien Danjou wrote:

On Thu, Jun 08 2017, Mike Bayer wrote:



So I wouldn't be surprised if new / existing openstack applications
express some gravitational pull towards using it as their own
datastore as well. I'll be trying to hang onto the etcd3 track as much
as possible so that if/when that happens I still have a job :).


Sounds like a recipe for disaster. :)


What architectural decision in any of Openstack is *not* considered by 
some subset of folks to be a "recipe for disaster" ?  :)








__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Mike Bayer]

On 06/08/2017 01:34 PM, Lance Bragstad wrote:
After digging into etcd a bit, one place this might be help deployer 
experience would be the handling of fernet keys for token encryption in 
keystone. Currently, all keys used to encrypt and decrypt tokens are 
kept on disk for each keystone node in the deployment. While simple, it 
requires operators to perform rotation on a single node and then push, 
or sync, the new key set to the rest of the nodes. This must be done in 
lock step in order to prevent early token invalidation and inconsistent 
token responses.


An alternative would be to keep the keys in etcd and make the fernet 
bits pluggable so that it's possible to read keys from disk or etcd 
(pending configuration). The advantage would be that operators could 
initiate key rotations from any keystone node in the deployment (or 
using etcd directly) and not have to worry about distributing the new 
key set. Since etcd associates metadata to the key-value pairs, we might 
be able to simplify the rotation strategy as well.


Interesting, I had the mis-conception that "fernet" keys no longer 
required any server-side storage (how is "kept-on-disk" now 
implemented?) .  We've had continuous issues with the pre-fernet 
Keystone tokens filling up databases, even when operators were correctly 
expunging old tokens; some environments just did so many requests that 
the keystone-token table still blew up to where MySQL can no longer 
delete from it without producing a too-large transaction for Galera.


So after all the "finally fernet solves this problem" we propose, hey 
lets put them *back* in the database :).  That's great.  But, lets 
please not leave "cleaning out old tokens" as some kind of 
cron/worry-about-it-later thing.  that was a terrible architectural 
decision, with apologies to whoever made it.if you're putting some 
kind of "we create an infinite, rapidly growing, 
turns-to-garbage-in-30-seconds" kind of data in a database, removing 
that data robustly and ASAP needs to be part of the process.








On Thu, Jun 8, 2017 at 11:37 AM, Mike Bayer > wrote:




On 06/08/2017 12:47 AM, Joshua Harlow wrote:

So just out of curiosity, but do people really even know what
etcd is good for? I am thinking that there should be some
guidance from folks in the community as to where etcd should be
used and where it shouldn't (otherwise we just all end up in a
mess).


So far I've seen a proposal of etcd3 as a replacement for memcached
in keystone, and a new dogpile connector was added to oslo.cache to
handle referring to etcd3 as a cache backend.  This is a really
simplistic / minimal kind of use case for a key-store.

But, keeping in mind I don't know anything about etcd3 other than
"it's another key-store", it's the only database used by Kubernetes
as a whole, which suggests it's doing a better job than Redis in
terms of "durable".   So I wouldn't be surprised if new / existing
openstack applications express some gravitational pull towards using
it as their own datastore as well.I'll be trying to hang onto
the etcd3 track as much as possible so that if/when that happens I
still have a job :).





Perhaps a good idea to actually give examples of how it should
be used, how it shouldn't be used, what it offers, what it
doesn't... Or at least provide links for people to read up on this.

Thoughts?

Davanum Srinivas wrote:

One clarification: Since
https://pypi.python.org/pypi/etcd3gw
 just
uses the HTTP API (/v3alpha) it will work under both
eventlet and
non-eventlet environments.

Thanks,
Dims


On Wed, Jun 7, 2017 at 6:47 AM, Davanum
Srinivas>  wrote:

Team,

Here's the update to the base services resolution from
the TC:
https://governance.openstack.org/tc/reference/base-services.html



First request is to Distros, Packagers, Deployers,
anyone who
installs/configures OpenStack:
Please make sure you have latest etcd 3.x available in your
environment for Services to use, Fedora already does, we
need help in
making sure all distros and architectures are covered.

Any project who want to use etcd v3 API via grpc, please
use:
https://pypi.python.org/pypi/etcd3
 (works only for
non-eventlet services)

Those that depend on eventlet, please use 

Re: [openstack-dev] [all] etcd3 as base service - update

[gordon chung]

On 09/06/17 12:37 AM, Joshua Harlow wrote:
> My thinking is that people should look over https://raft.github.io/ or
> http://thesecretlivesofdata.com/raft/ (or both or others...)
>

this was really useful. thanks for this! love how they described it so 
simply with visuals. spend a few minutes and look this ^

cheers,

-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Joshua Harlow]

Julien Danjou wrote:

On Thu, Jun 08 2017, Mike Bayer wrote:


So far I've seen a proposal of etcd3 as a replacement for memcached in
keystone, and a new dogpile connector was added to oslo.cache to handle
referring to etcd3 as a cache backend.  This is a really simplistic / minimal
kind of use case for a key-store.


etcd3 is not meant to be a cache. Synchronizing caching value using the
Raft protocol sounds a bit overkill. A cluster of memcached would be
probably of a better use.


Agreed from me,

My thinking is that people should look over https://raft.github.io/ or 
http://thesecretlivesofdata.com/raft/ (or both or others...)


At least read how it sort of works, before trying to put it everywhere 
(the same can and should be said for any new service), because its not a 
solution for all the things.


The other big thing to know is how writes happen in this kind of system, 
they all go through a single node (the leader, who sends the same data 
to followers and waits for a certain number of followers to respond 
before committing)


Anyways, with great power comes great responsibility...

IMHO just be careful and understand the technology before using it for 
things it may not really be good for. Oh ya and perhaps someone will 
want to finally take more advantage of 
https://docs.openstack.org/developer/taskflow/jobs.html#overview (which 
uses the same concepts etcd exposes to make highly available workflows 
that can survive node failure).





But, keeping in mind I don't know anything about etcd3 other than "it's another
key-store", it's the only database used by Kubernetes as a whole, which
suggests it's doing a better job than Redis in terms of "durable".


Not sure about that. And Redis has much more data structure than etcd,
that is can be faster/more efficient than etcd. But it does not have
Raft and a synchronisation protocol. Its clustering is rather poor in
comparison of etcd.


So I wouldn't be surprised if new / existing openstack applications
express some gravitational pull towards using it as their own
datastore as well. I'll be trying to hang onto the etcd3 track as much
as possible so that if/when that happens I still have a job :).


Sounds like a recipe for disaster. :)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Lance Bragstad]

On Thu, Jun 8, 2017 at 3:21 PM, Emilien Macchi  wrote:

> On Thu, Jun 8, 2017 at 7:34 PM, Lance Bragstad 
> wrote:
> > After digging into etcd a bit, one place this might be help deployer
> > experience would be the handling of fernet keys for token encryption in
> > keystone. Currently, all keys used to encrypt and decrypt tokens are
> kept on
> > disk for each keystone node in the deployment. While simple, it requires
> > operators to perform rotation on a single node and then push, or sync,
> the
> > new key set to the rest of the nodes. This must be done in lock step in
> > order to prevent early token invalidation and inconsistent token
> responses.
>
> This is what we discussed a few months ago :-)
>
> http://lists.openstack.org/pipermail/openstack-dev/2017-March/113943.html
>
> I'm glad it's coming back ;-)
>

Yep! I've proposed a pretty basic spec to backlog [0] in an effort to
capture the discussion. I've also noted the point Kevin brought up about
authorization in etcd (thanks, Kevin!)

If someone feels compelled to take that and run with it, they are more than
welcome to.

[0] https://review.openstack.org/#/c/472385/


> > An alternative would be to keep the keys in etcd and make the fernet bits
> > pluggable so that it's possible to read keys from disk or etcd (pending
> > configuration). The advantage would be that operators could initiate key
> > rotations from any keystone node in the deployment (or using etcd
> directly)
> > and not have to worry about distributing the new key set. Since etcd
> > associates metadata to the key-value pairs, we might be able to simplify
> the
> > rotation strategy as well.
> >
> > On Thu, Jun 8, 2017 at 11:37 AM, Mike Bayer  wrote:
> >>
> >>
> >>
> >> On 06/08/2017 12:47 AM, Joshua Harlow wrote:
> >>>
> >>> So just out of curiosity, but do people really even know what etcd is
> >>> good for? I am thinking that there should be some guidance from folks
> in the
> >>> community as to where etcd should be used and where it shouldn't
> (otherwise
> >>> we just all end up in a mess).
> >>
> >>
> >> So far I've seen a proposal of etcd3 as a replacement for memcached in
> >> keystone, and a new dogpile connector was added to oslo.cache to handle
> >> referring to etcd3 as a cache backend.  This is a really simplistic /
> >> minimal kind of use case for a key-store.
> >>
> >> But, keeping in mind I don't know anything about etcd3 other than "it's
> >> another key-store", it's the only database used by Kubernetes as a
> whole,
> >> which suggests it's doing a better job than Redis in terms of "durable".
> >> So I wouldn't be surprised if new / existing openstack applications
> express
> >> some gravitational pull towards using it as their own datastore as well.
> >> I'll be trying to hang onto the etcd3 track as much as possible so that
> >> if/when that happens I still have a job :).
> >>
> >>
> >>
> >>
> >>>
> >>> Perhaps a good idea to actually give examples of how it should be used,
> >>> how it shouldn't be used, what it offers, what it doesn't... Or at
> least
> >>> provide links for people to read up on this.
> >>>
> >>> Thoughts?
> >>>
> >>> Davanum Srinivas wrote:
> 
>  One clarification: Since https://pypi.python.org/pypi/etcd3gw just
>  uses the HTTP API (/v3alpha) it will work under both eventlet and
>  non-eventlet environments.
> 
>  Thanks,
>  Dims
> 
> 
>  On Wed, Jun 7, 2017 at 6:47 AM, Davanum Srinivas
>  wrote:
> >
> > Team,
> >
> > Here's the update to the base services resolution from the TC:
> > https://governance.openstack.org/tc/reference/base-services.html
> >
> > First request is to Distros, Packagers, Deployers, anyone who
> > installs/configures OpenStack:
> > Please make sure you have latest etcd 3.x available in your
> > environment for Services to use, Fedora already does, we need help in
> > making sure all distros and architectures are covered.
> >
> > Any project who want to use etcd v3 API via grpc, please use:
> > https://pypi.python.org/pypi/etcd3 (works only for non-eventlet
> > services)
> >
> > Those that depend on eventlet, please use the etcd3 v3alpha HTTP API
> > using:
> > https://pypi.python.org/pypi/etcd3gw
> >
> > If you use tooz, there are 2 driver choices for you:
> > https://github.com/openstack/tooz/blob/master/setup.cfg#L29
> > https://github.com/openstack/tooz/blob/master/setup.cfg#L30
> >
> > If you use oslo.cache, there is a driver for you:
> > https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33
> >
> > Devstack installs etcd3 by default and points cinder to it:
> > http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
> >
> > http://git.openstack.org/cgit/openstack-dev/devstack/tree/
> lib/cinder#n356
> >
> > Review in progress for 

Re: [openstack-dev] [all] etcd3 as base service - update

[Davanum Srinivas]

On Thu, Jun 8, 2017 at 4:07 PM, Drew Fisher  wrote:
>
>
> On 6/7/17 4:47 AM, Davanum Srinivas wrote:
>> Team,
>>
>> Here's the update to the base services resolution from the TC:
>> https://governance.openstack.org/tc/reference/base-services.html
>>
>> First request is to Distros, Packagers, Deployers, anyone who
>> installs/configures OpenStack:
>> Please make sure you have latest etcd 3.x available in your
>> environment for Services to use, Fedora already does, we need help in
>> making sure all distros and architectures are covered.
>
> As a Solaris OpenStack dev, I have a questions about this change.
>
> If Solaris were to *only* run the nova-compute service, and leave the
> rest of the OpenStack services to Linux, is etcd 3.x required on the
> compute node for Pike+ ?

Yes, this should be fine.

> Thanks!
>
> -Drew
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Fox, Kevin M]

See the footer at the bottom of this email.

From: jimi olugboyega [jimiolugboy...@gmail.com]
Sent: Thursday, June 08, 2017 1:19 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all] etcd3 as base service - update

Hello all,

I am wondering how I can unsubscribe from this mailing list.

Regards,
Olujimi Olugboyega.

On Wed, Jun 7, 2017 at 3:47 AM, Davanum Srinivas 
<dava...@gmail.com<mailto:dava...@gmail.com>> wrote:
Team,

Here's the update to the base services resolution from the TC:
https://governance.openstack.org/tc/reference/base-services.html

First request is to Distros, Packagers, Deployers, anyone who
installs/configures OpenStack:
Please make sure you have latest etcd 3.x available in your
environment for Services to use, Fedora already does, we need help in
making sure all distros and architectures are covered.

Any project who want to use etcd v3 API via grpc, please use:
https://pypi.python.org/pypi/etcd3 (works only for non-eventlet services)

Those that depend on eventlet, please use the etcd3 v3alpha HTTP API using:
https://pypi.python.org/pypi/etcd3gw

If you use tooz, there are 2 driver choices for you:
https://github.com/openstack/tooz/blob/master/setup.cfg#L29
https://github.com/openstack/tooz/blob/master/setup.cfg#L30

If you use oslo.cache, there is a driver for you:
https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33

Devstack installs etcd3 by default and points cinder to it:
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n356

Review in progress for keystone to use etcd3 for caching:
https://review.openstack.org/#/c/469621/

Doug is working on proposal(s) for oslo.config to store some
configuration in etcd3:
https://review.openstack.org/#/c/454897/

So, feel free to turn on / test with etcd3 and report issues.

Thanks,
Dims

--
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Fox, Kevin M]

hmm... a very interesting question

I would think control plane only.

Thanks,
Kevin

From: Drew Fisher [drew.fis...@oracle.com]
Sent: Thursday, June 08, 2017 1:07 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [all] etcd3 as base service - update

On 6/7/17 4:47 AM, Davanum Srinivas wrote:
> Team,
>
> Here's the update to the base services resolution from the TC:
> https://governance.openstack.org/tc/reference/base-services.html
>
> First request is to Distros, Packagers, Deployers, anyone who
> installs/configures OpenStack:
> Please make sure you have latest etcd 3.x available in your
> environment for Services to use, Fedora already does, we need help in
> making sure all distros and architectures are covered.

As a Solaris OpenStack dev, I have a questions about this change.

If Solaris were to *only* run the nova-compute service, and leave the
rest of the OpenStack services to Linux, is etcd 3.x required on the
compute node for Pike+ ?

Thanks!

-Drew



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Julien Danjou]

On Thu, Jun 08 2017, Mike Bayer wrote:

> So far I've seen a proposal of etcd3 as a replacement for memcached in
> keystone, and a new dogpile connector was added to oslo.cache to handle
> referring to etcd3 as a cache backend.  This is a really simplistic / minimal
> kind of use case for a key-store.

etcd3 is not meant to be a cache. Synchronizing caching value using the
Raft protocol sounds a bit overkill. A cluster of memcached would be
probably of a better use.

> But, keeping in mind I don't know anything about etcd3 other than "it's 
> another
> key-store", it's the only database used by Kubernetes as a whole, which
> suggests it's doing a better job than Redis in terms of "durable".

Not sure about that. And Redis has much more data structure than etcd,
that is can be faster/more efficient than etcd. But it does not have
Raft and a synchronisation protocol. Its clustering is rather poor in
comparison of etcd.

> So I wouldn't be surprised if new / existing openstack applications
> express some gravitational pull towards using it as their own
> datastore as well. I'll be trying to hang onto the etcd3 track as much
> as possible so that if/when that happens I still have a job :).

Sounds like a recipe for disaster. :)

-- 
Julien Danjou
/* Free Software hacker
   https://julien.danjou.info */


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Emilien Macchi]

On Thu, Jun 8, 2017 at 7:34 PM, Lance Bragstad  wrote:
> After digging into etcd a bit, one place this might be help deployer
> experience would be the handling of fernet keys for token encryption in
> keystone. Currently, all keys used to encrypt and decrypt tokens are kept on
> disk for each keystone node in the deployment. While simple, it requires
> operators to perform rotation on a single node and then push, or sync, the
> new key set to the rest of the nodes. This must be done in lock step in
> order to prevent early token invalidation and inconsistent token responses.

This is what we discussed a few months ago :-)

http://lists.openstack.org/pipermail/openstack-dev/2017-March/113943.html

I'm glad it's coming back ;-)

> An alternative would be to keep the keys in etcd and make the fernet bits
> pluggable so that it's possible to read keys from disk or etcd (pending
> configuration). The advantage would be that operators could initiate key
> rotations from any keystone node in the deployment (or using etcd directly)
> and not have to worry about distributing the new key set. Since etcd
> associates metadata to the key-value pairs, we might be able to simplify the
> rotation strategy as well.
>
> On Thu, Jun 8, 2017 at 11:37 AM, Mike Bayer  wrote:
>>
>>
>>
>> On 06/08/2017 12:47 AM, Joshua Harlow wrote:
>>>
>>> So just out of curiosity, but do people really even know what etcd is
>>> good for? I am thinking that there should be some guidance from folks in the
>>> community as to where etcd should be used and where it shouldn't (otherwise
>>> we just all end up in a mess).
>>
>>
>> So far I've seen a proposal of etcd3 as a replacement for memcached in
>> keystone, and a new dogpile connector was added to oslo.cache to handle
>> referring to etcd3 as a cache backend.  This is a really simplistic /
>> minimal kind of use case for a key-store.
>>
>> But, keeping in mind I don't know anything about etcd3 other than "it's
>> another key-store", it's the only database used by Kubernetes as a whole,
>> which suggests it's doing a better job than Redis in terms of "durable".
>> So I wouldn't be surprised if new / existing openstack applications express
>> some gravitational pull towards using it as their own datastore as well.
>> I'll be trying to hang onto the etcd3 track as much as possible so that
>> if/when that happens I still have a job :).
>>
>>
>>
>>
>>>
>>> Perhaps a good idea to actually give examples of how it should be used,
>>> how it shouldn't be used, what it offers, what it doesn't... Or at least
>>> provide links for people to read up on this.
>>>
>>> Thoughts?
>>>
>>> Davanum Srinivas wrote:

 One clarification: Since https://pypi.python.org/pypi/etcd3gw just
 uses the HTTP API (/v3alpha) it will work under both eventlet and
 non-eventlet environments.

 Thanks,
 Dims


 On Wed, Jun 7, 2017 at 6:47 AM, Davanum Srinivas
 wrote:
>
> Team,
>
> Here's the update to the base services resolution from the TC:
> https://governance.openstack.org/tc/reference/base-services.html
>
> First request is to Distros, Packagers, Deployers, anyone who
> installs/configures OpenStack:
> Please make sure you have latest etcd 3.x available in your
> environment for Services to use, Fedora already does, we need help in
> making sure all distros and architectures are covered.
>
> Any project who want to use etcd v3 API via grpc, please use:
> https://pypi.python.org/pypi/etcd3 (works only for non-eventlet
> services)
>
> Those that depend on eventlet, please use the etcd3 v3alpha HTTP API
> using:
> https://pypi.python.org/pypi/etcd3gw
>
> If you use tooz, there are 2 driver choices for you:
> https://github.com/openstack/tooz/blob/master/setup.cfg#L29
> https://github.com/openstack/tooz/blob/master/setup.cfg#L30
>
> If you use oslo.cache, there is a driver for you:
> https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33
>
> Devstack installs etcd3 by default and points cinder to it:
> http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
>
> http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n356
>
> Review in progress for keystone to use etcd3 for caching:
> https://review.openstack.org/#/c/469621/
>
> Doug is working on proposal(s) for oslo.config to store some
> configuration in etcd3:
> https://review.openstack.org/#/c/454897/
>
> So, feel free to turn on / test with etcd3 and report issues.
>
> Thanks,
> Dims
>
> --
> Davanum Srinivas :: https://twitter.com/dims




>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> 

Re: [openstack-dev] [all] etcd3 as base service - update

[jimi olugboyega]

Hello all,

I am wondering how I can unsubscribe from this mailing list.

Regards,
Olujimi Olugboyega.

On Wed, Jun 7, 2017 at 3:47 AM, Davanum Srinivas  wrote:

> Team,
>
> Here's the update to the base services resolution from the TC:
> https://governance.openstack.org/tc/reference/base-services.html
>
> First request is to Distros, Packagers, Deployers, anyone who
> installs/configures OpenStack:
> Please make sure you have latest etcd 3.x available in your
> environment for Services to use, Fedora already does, we need help in
> making sure all distros and architectures are covered.
>
> Any project who want to use etcd v3 API via grpc, please use:
> https://pypi.python.org/pypi/etcd3 (works only for non-eventlet services)
>
> Those that depend on eventlet, please use the etcd3 v3alpha HTTP API using:
> https://pypi.python.org/pypi/etcd3gw
>
> If you use tooz, there are 2 driver choices for you:
> https://github.com/openstack/tooz/blob/master/setup.cfg#L29
> https://github.com/openstack/tooz/blob/master/setup.cfg#L30
>
> If you use oslo.cache, there is a driver for you:
> https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33
>
> Devstack installs etcd3 by default and points cinder to it:
> http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
> http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n356
>
> Review in progress for keystone to use etcd3 for caching:
> https://review.openstack.org/#/c/469621/
>
> Doug is working on proposal(s) for oslo.config to store some
> configuration in etcd3:
> https://review.openstack.org/#/c/454897/
>
> So, feel free to turn on / test with etcd3 and report issues.
>
> Thanks,
> Dims
>
> --
> Davanum Srinivas :: https://twitter.com/dims
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Drew Fisher]

On 6/7/17 4:47 AM, Davanum Srinivas wrote:
> Team,
> 
> Here's the update to the base services resolution from the TC:
> https://governance.openstack.org/tc/reference/base-services.html
> 
> First request is to Distros, Packagers, Deployers, anyone who
> installs/configures OpenStack:
> Please make sure you have latest etcd 3.x available in your
> environment for Services to use, Fedora already does, we need help in
> making sure all distros and architectures are covered.

As a Solaris OpenStack dev, I have a questions about this change.

If Solaris were to *only* run the nova-compute service, and leave the
rest of the OpenStack services to Linux, is etcd 3.x required on the
compute node for Pike+ ?

Thanks!

-Drew



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Fox, Kevin M]

So, one thing to remember, I don't think etcd has an authz mechanism yet.

You usually want your fernet keys to be accessible by just the keystone nodes 
and no others.

This might require a etcd cluster just for keystone fernet tokens, which might 
work great. but is an operator overhead to install/maintain.


From: Lance Bragstad [lbrags...@gmail.com]
Sent: Thursday, June 08, 2017 10:34 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all] etcd3 as base service - update

After digging into etcd a bit, one place this might be help deployer experience 
would be the handling of fernet keys for token encryption in keystone. 
Currently, all keys used to encrypt and decrypt tokens are kept on disk for 
each keystone node in the deployment. While simple, it requires operators to 
perform rotation on a single node and then push, or sync, the new key set to 
the rest of the nodes. This must be done in lock step in order to prevent early 
token invalidation and inconsistent token responses.

An alternative would be to keep the keys in etcd and make the fernet bits 
pluggable so that it's possible to read keys from disk or etcd (pending 
configuration). The advantage would be that operators could initiate key 
rotations from any keystone node in the deployment (or using etcd directly) and 
not have to worry about distributing the new key set. Since etcd associates 
metadata to the key-value pairs, we might be able to simplify the rotation 
strategy as well.

On Thu, Jun 8, 2017 at 11:37 AM, Mike Bayer 
<mba...@redhat.com<mailto:mba...@redhat.com>> wrote:


On 06/08/2017 12:47 AM, Joshua Harlow wrote:
So just out of curiosity, but do people really even know what etcd is good for? 
I am thinking that there should be some guidance from folks in the community as 
to where etcd should be used and where it shouldn't (otherwise we just all end 
up in a mess).

So far I've seen a proposal of etcd3 as a replacement for memcached in 
keystone, and a new dogpile connector was added to oslo.cache to handle 
referring to etcd3 as a cache backend.  This is a really simplistic / minimal 
kind of use case for a key-store.

But, keeping in mind I don't know anything about etcd3 other than "it's another 
key-store", it's the only database used by Kubernetes as a whole, which 
suggests it's doing a better job than Redis in terms of "durable".   So I 
wouldn't be surprised if new / existing openstack applications express some 
gravitational pull towards using it as their own datastore as well.I'll be 
trying to hang onto the etcd3 track as much as possible so that if/when that 
happens I still have a job :).





Perhaps a good idea to actually give examples of how it should be used, how it 
shouldn't be used, what it offers, what it doesn't... Or at least provide links 
for people to read up on this.

Thoughts?

Davanum Srinivas wrote:
One clarification: Since https://pypi.python.org/pypi/etcd3gw just
uses the HTTP API (/v3alpha) it will work under both eventlet and
non-eventlet environments.

Thanks,
Dims


On Wed, Jun 7, 2017 at 6:47 AM, Davanum 
Srinivas<dava...@gmail.com<mailto:dava...@gmail.com>>  wrote:
Team,

Here's the update to the base services resolution from the TC:
https://governance.openstack.org/tc/reference/base-services.html

First request is to Distros, Packagers, Deployers, anyone who
installs/configures OpenStack:
Please make sure you have latest etcd 3.x available in your
environment for Services to use, Fedora already does, we need help in
making sure all distros and architectures are covered.

Any project who want to use etcd v3 API via grpc, please use:
https://pypi.python.org/pypi/etcd3 (works only for non-eventlet services)

Those that depend on eventlet, please use the etcd3 v3alpha HTTP API using:
https://pypi.python.org/pypi/etcd3gw

If you use tooz, there are 2 driver choices for you:
https://github.com/openstack/tooz/blob/master/setup.cfg#L29
https://github.com/openstack/tooz/blob/master/setup.cfg#L30

If you use oslo.cache, there is a driver for you:
https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33

Devstack installs etcd3 by default and points cinder to it:
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n356

Review in progress for keystone to use etcd3 for caching:
https://review.openstack.org/#/c/469621/

Doug is working on proposal(s) for oslo.config to store some
configuration in etcd3:
https://review.openstack.org/#/c/454897/

So, feel free to turn on / test with etcd3 and report issues.

Thanks,
Dims

--
Davanum Srinivas :: https://twitter.com/dims




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?s

Re: [openstack-dev] [all] etcd3 as base service - update

[Lance Bragstad]

After digging into etcd a bit, one place this might be help deployer
experience would be the handling of fernet keys for token encryption in
keystone. Currently, all keys used to encrypt and decrypt tokens are kept
on disk for each keystone node in the deployment. While simple, it requires
operators to perform rotation on a single node and then push, or sync, the
new key set to the rest of the nodes. This must be done in lock step in
order to prevent early token invalidation and inconsistent token responses.

An alternative would be to keep the keys in etcd and make the fernet bits
pluggable so that it's possible to read keys from disk or etcd (pending
configuration). The advantage would be that operators could initiate key
rotations from any keystone node in the deployment (or using etcd directly)
and not have to worry about distributing the new key set. Since etcd
associates metadata to the key-value pairs, we might be able to simplify
the rotation strategy as well.

On Thu, Jun 8, 2017 at 11:37 AM, Mike Bayer  wrote:

>
>
> On 06/08/2017 12:47 AM, Joshua Harlow wrote:
>
>> So just out of curiosity, but do people really even know what etcd is
>> good for? I am thinking that there should be some guidance from folks in
>> the community as to where etcd should be used and where it shouldn't
>> (otherwise we just all end up in a mess).
>>
>
> So far I've seen a proposal of etcd3 as a replacement for memcached in
> keystone, and a new dogpile connector was added to oslo.cache to handle
> referring to etcd3 as a cache backend.  This is a really simplistic /
> minimal kind of use case for a key-store.
>
> But, keeping in mind I don't know anything about etcd3 other than "it's
> another key-store", it's the only database used by Kubernetes as a whole,
> which suggests it's doing a better job than Redis in terms of "durable".
>  So I wouldn't be surprised if new / existing openstack applications
> express some gravitational pull towards using it as their own datastore as
> well.I'll be trying to hang onto the etcd3 track as much as possible so
> that if/when that happens I still have a job :).
>
>
>
>
>
>> Perhaps a good idea to actually give examples of how it should be used,
>> how it shouldn't be used, what it offers, what it doesn't... Or at least
>> provide links for people to read up on this.
>>
>> Thoughts?
>>
>> Davanum Srinivas wrote:
>>
>>> One clarification: Since https://pypi.python.org/pypi/etcd3gw just
>>> uses the HTTP API (/v3alpha) it will work under both eventlet and
>>> non-eventlet environments.
>>>
>>> Thanks,
>>> Dims
>>>
>>>
>>> On Wed, Jun 7, 2017 at 6:47 AM, Davanum Srinivas
>>> wrote:
>>>
 Team,

 Here's the update to the base services resolution from the TC:
 https://governance.openstack.org/tc/reference/base-services.html

 First request is to Distros, Packagers, Deployers, anyone who
 installs/configures OpenStack:
 Please make sure you have latest etcd 3.x available in your
 environment for Services to use, Fedora already does, we need help in
 making sure all distros and architectures are covered.

 Any project who want to use etcd v3 API via grpc, please use:
 https://pypi.python.org/pypi/etcd3 (works only for non-eventlet
 services)

 Those that depend on eventlet, please use the etcd3 v3alpha HTTP API
 using:
 https://pypi.python.org/pypi/etcd3gw

 If you use tooz, there are 2 driver choices for you:
 https://github.com/openstack/tooz/blob/master/setup.cfg#L29
 https://github.com/openstack/tooz/blob/master/setup.cfg#L30

 If you use oslo.cache, there is a driver for you:
 https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33

 Devstack installs etcd3 by default and points cinder to it:
 http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
 http://git.openstack.org/cgit/openstack-dev/devstack/tree/li
 b/cinder#n356

 Review in progress for keystone to use etcd3 for caching:
 https://review.openstack.org/#/c/469621/

 Doug is working on proposal(s) for oslo.config to store some
 configuration in etcd3:
 https://review.openstack.org/#/c/454897/

 So, feel free to turn on / test with etcd3 and report issues.

 Thanks,
 Dims

 --
 Davanum Srinivas :: https://twitter.com/dims

>>>
>>>
>>>
>>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 

Re: [openstack-dev] [all] etcd3 as base service - update

[Mike Bayer]

On 06/08/2017 12:47 AM, Joshua Harlow wrote:
So just out of curiosity, but do people really even know what etcd is 
good for? I am thinking that there should be some guidance from folks in 
the community as to where etcd should be used and where it shouldn't 
(otherwise we just all end up in a mess).


So far I've seen a proposal of etcd3 as a replacement for memcached in 
keystone, and a new dogpile connector was added to oslo.cache to handle 
referring to etcd3 as a cache backend.  This is a really simplistic / 
minimal kind of use case for a key-store.


But, keeping in mind I don't know anything about etcd3 other than "it's 
another key-store", it's the only database used by Kubernetes as a 
whole, which suggests it's doing a better job than Redis in terms of 
"durable".   So I wouldn't be surprised if new / existing openstack 
applications express some gravitational pull towards using it as their 
own datastore as well.I'll be trying to hang onto the etcd3 track as 
much as possible so that if/when that happens I still have a job :).






Perhaps a good idea to actually give examples of how it should be used, 
how it shouldn't be used, what it offers, what it doesn't... Or at least 
provide links for people to read up on this.


Thoughts?

Davanum Srinivas wrote:

One clarification: Since https://pypi.python.org/pypi/etcd3gw just
uses the HTTP API (/v3alpha) it will work under both eventlet and
non-eventlet environments.

Thanks,
Dims


On Wed, Jun 7, 2017 at 6:47 AM, Davanum Srinivas  
wrote:

Team,

Here's the update to the base services resolution from the TC:
https://governance.openstack.org/tc/reference/base-services.html

First request is to Distros, Packagers, Deployers, anyone who
installs/configures OpenStack:
Please make sure you have latest etcd 3.x available in your
environment for Services to use, Fedora already does, we need help in
making sure all distros and architectures are covered.

Any project who want to use etcd v3 API via grpc, please use:
https://pypi.python.org/pypi/etcd3 (works only for non-eventlet 
services)


Those that depend on eventlet, please use the etcd3 v3alpha HTTP API 
using:

https://pypi.python.org/pypi/etcd3gw

If you use tooz, there are 2 driver choices for you:
https://github.com/openstack/tooz/blob/master/setup.cfg#L29
https://github.com/openstack/tooz/blob/master/setup.cfg#L30

If you use oslo.cache, there is a driver for you:
https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33

Devstack installs etcd3 by default and points cinder to it:
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n356 



Review in progress for keystone to use etcd3 for caching:
https://review.openstack.org/#/c/469621/

Doug is working on proposal(s) for oslo.config to store some
configuration in etcd3:
https://review.openstack.org/#/c/454897/

So, feel free to turn on / test with etcd3 and report issues.

Thanks,
Dims

--
Davanum Srinivas :: https://twitter.com/dims






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Davanum Srinivas]

Josh,

Of the initial targets, we have covered the use of tooz and oslo.cache
with etcd already. One thing that remains is Jay's os-lively PoC [1].

So +1 to more knowledgeable folks chiming in with stuff that can be
done and other scenarios. I'll let nature take its course here :)

Thanks,
Dims

[1] https://github.com/jaypipes/os-lively/

On Thu, Jun 8, 2017 at 12:47 AM, Joshua Harlow  wrote:
> So just out of curiosity, but do people really even know what etcd is good
> for? I am thinking that there should be some guidance from folks in the
> community as to where etcd should be used and where it shouldn't (otherwise
> we just all end up in a mess).
>
> Perhaps a good idea to actually give examples of how it should be used, how
> it shouldn't be used, what it offers, what it doesn't... Or at least provide
> links for people to read up on this.
>
> Thoughts?
>
>
> Davanum Srinivas wrote:
>>
>> One clarification: Since https://pypi.python.org/pypi/etcd3gw just
>> uses the HTTP API (/v3alpha) it will work under both eventlet and
>> non-eventlet environments.
>>
>> Thanks,
>> Dims
>>
>>
>> On Wed, Jun 7, 2017 at 6:47 AM, Davanum Srinivas
>> wrote:
>>>
>>> Team,
>>>
>>> Here's the update to the base services resolution from the TC:
>>> https://governance.openstack.org/tc/reference/base-services.html
>>>
>>> First request is to Distros, Packagers, Deployers, anyone who
>>> installs/configures OpenStack:
>>> Please make sure you have latest etcd 3.x available in your
>>> environment for Services to use, Fedora already does, we need help in
>>> making sure all distros and architectures are covered.
>>>
>>> Any project who want to use etcd v3 API via grpc, please use:
>>> https://pypi.python.org/pypi/etcd3 (works only for non-eventlet services)
>>>
>>> Those that depend on eventlet, please use the etcd3 v3alpha HTTP API
>>> using:
>>> https://pypi.python.org/pypi/etcd3gw
>>>
>>> If you use tooz, there are 2 driver choices for you:
>>> https://github.com/openstack/tooz/blob/master/setup.cfg#L29
>>> https://github.com/openstack/tooz/blob/master/setup.cfg#L30
>>>
>>> If you use oslo.cache, there is a driver for you:
>>> https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33
>>>
>>> Devstack installs etcd3 by default and points cinder to it:
>>> http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
>>> http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n356
>>>
>>> Review in progress for keystone to use etcd3 for caching:
>>> https://review.openstack.org/#/c/469621/
>>>
>>> Doug is working on proposal(s) for oslo.config to store some
>>> configuration in etcd3:
>>> https://review.openstack.org/#/c/454897/
>>>
>>> So, feel free to turn on / test with etcd3 and report issues.
>>>
>>> Thanks,
>>> Dims
>>>
>>> --
>>> Davanum Srinivas :: https://twitter.com/dims
>>
>>
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Thierry Carrez]

Joshua Harlow wrote:
> So just out of curiosity, but do people really even know what etcd is
> good for? I am thinking that there should be some guidance from folks in
> the community as to where etcd should be used and where it shouldn't
> (otherwise we just all end up in a mess).
> 
> Perhaps a good idea to actually give examples of how it should be used,
> how it shouldn't be used, what it offers, what it doesn't... Or at least
> provide links for people to read up on this.
> 
> Thoughts?

I think that's a great idea. More generally, we should document base
services, the benefits of each tech, and what data should be in
database, what data should be in a message queue, and what data should
be in etcd. It feels like the Project Team Guide could be a place where
such information could live...

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Joshua Harlow]

So just out of curiosity, but do people really even know what etcd is 
good for? I am thinking that there should be some guidance from folks in 
the community as to where etcd should be used and where it shouldn't 
(otherwise we just all end up in a mess).


Perhaps a good idea to actually give examples of how it should be used, 
how it shouldn't be used, what it offers, what it doesn't... Or at least 
provide links for people to read up on this.


Thoughts?

Davanum Srinivas wrote:

One clarification: Since https://pypi.python.org/pypi/etcd3gw just
uses the HTTP API (/v3alpha) it will work under both eventlet and
non-eventlet environments.

Thanks,
Dims


On Wed, Jun 7, 2017 at 6:47 AM, Davanum Srinivas  wrote:

Team,

Here's the update to the base services resolution from the TC:
https://governance.openstack.org/tc/reference/base-services.html

First request is to Distros, Packagers, Deployers, anyone who
installs/configures OpenStack:
Please make sure you have latest etcd 3.x available in your
environment for Services to use, Fedora already does, we need help in
making sure all distros and architectures are covered.

Any project who want to use etcd v3 API via grpc, please use:
https://pypi.python.org/pypi/etcd3 (works only for non-eventlet services)

Those that depend on eventlet, please use the etcd3 v3alpha HTTP API using:
https://pypi.python.org/pypi/etcd3gw

If you use tooz, there are 2 driver choices for you:
https://github.com/openstack/tooz/blob/master/setup.cfg#L29
https://github.com/openstack/tooz/blob/master/setup.cfg#L30

If you use oslo.cache, there is a driver for you:
https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33

Devstack installs etcd3 by default and points cinder to it:
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n356

Review in progress for keystone to use etcd3 for caching:
https://review.openstack.org/#/c/469621/

Doug is working on proposal(s) for oslo.config to store some
configuration in etcd3:
https://review.openstack.org/#/c/454897/

So, feel free to turn on / test with etcd3 and report issues.

Thanks,
Dims

--
Davanum Srinivas :: https://twitter.com/dims






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] etcd3 as base service - update

[Davanum Srinivas]

One clarification: Since https://pypi.python.org/pypi/etcd3gw just
uses the HTTP API (/v3alpha) it will work under both eventlet and
non-eventlet environments.

Thanks,
Dims


On Wed, Jun 7, 2017 at 6:47 AM, Davanum Srinivas  wrote:
> Team,
>
> Here's the update to the base services resolution from the TC:
> https://governance.openstack.org/tc/reference/base-services.html
>
> First request is to Distros, Packagers, Deployers, anyone who
> installs/configures OpenStack:
> Please make sure you have latest etcd 3.x available in your
> environment for Services to use, Fedora already does, we need help in
> making sure all distros and architectures are covered.
>
> Any project who want to use etcd v3 API via grpc, please use:
> https://pypi.python.org/pypi/etcd3 (works only for non-eventlet services)
>
> Those that depend on eventlet, please use the etcd3 v3alpha HTTP API using:
> https://pypi.python.org/pypi/etcd3gw
>
> If you use tooz, there are 2 driver choices for you:
> https://github.com/openstack/tooz/blob/master/setup.cfg#L29
> https://github.com/openstack/tooz/blob/master/setup.cfg#L30
>
> If you use oslo.cache, there is a driver for you:
> https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33
>
> Devstack installs etcd3 by default and points cinder to it:
> http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
> http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n356
>
> Review in progress for keystone to use etcd3 for caching:
> https://review.openstack.org/#/c/469621/
>
> Doug is working on proposal(s) for oslo.config to store some
> configuration in etcd3:
> https://review.openstack.org/#/c/454897/
>
> So, feel free to turn on / test with etcd3 and report issues.
>
> Thanks,
> Dims
>
> --
> Davanum Srinivas :: https://twitter.com/dims



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] etcd3 as base service - update

[Davanum Srinivas]

Team,

Here's the update to the base services resolution from the TC:
https://governance.openstack.org/tc/reference/base-services.html

First request is to Distros, Packagers, Deployers, anyone who
installs/configures OpenStack:
Please make sure you have latest etcd 3.x available in your
environment for Services to use, Fedora already does, we need help in
making sure all distros and architectures are covered.

Any project who want to use etcd v3 API via grpc, please use:
https://pypi.python.org/pypi/etcd3 (works only for non-eventlet services)

Those that depend on eventlet, please use the etcd3 v3alpha HTTP API using:
https://pypi.python.org/pypi/etcd3gw

If you use tooz, there are 2 driver choices for you:
https://github.com/openstack/tooz/blob/master/setup.cfg#L29
https://github.com/openstack/tooz/blob/master/setup.cfg#L30

If you use oslo.cache, there is a driver for you:
https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L33

Devstack installs etcd3 by default and points cinder to it:
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/etcd3
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n356

Review in progress for keystone to use etcd3 for caching:
https://review.openstack.org/#/c/469621/

Doug is working on proposal(s) for oslo.config to store some
configuration in etcd3:
https://review.openstack.org/#/c/454897/

So, feel free to turn on / test with etcd3 and report issues.

Thanks,
Dims

-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev