Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-30 Thread Julien Danjou
On Mon, Sep 29 2014, Morgan Fainberg wrote: The big issue you're going to run into is locking. The indexes need to have a distributed lock that guarantees that each index is read/updated/released atomically (similar to the SQL transaction). The way memcache and redis handle this is by trying

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-30 Thread Morgan Fainberg
) openstack-dev@lists.openstack.org Subject:  Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift? +1 Lets not continue to expand the usage of persisted tokens :-/ We can look at moving away from persisted tokens, but as of the Kilo cycle there is no plan to eliminate

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Julien Danjou
On Mon, Sep 29 2014, Jay Pipes wrote: What if we wrote a token driver in Keystone that uses Swift for backend storage? Yay! I already wrote a PoC to that: https://review.openstack.org/#/c/86016/ It has been rejected because this patch didn't use the generic approach that Keystone tries to

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Jay Pipes
On 09/29/2014 12:15 PM, Julien Danjou wrote: On Mon, Sep 29 2014, Jay Pipes wrote: What if we wrote a token driver in Keystone that uses Swift for backend storage? Yay! I already wrote a PoC to that: https://review.openstack.org/#/c/86016/ Sweet! :) It has been rejected because this

[openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Jay Pipes
Hey Stackers, So, I had a thought this morning (uh-oh, I know...). What if we wrote a token driver in Keystone that uses Swift for backend storage? I have long been an advocate of the memcache token driver versus the SQL driver for performance reasons. However, the problem with the memcache

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Lance Bragstad
On Mon, Sep 29, 2014 at 11:25 AM, Jay Pipes jaypi...@gmail.com wrote: On 09/29/2014 12:15 PM, Julien Danjou wrote: On Mon, Sep 29 2014, Jay Pipes wrote: What if we wrote a token driver in Keystone that uses Swift for backend storage? Yay! I already wrote a PoC to that:

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Julien Danjou
On Mon, Sep 29 2014, Jay Pipes wrote: Any objection to me taking up the work? Was there any associated blueprint for it? As said on IRC, go ahead. There's no blueprint associated AFAIK. :) -- Julien Danjou /* Free Software hacker http://julien.danjou.info */ signature.asc Description:

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Morgan Fainberg
On Monday, September 29, 2014, Julien Danjou jul...@danjou.info wrote: On Mon, Sep 29 2014, Jay Pipes wrote: Any objection to me taking up the work? Was there any associated blueprint for it? As said on IRC, go ahead. There's no blueprint associated AFAIK. :) -- Julien Danjou /* Free

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Dmitry Mescheryakov
Hey Jay, Did you consider Swift's eventual consistency? The general use case for many OpenStack application is: 1. obtain the token from Keystone 2. perform some operation in OpenStack providing token as credentials. As a result of operation #1 the token will be saved into Swift by the

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Chmouel Boudjnah
On Mon, Sep 29, 2014 at 11:47 PM, Dmitry Mescheryakov dmescherya...@mirantis.com wrote: As a result of operation #1 the token will be saved into Swift by the Keystone. But due to eventual consistency it could happen that validation of token in operation #2 will not see the saved token.

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Clay Gerrard
On Mon, Sep 29, 2014 at 2:53 PM, Chmouel Boudjnah chmo...@enovance.com wrote: eventual consistency will only affect container listing and I don't think there is a need for container listing in that driver. well now hold on... if you're doing an overwrite in the face of server failures you

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Chmouel Boudjnah
On 30/09/2014 01:05, Clay Gerrard wrote: eventual consistency will only affect container listing and I don't think there is a need for container listing in that driver. well now hold on... if you're doing an overwrite in the face of server failures you could still get a stale read if a

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Clint Byrum
Excerpts from Clay Gerrard's message of 2014-09-29 16:05:14 -0700: On Mon, Sep 29, 2014 at 2:53 PM, Chmouel Boudjnah chmo...@enovance.com wrote: eventual consistency will only affect container listing and I don't think there is a need for container listing in that driver. well

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Clay Gerrard
On Mon, Sep 29, 2014 at 4:15 PM, Clint Byrum cl...@fewbar.com wrote: It would, however, be bad to get a 404 for something that is otherwise present.. as that will result in an erroneous failure for the client. That almost never happens, but is possible if all the primaries are down*, a

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Morgan Fainberg
][swift] Has anybody considered storing tokens in Swift? Excerpts from Clay Gerrard's message of 2014-09-29 16:05:14 -0700: On Mon, Sep 29, 2014 at 2:53 PM, Chmouel Boudjnah wrote: eventual consistency will only affect container listing and I don't think there is a need

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Adam Young
On 09/29/2014 12:12 PM, Jay Pipes wrote: Hey Stackers, So, I had a thought this morning (uh-oh, I know...). What if we wrote a token driver in Keystone that uses Swift for backend storage? I have long been an advocate of the memcache token driver versus the SQL driver for performance

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Clint Byrum
Excerpts from Adam Young's message of 2014-09-29 20:22:35 -0700: On 09/29/2014 12:12 PM, Jay Pipes wrote: Hey Stackers, So, I had a thought this morning (uh-oh, I know...). What if we wrote a token driver in Keystone that uses Swift for backend storage? I have long been an

Re: [openstack-dev] [keystone][swift] Has anybody considered storing tokens in Swift?

2014-09-29 Thread Joshua Harlow
+1 Lets not continue to expand the usage of persisted tokens :-/ We should be trying to move away from such types of persistence and its associated complexity IMHO. Most major websites don't need to have tokens that are saved around regions in there internal backend databases, just so people