Re: [Openstack] Glance/Swift tenant specific storage

2012-07-13 Thread Dan Prince
Eglynn pointed out some concurrency concerns in this review that I wanted to 
highlight:

  https://review.openstack.org/#/c/9552/

The issue is we need to provide Glance stores access to the request context. 
The current thought is that we'd like to modify Glance to create new Store 
objects for each request (thus avoiding concurrency issues). The existing 
Glance model creates singleton Store objects at startup. Before I go changing 
Glance in this regard I wanted to ask...

Does this seem reasonable?

We can still in create Store instances at startup for validation purposes. 
Although now that I think of it I find the log messages Glance generates from 
missing S3 and or Swift configs to be somewhat unnecessary. Perhaps we should 
rethink this startup validation as well?


Eoghan: Thanks for the detail comments on the reviews. Keep them coming.


Dan 


- Original Message -
 From: Dan Prince dpri...@redhat.com
 To: OpenStack Mailing List openstack@lists.launchpad.net
 Sent: Sunday, July 8, 2012 9:59:54 PM
 Subject: Glance/Swift tenant specific storage
 
 I started working on the Glance swift-tenant-specific-storage
 blueprint last week.
 
 I've got a working branch in play here:
 
   https://github.com/dprince/glance/commits/swift_multi_tenant3
 
 Some details on how I've done things so far:
 
  * Update Glance so that it uses the service catalog for each user to
  obtain the Swift storage URL.
 
  * Provide backend stores access to the context. Glance Essex doesn't
  give stores access to the RequestContext (auth token). We'll need
  this information for tenant specific storage if we want to be able
  to access individual swift accounts.
 
  * Store images in separate containers. Swift only allows individual
  ACL's to be set per container... not per object. As such it appears
  we'll need to store each image in a separate container in order to
  support setting public and/or individual read/write access on each
  image.
 
  * Set 'public' access for images in Swift.
 
  * Set 'private' read and/or write access for Glance image members
  which have been granted access to specific images.
 
  * Delayed delete (scrubber) will require an authenticated context in
  order to delete Swift images from the backend. Glance can either be
  make to grant write access to this account (for all images) or an
  administrative Swift account could be used to run the delayed
  delete operation.
 
  * Maintain full support with the existing single tenant Glance swift
  storage scheme.
 
 
 
 I made some general implementation notes up on this wiki page as
 well:
 
   http://wiki.openstack.org/GlanceSwiftTenantSpecificStorage
 
 I'm anxious to get things up for review but before I do so I wanted
 to ask if this implementation looks reasonable? Any thoughts or
 feedback?
 
 Dan

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Glance/Swift tenant specific storage

2012-07-09 Thread Dan Prince


- Original Message -
 From: David Kranz david.kr...@qrclab.com
 To: Dan Prince dpri...@redhat.com
 Sent: Monday, July 9, 2012 9:15:00 AM
 Subject: Re: [Openstack] Glance/Swift tenant specific storage
 
 Dan, I am involved in a project (not Glance) that is doing something
 similar to this though it is at an early stage. One unresolved issue
 was
 what happens when the auth_token used to access Swift expires?

If the auth_token expired the request would fail. I'm not sure how common this 
scenario would be however. I suppose if you are expiring tokens very quickly 
then it could happen... otherwise it would take a really long Glance task to be 
able to hit this right?


 Unless Glance stores the user's password, and in a reversible way, how does
 it
 get a new token?


It can't. The user would have to re-submit the request to upload, download, or 
access the image data with a fresh token. Note: There are still some Glance 
tasks like delayed delete which already required a username:password (stored in 
a config file) to work. With Multi-tenant this would still be the case.

One thing to note: Glance isn't storing auth_tokens in this new implementation. 
It simply obtains the token from the incoming request and uses that to access 
Swift accordingly. This is actually the same thing Nova does to access 
Glance... so presumably any token expiration issue you'd hit with Glance-Swift 
integration would also potentially happen between Nova-Glance.


 This seems like a keystone use case that is not
 clear
 unless I am missing something.
 
   -David
 
 On 7/8/2012 9:59 PM, Dan Prince wrote:
  I started working on the Glance swift-tenant-specific-storage
  blueprint last week.
 
  I've got a working branch in play here:
 
 https://github.com/dprince/glance/commits/swift_multi_tenant3
 
  Some details on how I've done things so far:
 
* Update Glance so that it uses the service catalog for each user
to obtain the Swift storage URL.
 
* Provide backend stores access to the context. Glance Essex
doesn't give stores access to the RequestContext (auth token).
We'll need this information for tenant specific storage if we
want to be able to access individual swift accounts.
 
* Store images in separate containers. Swift only allows
individual ACL's to be set per container... not per object. As
such it appears we'll need to store each image in a separate
container in order to support setting public and/or individual
read/write access on each image.
 
* Set 'public' access for images in Swift.
 
* Set 'private' read and/or write access for Glance image members
which have been granted access to specific images.
 
* Delayed delete (scrubber) will require an authenticated context
in order to delete Swift images from the backend. Glance can
either be make to grant write access to this account (for all
images) or an administrative Swift account could be used to run
the delayed delete operation.
 
* Maintain full support with the existing single tenant Glance
swift storage scheme.
 
  
 
  I made some general implementation notes up on this wiki page as
  well:
 
 http://wiki.openstack.org/GlanceSwiftTenantSpecificStorage
 
  I'm anxious to get things up for review but before I do so I wanted
  to ask if this implementation looks reasonable? Any thoughts or
  feedback?
 
  Dan
 
  ___
  Mailing list: https://launchpad.net/~openstack
  Post to : openstack@lists.launchpad.net
  Unsubscribe : https://launchpad.net/~openstack
  More help   : https://help.launchpad.net/ListHelp
 
 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp