Re: [Openstack] Deprecating nova-objectstore
-Original Message- From: Devin Carlen [mailto:devin.car...@gmail.com] Sent: 17 January 2011 20:08 To: Jay Pipes Cc: Thierry Carrez; Ewan Mellor; openstack@lists.launchpad.net Subject: Re: [Openstack] Deprecating nova-objectstore On Jan 17, 2011, at 11:59 AM, Jay Pipes wrote: On Mon, Jan 17, 2011 at 2:19 PM, Thierry Carrez thie...@openstack.org wrote: Jay Pipes wrote: I think the big difference is that clients would be talking to Glance and its RESTlike/JSON API, not to an S3 API front-end like nova-objectstore. The big question is whether OpenStack wants to support Amazon S3 as an objectstore at all, instead of just Swift, which now can communicate via an S3 API (http://swift.openstack.org/misc.html#module- swift.common.middleware.swift3). OK, my understanding was that euca-upload-bundle is talking directly to an S3-like backend, so if we want to support the EC2 way of registering images we need some S3-compatible server... Precisely. This is why nova-objectstore cannot be removed even if Glance supports S3 as a backend :) For serious deployments I guess we would use Swift S3 frontend. For demo cases, do we need a simpler solution ? And if yes, which one ? Depends on whether supporting euca-upload-bundle is going to be a priority once alternate openstack-API-speaking client tools are completed... :) My guess is that it won't, and nova-objectstore will eventually move into /ext or /plugin for those who will want to continue using euca-upload-bundle directly. I tend to disagree with this. Anyone who is using eucatools is going to want to use euca-upload-bundle. This is something we should continue to support. Not supporting euca-upload-bundle means we don't have a fully compatible EC2 implementation. Devin, you lost me. Can't we make euca-upload-bundle work against Swift's S3 frontend? Why do you say that we need nova-objectstore? Ewan. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Deprecating nova-objectstore
Hello everyone, The objectstore in nova is still alive in Bexar. The long term plan seems to be to get rid of it: it's still using eventlet and Glance is now covering most of its featureset. It is probably too late to remove it for Bexar, but we need to have a plan for Cactus. IIUC the objectstore still provides an easy way to do EC2-style image upload/registration. To do it with Glance, AFAICT you need to use an S3 backend (Swift with the S3 frontend or Amazon's S3 itself). That sounds a bit overkill, so does that mean we need a basic S3-like service to cover for that use case ? Should it be provided by Glance and reuse Glance filesystem backend, or be done outside Glance and reuse the S3 backend ? Thoughts ? -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Deprecating nova-objectstore
We could do it in two steps. You could set up nova-compute - Glance - nova-objectstore (testing purposes only). This would allow us to remove the S3 code from Nova, but people could still use nova-objectstore if they don't want to set up Swift. Incidentally, it's also possible for Glance to use a local filesystem rather than a remote store at all. This is perfectly reasonable for small setups. Ewan. -Original Message- From: openstack-bounces+ewan.mellor=citrix@lists.launchpad.net [mailto:openstack-bounces+ewan.mellor=citrix@lists.launchpad.net] On Behalf Of Thierry Carrez Sent: 17 January 2011 14:32 To: openstack@lists.launchpad.net Subject: [Openstack] Deprecating nova-objectstore Hello everyone, The objectstore in nova is still alive in Bexar. The long term plan seems to be to get rid of it: it's still using eventlet and Glance is now covering most of its featureset. It is probably too late to remove it for Bexar, but we need to have a plan for Cactus. IIUC the objectstore still provides an easy way to do EC2-style image upload/registration. To do it with Glance, AFAICT you need to use an S3 backend (Swift with the S3 frontend or Amazon's S3 itself). That sounds a bit overkill, so does that mean we need a basic S3-like service to cover for that use case ? Should it be provided by Glance and reuse Glance filesystem backend, or be done outside Glance and reuse the S3 backend ? Thoughts ? -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Deprecating nova-objectstore
-Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: 17 January 2011 17:17 To: Ewan Mellor Cc: Thierry Carrez; openstack@lists.launchpad.net Subject: Re: [Openstack] Deprecating nova-objectstore On Mon, Jan 17, 2011 at 10:53 AM, Ewan Mellor ewan.mel...@eu.citrix.com wrote: We could do it in two steps. You could set up nova-compute - Glance - nova-objectstore (testing purposes only). This would allow us to remove the S3 code from Nova, but people could still use nova- objectstore if they don't want to set up Swift. This is already done in Bexar, as Chris MacGown completed the S3 backend for Glance. What is NOT the same, though, is that people would not be speaking the s3 API as they do now... they would speak the Glance REST-like API instead... Yes, that's what I meant -- make Glance the only thing that speaks S3, and then we can remove the S3 code from Nova in favour of Glance. Plus, objects are a superset of images; Glance only stores images, not all objects... What does this mean? I didn't even know that anyone was distinguishing between an image and an object. Thanks, Ewan. Incidentally, it's also possible for Glance to use a local filesystem rather than a remote store at all. This is perfectly reasonable for small setups. I think the big difference is that clients would be talking to Glance and its RESTlike/JSON API, not to an S3 API front-end like nova-objectstore. The big question is whether OpenStack wants to support Amazon S3 as an objectstore at all, instead of just Swift, which now can communicate via an S3 API (http://swift.openstack.org/misc.html#module- swift.common.middleware.swift3). -jay Ewan. -Original Message- From: openstack-bounces+ewan.mellor=citrix@lists.launchpad.net [mailto:openstack-bounces+ewan.mellor=citrix@lists.launchpad.net] On Behalf Of Thierry Carrez Sent: 17 January 2011 14:32 To: openstack@lists.launchpad.net Subject: [Openstack] Deprecating nova-objectstore Hello everyone, The objectstore in nova is still alive in Bexar. The long term plan seems to be to get rid of it: it's still using eventlet and Glance is now covering most of its featureset. It is probably too late to remove it for Bexar, but we need to have a plan for Cactus. IIUC the objectstore still provides an easy way to do EC2-style image upload/registration. To do it with Glance, AFAICT you need to use an S3 backend (Swift with the S3 frontend or Amazon's S3 itself). That sounds a bit overkill, so does that mean we need a basic S3-like service to cover for that use case ? Should it be provided by Glance and reuse Glance filesystem backend, or be done outside Glance and reuse the S3 backend ? Thoughts ? -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Deprecating nova-objectstore
What's the difference between an image and an object? They're all just blobs of bits as far as I thought. I think of an image as data+metadata. The data is stored as objects where the image can span multiple objects. The metadata maybe stored in the form of a manifest-file, or, as with Glance, is stored as an entry in the registry. As has been discussed on the ML, there is a need for a container format (OVF?), which can encapsulate both the data and metadata if we want to be able to shuffle around fully contained images. As it stands with Glance, the data and metadata are separate; so, in order to replicate an image, you need to pull the image's objects and, in a separate step, fetch its metadata. From: openstack-bounces+rick.harris=rackspace@lists.launchpad.net [openstack-bounces+rick.harris=rackspace@lists.launchpad.net] on behalf of Ewan Mellor [ewan.mel...@eu.citrix.com] Sent: Monday, January 17, 2011 12:12 PM To: 'Jay Pipes' Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Deprecating nova-objectstore -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: 17 January 2011 18:08 To: Ewan Mellor Cc: Thierry Carrez; openstack@lists.launchpad.net Subject: Re: [Openstack] Deprecating nova-objectstore On Mon, Jan 17, 2011 at 12:38 PM, Ewan Mellor ewan.mel...@eu.citrix.com wrote: -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: 17 January 2011 17:17 To: Ewan Mellor Cc: Thierry Carrez; openstack@lists.launchpad.net Subject: Re: [Openstack] Deprecating nova-objectstore On Mon, Jan 17, 2011 at 10:53 AM, Ewan Mellor ewan.mel...@eu.citrix.com wrote: We could do it in two steps. You could set up nova-compute - Glance - nova-objectstore (testing purposes only). This would allow us to remove the S3 code from Nova, but people could still use nova- objectstore if they don't want to set up Swift. This is already done in Bexar, as Chris MacGown completed the S3 backend for Glance. What is NOT the same, though, is that people would not be speaking the s3 API as they do now... they would speak the Glance REST-like API instead... Yes, that's what I meant -- make Glance the only thing that speaks S3, and then we can remove the S3 code from Nova in favour of Glance. Plus, objects are a superset of images; Glance only stores images, not all objects... What does this mean? I didn't even know that anyone was distinguishing between an image and an object. All images are objects but not all objects are images. Swift/S3 can be used to store anything, not just images, that's what I meant. :) What's the difference between an image and an object? They're all just blobs of bits as far as I thought. Ewan. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Deprecating nova-objectstore
On Jan 17, 2011, at 11:59 AM, Jay Pipes wrote: On Mon, Jan 17, 2011 at 2:19 PM, Thierry Carrez thie...@openstack.org wrote: Jay Pipes wrote: I think the big difference is that clients would be talking to Glance and its RESTlike/JSON API, not to an S3 API front-end like nova-objectstore. The big question is whether OpenStack wants to support Amazon S3 as an objectstore at all, instead of just Swift, which now can communicate via an S3 API (http://swift.openstack.org/misc.html#module-swift.common.middleware.swift3). OK, my understanding was that euca-upload-bundle is talking directly to an S3-like backend, so if we want to support the EC2 way of registering images we need some S3-compatible server... Precisely. This is why nova-objectstore cannot be removed even if Glance supports S3 as a backend :) For serious deployments I guess we would use Swift S3 frontend. For demo cases, do we need a simpler solution ? And if yes, which one ? Depends on whether supporting euca-upload-bundle is going to be a priority once alternate openstack-API-speaking client tools are completed... :) My guess is that it won't, and nova-objectstore will eventually move into /ext or /plugin for those who will want to continue using euca-upload-bundle directly. I tend to disagree with this. Anyone who is using eucatools is going to want to use euca-upload-bundle. This is something we should continue to support. Not supporting euca-upload-bundle means we don't have a fully compatible EC2 implementation. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
